CFexpress for Less?
What's this now?
Imagine my surprise when I learned that CFexpress cards actually use NVMe at their core. It turns out CFexpress Type B uses two PCIe Gen3 lanes, with a maximum duplex speed of 2 GB/s. Conveniently, the Type B form factor is big enough that a 2230 M.2 drive can fit within it.
As a result, the "DIY CFexpress" adapters use a PCB that has no active components to effectively translate one connector to another connector. I really like that these adapters have no active components, because, it seems to me, that should limit the potential for variability between sellers/products (barring signal quality issues, but the traces are quite short). Also, as compared to a normal CFexpress card, these adapters include an outer case that is entirely CNC'd aluminum. I am hopeful that will to dissipate heat better than the normal plastic/metal mix.
Pictured above are two adapters from Sintech, which are also linked here for convenience (raw aluminum | anodized black). Surprisingly, buying from AliExpress doesn't seem to offer much in the way of savings (at least that I could see).
That said, going the DIY route has the potential to offer substantial cost savings. Call it $45 for an adapter and $160 for a 1 TB 2230 M.2 drive (or even apparently $170 for a 2 TB drive), and you have a capac-eous (pronounced kuh-PAY-shus) CFexpress card for ~$150-$750 less higher-end CFexpress card options. As an aside, that 2 TB price seems shockingly low to me; I paid closer ot $400 for the 2 TB drive that I had handy, but that was a long time ago and flash prices are rumored to continue plummeting this year.
Having put together two DIY cards using the above-linked drives (the raw aluminum is the 1 TB Sabrent 2230 and the anodized black is the 2 TB Micron 2400), I can say that my initial impression was very favorable. Yes, you give up two PCIe lanes and it's only Gen3, but cameras don't really need that much bandwidth (I mean, beyond the already generous 2 GB/s bandwidth contemplated by the Type B specification).
So, where's the catch?
Well, physically speaking, it is a tight fit. I think I remember the documentation mentioning not to use too much thermal paste (yes, paste; what a mess), otherwise it could negatively impact the thickness of the card and cause fitment issues. I had initially used a graphite sheet and I also tried a 0.5 mm thermal pad, and both definitely made it difficult to pop the card in and out of the camera. Beyond that, I'm pretty happy with the design and the included stickers are a nice thought (though there is no 2 TB sticker!).
Technologically speaking, I think there is a bit more of a catch. And that brings us to the motivation for this blog post. After the realization that CFexpress cards are really just NVMe drives, I sicced smartctl on my 256 GB SanDisk card to see what was reported back:
=== START OF INFORMATION SECTION ===
Model Number: WDC SanDisk CFexpress B SDCFE-256G NVMFW
Serial Number: --
Firmware Version: RO106CFP
PCI Vendor/Subsystem ID: 0x15b7
IEEE OUI Identifier: 0x001b44
Total NVM Capacity: 256,060,514,304 [256 GB]
Unallocated NVM Capacity: 0
Controller ID: 1
NVMe Version: 1.3
Number of Namespaces: 1
Local Time is: Sun Apr 30 16:38:47 2023
Firmware Updates (0x14): 2 Slots, no Reset required
Optional Admin Commands (0x0016): Format Frmw_DL Self_Test
Optional NVM Commands (0x001f): Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Log Page Attributes (0x02): Cmd_Eff_Lg
Maximum Data Transfer Size: 8192 Pages
Warning Comp. Temp. Threshold: 80 Celsius
Critical Comp. Temp. Threshold: 90 Celsius
Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 2.50W - - 0 0 0 0 0 0
1 + 2.50W - - 1 1 1 1 0 0
2 + 1.70W - - 2 2 2 2 0 0
3 - 0.0250W - - 3 3 3 3 5000 9000
4 - 0.0025W - - 4 4 4 4 5000 44000
=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
SMART/Health Information (NVMe Log 0x02)
Critical Warning: 0x00
Temperature: 31 Celsius
Available Spare: 100%
Available Spare Threshold: 10%
Percentage Used: 0%
Data Units Read: 702,309 [359 GB]
Data Units Written: 622,354 [318 GB]
Host Read Commands: 4,701,192
Host Write Commands: 630,513
Controller Busy Time: 40
Power Cycles: 5,015
Power On Hours: 61
Unsafe Shutdowns: 43
Media and Data Integrity Errors: 0
Error Information Log Entries: 0
Warning Comp. Temperature Time: 0
Critical Comp. Temperature Time: 0
Thermal Temp. 1 Transition Count: 15
Thermal Temp. 1 Total Time: 2554Interesting. But mainly for reasons that will become apparent later.
Now, I didn't actually look at the SanDisk CFexpress card first. I had only looked at the SanDisk card after evaluating the performance of the DIY cards. In testing, the 1 TB card appeared to perform better than the 2 TB card. The 1 TB Sabrent M.2 drive is TLC, while the 2 TB Micron M.2 drive is QLC. There actually isn't much discussion of DIY CFexpress cards that I could find, but I did see some suggestion that TLC flash would fare better than QLC flash, so perhaps that result wasn't surprising.
Don't get me wrong, I could write unlimited burst RAW to both DIY cards, and 4k30 wasn't a problem either. The difference was only apparent when trying to record 4k60. The 1 TB card seemingly had no problems writing "up to 360 Mbps," per the camera manual, while the 2 TB card would choke. It didn't really make sense though, because, at least in testing using a CFexpress reader, both cards were more than capable.
I'd also seen something on Anglebird's website that I thought was strange. Angelbird notes that the 4 TB card is only compatible with the Nikon Z9 and a RED 8K camera:
I have nothing to corroborate what follows, because I'm not going to spend $1.8k on a CFexpress card (and I haven't reached out to Angelbird), but I'm wondering if it isn't because of power consumption. I'll note that other (more?) plausible explanations include, but are not limited to, firmware limitations (hellooooo FAT) or even just a slight increase in card thickness.
But, before I continue blathering, let's take a look at the smartctl output for the other cards.
For the 1 TB Sabrent card:
=== START OF INFORMATION SECTION ===
Model Number: Sabrent SB-2130-1TB
Serial Number: --
Firmware Version: R21B47.1
PCI Vendor/Subsystem ID: 0x1987
IEEE OUI Identifier: 0x6479a7
Total NVM Capacity: 1,024,209,543,168 [1.02 TB]
Unallocated NVM Capacity: 0
Controller ID: 0
NVMe Version: 1.4
Number of Namespaces: 1
Local Time is: Sun Apr 30 14:47:39 2023 MDT
Firmware Updates (0x14): 2 Slots, no Reset required
Optional Admin Commands (0x0017): Security Format Frmw_DL Self_Test
Optional NVM Commands (0x005f): Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat Timestmp
Log Page Attributes (0x1e): Cmd_Eff_Lg Ext_Get_Lg Telmtry_Lg Pers_Ev_Lg
Maximum Data Transfer Size: 64 Pages
Warning Comp. Temp. Threshold: 83 Celsius
Critical Comp. Temp. Threshold: 85 Celsius
Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 5.00W - - 0 0 0 0 0 0
1 + 2.40W - - 1 1 1 1 0 0
2 + 1.92W - - 2 2 2 2 0 0
3 - 0.0700W - - 3 3 3 3 5000 10000
4 - 0.0050W - - 4 4 4 4 6000 44000And for the 2 TB Micron card:
=== START OF INFORMATION SECTION ===
Model Number: Micron_2400_EEFDKBK2T0QFM
Serial Number: --
Firmware Version: 23A24013
PCI Vendor/Subsystem ID: 0x1344
IEEE OUI Identifier: 0x00a075
Controller ID: 0
NVMe Version: 1.4
Number of Namespaces: 1
Local Time is: Sun Apr 30 14:53:28 2023 MDT
Firmware Updates (0x14): 2 Slots, no Reset required
Optional Admin Commands (0x0017): Security Format Frmw_DL Self_Test
Optional NVM Commands (0x005f): Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat Timestmp
Log Page Attributes (0x1e): Cmd_Eff_Lg Ext_Get_Lg Telmtry_Lg Pers_Ev_Lg
Maximum Data Transfer Size: 32 Pages
Warning Comp. Temp. Threshold: 87 Celsius
Critical Comp. Temp. Threshold: 90 Celsius
Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 9.00W - - 0 0 0 0 0 0
1 + 4.60W - - 1 1 1 1 0 0
2 + 3.80W - - 2 2 2 2 0 0
3 - 0.0250W - - 3 3 3 3 5000 3000
4 - 0.0040W - - 3 3 3 3 8000 35000Do you see Watt I mean?
And finally, my theory:
I'm wondering if CFexpress cards are as expensive as they are because their firmware is specially tuned to provide consistent/minimum write speeds within a certain power budget. Take a look at the maximum Power State that was reported for each of the three cards:
- 2.5 W for the 256 GB SanDisk card
- 5.0 W for the 1 TB Sabrent card (+100% compared to SanDisk)
- 9.0 W for the 2 TB Micron card (+260% compared to SanDisk)
Interestingly, Angelbird lists the maximum power consumption as 2.4 W for the whole line of MK2 cards, but maybe the website is wrong and the 4 TB card is different? Who knows. It's literally unknowable.
But maybe, just maybe, my camera did not like supplying NINE watts to the Micron card. I actually hadn't paid much attention to NVMe Power States in the past, but it turns out that they are configurable!
Throttle Thyself
In researching NVMe Power States, I found a blog post where one enterprising individual was trying to achieve the opposite goal: they had a drive that was exhibiting reduced performance due to Power State limitations and they were trying to UNLEASH THE DRIVE.
Armed with the Power States returned by smartctl and the knowledge I'd gleaned from the blog post, I used the command below to set a different, lower maximum power state:
$ sudo nvme set-feature /dev/nvme1 -f 2 -v 2 -sTo break it down: nvme is in the nvme-cli package, set-feature sets a specified NVMe feature (i.e., 2 for Power Management, as specified by -f). The value -v is specified to be 2, which corresponds to Power State #2 (3.8 W) as reported by smartctl. I'll also note that -s is needed to ensure the change persists across resets (which was unfortunately omitted from the blog post I found). Finally, one can confirm that the change was successful using the command below, which should thus indicate Power State #2 is set rather than Power State #0.
$ sudo nvme get-feature /dev/nvme1 -f 2 -HIn testing the card's speed using a CFexpress card reader, the card definitely does exhibit reduced performance (now writing closer to 400 MB/s), but hopefully it also consumes less power and generates less heat. When tested in the camera, Power State #1 (4.6 W, which is closer to the Sabrent card) enabled longer 4k60 recording, though it still ultimately failed. I eventually settled on Power State #2, since all other functionality remained unchanged at a lower maximum power drawn and 4k60 still failed.
Closing Thoughts
Overall, I feel good about the 2 TB Micron card and hopefully having changed the maximum power consumption from 9.0 W to 3.8 W. I say "hopefully", becuase I wonder if the camera might still set its own Power State (in which case real CFexpress cards would still be desirable for their more specialized firmware and Power States). I suppose this could be established through testing, and I will update this post if I do so.
I may also try to change the maximum Power State for the 1 TB Sabrent card (e.g., from 5.0 W to 2.4 W) to see how that impacts performance, given Power State #1 is the same as the stated power draw for the Angelbird cards and slightly below the SanDisk card that I have. Again, I will update this if I end up testing that.
Other Notes
Unfortunately, nvme-cli doesn't know how to work with NVMe drives via USB, which is most CFexpress card readers. I have a Thunderbolt CFexpress card reader that I use on my Apple Silicon Mac (which properly reports the cards as NVMe devices), but nvme-cli does not compile on macOS. Super.
So what did I do? Well, perhaps obviously, you can plop the 2230 M.2 drive in a normal M.2 slot, which is what I did using one of my Linux machines. It was very slow and inconvenient, as I had to move the M.2 drive between the Linux box and the CFexpress card adapter (not to mention the thermal paste mess that ensued).
To the extent I engage in future testing, I've ordered a CFexpress to PCIe adapter to enable me to use nvme-cli with the DIY CFexpress cards, which should alleviate some of that friction. (And, for those keeping track, will mean I'm going from 2230 M.2 => CFexpress adapter => PCIe slot, which just makes sense.)
2023.05.08 Update: The CFexpress to PCIe adapter didn't work... I'll have to see if there's another option.
2023.05.17 Update: I ordered Sabrent's new 2 TB CFexpress card. Output of smartctl is below:
=== START OF INFORMATION SECTION ===
Model Number: Sabrent Rocket CFX Pro
Serial Number: --
Firmware Version: XXIB47.1
PCI Vendor/Subsystem ID: 0x1987
IEEE OUI Identifier: 0x6479a7
Total NVM Capacity: 2,048,408,248,320 [2.04 TB]
Unallocated NVM Capacity: 0
Controller ID: 1
NVMe Version: 1.4
Number of Namespaces: 1
Local Time is: Wed May 17 13:49:04 2023 MDT
Firmware Updates (0x14): 2 Slots, no Reset required
Optional Admin Commands (0x0017): Security Format Frmw_DL Self_Test
Optional NVM Commands (0x005f): Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat Timestmp
Log Page Attributes (0x1e): Cmd_Eff_Lg Ext_Get_Lg Telmtry_Lg Pers_Ev_Lg
Maximum Data Transfer Size: 8192 Pages
Warning Comp. Temp. Threshold: 83 Celsius
Critical Comp. Temp. Threshold: 85 Celsius
Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 5.00W - - 0 0 0 0 0 0
1 + 2.40W - - 1 1 1 1 0 0
2 + 1.92W - - 2 2 2 2 0 0
3 - 0.0300W - - 3 3 3 3 5000 10000
4 - 0.0050W - - 4 4 4 4 6000 44000I'll note this is largely similar to the 1 TB 2230 NVMe drive, including a max power consumption of 5 W. I may try to set a max Power State of 2.4 W if I can find a good CFexpress/PCIe solution and see what happens. Also, makes me want to get a card that is actually advertised at 2.4 W, but is reduced power consumption worth spending 2x more?? ...probably. I think first I will need to confirm whether these cards actually exhibit increased power consumption.
Interestingly, "Maximum Data Transfer Size" for the CFexpress card is 8,192 pages, which is consistent with the SanDisk card and substantially more than either M.2 NVMe drive (32 or 64 pages).
My other thought is my DIY 1 TB Sabrent CFexpress card is probably actually pretty close to the actual 1 TB Sabrent CFexpress card, for probably nearly $250 less. Heckin' deal.