last day (15 days later) » 

6:46 PM
2
Q: Why am I seeing low transfer speed with Robocopy between two external SSDs?

Tyrone HirtI'm using Robocopy within a PowerShell script to backup my main SSD to another SSD, but I'm having problems with the file transfer speed. While in Windows Explorer I have writing speeds of 300~400 MB/s, in Robocopy the average is 80 MB/s. My command line is this: robocopy "$($source = $drv;$source)"

 
Try it without /B.
 
/j should make it slower, but.. it's worth a shot, check if /j makes a difference
My best guess is that Explorer.exe pre-allocates all space, and robocopy don't, which causes robocopy to be slower (it's complicated but pre-allocation causes more sequential writes with fewer, bigger extents, and less metadata updating..)
you could try /iomaxsize:10M too, worth a shot
 
@hanshenrik This pre-allocation point that you mentioned is interesting, since I have enabled the "Best performance" and "Enable cache recording on the device" option in the unit policies. I don't know if Robocopy takes advantage of this.
I was doing some more tests, and apparently there is something strange with the copy in Windows Explorer too, because the speed is going up and down, it seems something related to pre-allocation. For example, file copying on the 40Gbps Thunderbolt drive starts at rates of 1.5GB/s and within about 10 seconds of copying goes to 0, then goes up again and oscillates, eventually returning to 0.
 
Sounds like typical storage drive data caching issues. The cache doesn’t get filled and it is fast but otherwise if the cache is full, speed degrades.
 
@Giacomo1968 Yes, it really seems a storage drive data cache issue, I added some more information that helps you visualize this.
 
6:46 PM
Look into Host Memory Buffer. As far as I know, HMB does not work over USB, and many SSD drives use this so as to avoid having their own memory buffer. Since you are using an "internal use" SSD with an enclosure, this might be a source for your problem.
 
@Yorik I didn't know that, there really is a lot of information in this IT environment... it seems like a trick, you go through the pot of gold and it always goes further... I bought external SSDs to speed up my backup process and have more reliability, now I discover that they don't work well externally if they don't have their own DRAM, I had no idea about that...
Anyway, I did a little research and it seems that the MP600 Core XT uses HMB and this feature is available using Thunderbolt 4, since Thunderbolt 4 basically works as a PCIe 4.0 interface (I checked CrystalDiskInfo and made sure it is using this mode transfer), even so I still get oscillations when copying files from my internal SSD to this SSD connected to Thunderbolt 4, is that right?
Another question, is there a way to minimize these speed impacts that the cache is causing by using the unit via USB 3.2? Maybe a way to generate a cache in RAM during the copy process?
 
@TyroneHirt The MP600 Core uses something called QLC NAND to store the data. Writing to QLC is slower than writing to other types of NAND, so what the drive controller can do is treat a portion of the storage as SLC NAND, which is faster to write to, but it will only use a relatively small portion in that way. Once that portion is full, the drive controller has to transfer it to the "normal" QLC part, during which time the drive is much slower. However, that does not explain why Explorer copies at more than 3x the speed of robocopy.
 
@AndrewMorton That explains a lot! So I believe that these fluctuations are normal in these SSDs that are more focused on cost-benefit, right? However, is there any way to minimize cache impacts on the SSD that is working on USB 3.2?
 
@TyroneHirt Maybe the drives need to run the TRIM command, which tells them to tidy up their storage so they can run at full speed again. From a quick search, you may be able to use the "Optimize" option in Corsair SSD Toolbox.
@TyroneHirt Or you may be able to right-click on the drive in Explorer, choose "Properties", choose the "Tools" tab, click "Optimise", select one of the external drives, then click the "Optimise" button. I don't know if it can send a TRIM command over USB - it may also depend on the enclosure.
 
@AndrewMorton I will perform the recommended TRIM actions. Thank you very much for the explanations! Can you tell me if there is a way to "emulate" a cache for these Corsair SSDs that don't have a DRAM chip? I was reading something about a software called PrimoCache, but I didn't really understand if it helps in these situations...
 
6:46 PM
@TyroneHirt No, PrimoCache will not help with this, from what I can see. It is intended for HDDs, or perhaps network drives.
 
@AndrewMorton Basically there's not much way around this caching issue, right? Would it be better to replace these SSDs with SSDs with a DRAM chip? Thank you very much for your attention.
 
1NN
Your TOP Speed of 400MB/s is 10x too low for a Corsair MP600 Core. That SSD should SATURATE a 20Gbps connection! See this independent review. So there's something fishy going on. I'd suspect a non-genuine cheap imitation drive.
 
@TyroneHirt The easy way is to have more patience ;) Did doing a TRIM make any difference?
 
@1NN The speed is correct since I am using one of them on a USB 3.2 Gen 1 connection. The MP600 Core XT that is connected to Thunderbolt 4 reaches a maximum speed of 1.7GB/s, it stays at that speed for about 10-15 seconds, then it stays oscillating. Apparently this is normal for a drive without DRAM.
@AndrewMorton TRIM made a bit of a difference. However, having more patience is not an alternative for me, I need agility in my daily life, which is why I opted for NVME SSDs. I'm getting a constant speed of 80MB/s using the SSD this way, which is less than some HDDs can do. I will replace them with models with DRAM if that is the solution to have constant high speeds.
 
@TyroneHirt Another possible reason for the SSD going slow is that it is overheating. Exactly which model of Orico drive enclosure are you using? Does it feel hot to the touch a couple of minutes into the test? If not, it should - maybe there is a thermal interface pad missing.
@TyroneHirt And as 1NN commented, there is a possibility that they are fake drives - did you get them from a reputable supplier, or perhaps somewhere like eBay or AliExpress?
Does CrystalDiskMark give reasonable speeds for the drives?
And once the DRAM capacity is exceeded, you will see the oscillating behaviour again anyway.
 
7:50 PM
@TyroneHirt (Sorry, I may never see your replies as these comments have been moved to "chat".)
 
 
2 hours later…
9:47 PM
@AndrewMorton I'm sure they're not fake drives, I bought them from an authorized dealer here in my country. As for the point about SSD with DRAM, I could be wrong, but as far as I know its function is to be constantly rewritten during the copy process, so it would be very difficult to experience drops in speed due to them being full. But I could be wrong.
Overheating is also not a problem, the temperature of the SSDs never exceeds 52°C, I was monitoring it during the copying process.
 

last day (15 days later) »