Search This Blog

2015-07-21

Limiting Bandwidth Using Robocopy

Robocopy does not have any built in bandwidth limiter by rate specifically.

But what you can do is use the inner packet gap switch /ipg:n to simulate a lower bandwidth by adding more delay between packets.

The equation to use to calculate is as follows:

Where
D is the calculated latency or inner packet gap
Bd is the desired bandwidth you would like to transfer at
Ba is the available bandwidth to you

Example:
If my
Bd is 200kbps
Ba is 1000kbps

D=((1000-200)/(1000*200))*512*1000=2048


Therefore my /ipg:n will be /ipg:2048

References:
https://yellowtriangle.wordpress.com/2012/06/28/bandwidth-throttling-with-robocopy/
http://www.zeda.nl/index.php/en/copy-files-on-slow-links
https://technet.microsoft.com/en-us/library/cc733145.aspx

6 comments:

  1. I have seen this formula elsewhere and I think it is bogus.

    The robocopy /IPG parameter specifies the inter-packet gap in milliseconds. The formula above inserts a full 2.048 seconds between each packet. In order to determine mathematically what the bandwidth utilization of a transfer that sends a single packet every 2 seconds is we would need to know what the packet size (MTU) is. To use the most common scenario let's use 1500 bytes, or (8*1500)=12,000 bits. If we are sending a single packet every 2 seconds then we are transferring at a rate of 12,000/2 or 6Kbps, not 200Kbps. And the fact that MTU is not even a factor in the formula tells me that it can't possibly be valid.

    ReplyDelete
  2. Actually the formula is correct because the value 512 in the above formula indicates a 64KB packet which is the size that robocopy uses. This was not stated in the explanation, but it is accounted for.

    ReplyDelete
  3. Robocopy cannot generate a 64k packet because the normal MTU is 1500 bytes. Somehow I don't think a 64kB length packet will traverse an interface whose data path is limited to 1500 bytes without fragmentation.

    ReplyDelete
  4. This comment has been removed by the author.

    ReplyDelete
  5. While I don't see it in any official Microsoft documentation, many other sources seem to agree with a 64kb packet. If you knew how, you could of course work out from Robocopy itself what it's doing. And it's relatively trivial to test whether the IPG gives the expected result assuming the packet size is indeed 64kb.

    I don't really know why Robocopy uses 64kb packets. I have read that on XP the CopyFileEx API also used 64 kb blocks for copying locally. 60kb for copying over networks, well SMB. This was related to I/O limitations which were changed in Vista https://blogs.technet.microsoft.com/markrussinovich/2008/02/04/inside-vista-sp1-file-copy-improvements/ . So perhaps it's related to that.

    About the MTU issue, I think you're missing the point. There's no suggestion that these are discreet network packets. Per my earlier comment, perhaps it'll help if you substitute the word 'block' or something else for 'packet'. AFAIK, Robocopy doesn't even have any real built in support for networks. It relies entirely on the OS e.g. SMB to deal with the network transmission, just the same as it does when you're not using networks. Precisely what blocks or packets are being used for the data transmission at the network layer or interface layer is not something that Robocopy likely cares or maybe even knows about. And I presume SMB has some overhead too, so it's not like every single network packet is always just the copied data (and other data required by the network protocol itself).

    ReplyDelete