Discussion between Yosef Gunsburg and ytti

Stack Exchange

09:34

1

A: Determining optimal bandwidth needs for the office

SNMP is very handy for this type of monitoring. For just one router, you can download the free version of PRTG and set it up to monitor the interface's bandwidth. On your router you are going to need to enable SNMP access. To do so, enter snmp-server community [name of your choice] RO in global-...

ytti

It's very hard with SNMP to see actual peak-rate, as it's averaged data. Optimally we'd see all data send with at least millisecond precision or greater, then we could calculate peak-demand.

Yosef Gunsburg

@ytti yes, but if you aren't hitting your peak consistently over time, then you may not need all the extra bandwidth. If you hit 20Mbps for a few seconds, but average 2Mbps, the peak is not characteristic of your traffic flow and overall link saturation.

ytti

If you hit 20Mbps for few seconds, and you downgrade link to 2Mbps, you'll be dropping frames at those few seconds. And you'll experience high latency in other smaller peaks.

Yosef Gunsburg

@ytti say you have a user downloading a 100MB file. It will peak during the download, but will not drop frames if you scale the link down. With less bandwidth, the L4 protocols will take care of those issues with windowing.

ytti

Only way to scale down the transfer is to drop frames, that is how TCP works :)

Yosef Gunsburg

09:34

It will never scale up in the first place :)

ytti

Only way to avoid it scaling up, is to have maximum TCP window small enough.

Yosef Gunsburg

Right

ytti

per default TCP Window will scale to 64kB

Yosef Gunsburg

And it will quickly scale back

Wait, that's the default?

ytti

yes

Yosef Gunsburg

09:36

ah

ytti

and your bandwidth is then limited to Window/RTT

If your delay is 10ms, that is 64kB/10ms = 51.2Mbps

but usually all operating systems have 'window scaling' turned on

Yosef Gunsburg

What I'm getting at is that if you have a 100Mbps link, and someone decides to download 600MB, the link will be saturated for the file transfer, but that doesn't mean you need a 100M connection

ytti

which will allow TCP Window to scale almost abitrarily

Yosef Gunsburg

64KB/10ms

how'd you get 51.2?

ytti

(64*8)/0.01/1000 = 51.2Mbps

64 we multiply by 8, to get bits

then we divide that by 10ms, i.e. 0.01s

then we have bandwidth in kbps

we divide that by 1000 to get Mbps

anyhow it's of course debatable when you need faster connection and when not

Yosef Gunsburg

09:40

absolutely

and it really depends on the type of traffic

ytti

if you only burst over rate for say 10ms, then you can queue it

but then you'd need to implement QoS

so that when you do drop packets, you're not dropping VoIP, but you're dropping the bulk trasnfer

Yosef Gunsburg

if you see file transfers (user based) taking up the link for a couple seconds here and there

ytti

but only way to get TCP to reduce its window size, is to drop packets, thats absolutely normal and expected way TCP finds congestion

Yosef Gunsburg

vs. sql transactions

etc

ytti

couple seconds is VERY long time

Yosef Gunsburg

09:41

for a file trans?

ytti

if you don't do QoS, all traffic is impacted for the duration

Yosef Gunsburg

hmm

ytti

if you need to drop packets at the link, you need to make sure, you're actually dropping the highspeed TCP transfer

Yosef Gunsburg

good point

ytti

if you congest your link, you need QoS

so if OP wants to downgrade his link to a rate where he starts dropping packets, he also needs to implement QoS

Yosef Gunsburg

09:42

I wouldn't suggest going lower than 10M

:)

ytti

or then he could measure what is actual peak-rate, and downgrade there, where he is not yet dropping anything, then he can live without QoS

ACK, TCP is aggressive

it will take all bandwidth there is available

so if LAN is 100M or 1GE

Yosef Gunsburg

yea. every user download will congest

ytti

but right now OP is claiming there is no congestion (or maybe is looking to see if there is)

Yosef Gunsburg

Oh... Up there you were referring to the BDP?

That always makes me craaazy.

ytti

Yes, rate is determined by window and RTT

RTT * BW = Window

so we can solve BW from that by: Window/RTT = BW

Yosef Gunsburg

09:48

Optimizing TCP is fun...

lol

ytti

or we can solve RTT by Window/BW = RTT

so if we know 2 of those variables, we can calculate the 3rd

Yosef Gunsburg

Ah, I see

ytti

so if you really want to limit TCP from growing, you could force the operating system to use really small tcp windows

Yosef Gunsburg

Many implementations throw the window through the roof anyways

ytti

but that would be manual configuration, as the operating system which is installed (windows7, 8, OSX, linux) all support not only 64kB windows, but window scaling

Yosef Gunsburg

09:50

They start with a scaled window

ytti

yes, all modern OS

so you're not even limited to the 64kB anymore

Yosef Gunsburg

I know a university that does it

they set the multiplier

g

ytti

it's on by default in every modern OS

no need to manually configure

Windows 95 didn't have TCP window scaling turned on by default yet

i'm not sure about Windows XP

but Windows7 definitely has it turned on

Yosef Gunsburg

So that sets the window to a large value?

from the start?

ytti

yes, it's an option you signal at TCP handshake

which is just single integer, telling what number you use to multiply the reported TCP window size

Yosef Gunsburg

09:52

yup. so Win7 for example will set that up

ytti

(so to know actual tcp window size, you need to see the transfer from the start, doing capture in the middle of the transfer, and you don't know what was the window scaling factor)

ack

Yosef Gunsburg

right

ytti

but even 64kB is quite large low latency links, like fibre

as 1ms == 100km, RTT

so you could get >50Mbps on 10ms, i..e 1000km!

Yosef Gunsburg

64KBps

/.01

/1000

?

= 51.2

~ 50Mbps

ytti

yes

well no 64kBps, but 64kB

it's not per second, it's window size

window / rtt = bw

Yosef Gunsburg

09:56

yea

ytti

window == 64kB, i.e. 64*8 == 512kilobits

delay == 10ms, i.e 0.01s

512kilobits / 0.01s = 51200kbps

51200kbps / 1000 = 51.2Mbps

Yosef Gunsburg

Thanks!

ytti

np

Yosef Gunsburg

(Math escapes me :))

ytti

it's one division :)

the math isn't the problem to most, it's the unit conversions that seem to be problematic to people

especially to people from US, don't always seem to be familiar with SI units

Yosef Gunsburg

09:58

I remember trying to figure out BDP and RTT and window scaling TCP extensions...

Drove me in circles

High speed satellite link, high latency

Long fats

ytti

yeah satellite links can easily have more than 600ms latency

so you need really large window to get decent bandwidth

as RTT*BW = Window

if we wan to run 50Mbps on 600ms satellite link

it's 0.6s * (50/8) = 3.75MB

over 60 times higher window that you can even get without window scaling option!

Yosef Gunsburg

whoa

yea, the conversion throws me off every time

Mb MB Kb KB seconds, msec

you'd think I'd get the hang eventually ;)

ytti

geostat orbit is about 36000km, we need to multiple that by 4 (we go to the orbit, then come back, then go to the orbit, then come back). So we get 144000km which we divide by c (speed of light) and we get 480ms

that is absolutely minimum delay you could ever hope to get for geostat orbit

Transcript for

Discussion between Yosef Gunsburg and…