BitTorrent over UDP: End of the World or just End of the Beginning?

A column in The Register claims, amongst much wailing and gnashing of teeth, that implementation of BitTorrent-over-UDP (dubbed uTP) in the new alpha version of uTorrent, one of the official BitTorrent client applications, will end the Internet as we know it and completely congest the network.  The title is “Bittorrent declares war on VoIP, gamers.”

Considering the fact that BitTorrent, Inc., has, if anything, always gone out of it’s way to avoid declaring war on anybody, this seemed to me a little bit odd.

I’ll admit that it kind of worried me – TCP has traffic congestion management built into the protocol, UDP does not.  When UDP and TCP exist on the same network (for example, when rolling out VoIP on a corporate network), QoS policies are needed to keep UDP from taking up all the bandwidth while TCP meekly  throttles back.   Jim McQuaid has a Whiteboard series video up about it, and called it “Nice Guys Finish Last.”

The reason that UDP is popular is that it’s a lightweight protocol that doesn’t do much handshaking.  It sends the data “that-a-way” and doesn’t particularly care if it makes it.  That makes it perfect for VoIP, gaming, and other Internet protocols where latency is more important than throughput.  TCP, with congestion control and packet confirmation built in, sacrifices latency for accuracy.  A dropped packet in a phone conversation isn’t much to worry about, but a half-second delay is extremely annoying.  On the other hand, a half-second delay in downloading a computer program isn’t much to worry about, but a dropped packet means that the program won’t run. 

So, as a general rule, TCP runs data apps, while UDP runs real-time apps.  Rudimentary QoS policies based on giving UDP packets higher priority may not be perfect, but they can be a good start to improving performance on simpler networks. 

My concern was that putting BitTorrent, a non-latency sensitive application – on UDP would result in it receiving higher priority traffic.  But after speaking to Simon Morris, the Vice President of Product Management at BitTorrent, Inc., I was assured that this wasn’t the case. Morris explained:

[Editor's Note: Some words got a little garbled in the phone conversation I had with Simon Morris, and he sent me an e-mail with clarifications. I've made corrections via strikethroughs. --ed.]

“BitTorrent obviously needs to be a accuracy-sensitive protocol but we believe it needs to also be …MORE sensitive to latency than TCP, not less sensitive.

This is to say that with uTP, we have taken UDP, implemented a layer of reliability and spent a great deal of time implementing a congestion control mechanism that is better than the one used in TCP (better = faster to detect issues, faster to react).

It’s not QoS, but rather a congestion management mechanism implemented at the end-user’s layer 7 [Application]. This will stop uTP from eating up traffic bandwidth reserved for latency sensitive apps like VoIP and gaming. What’s more, the congestion management mechanism isn’t something we implemented as an afterthought – it’s the whole point of uTP.

In short, it seems that rather than using UDP as a way of getting around TCP’s traffic congestion features, the new protocol is rebuilding better traffic congestion features at layer 7, using the lightweight UDP protocol as a simple base.  In short, to mangle a metaphor, they’re re-inventing a better wheel. 

If uTP does congestion control for BitTorrent as an application, this could provide an answer to BitTorrent critics, and ISPs who claim that BT throttling is necessary because it’s “eating up bandwidth.” I asked how uTP implemented congestion control at Layer 7 [Application] rather than Layer 4 [Transport].

Morris: “So basically, what TCP does is that it stops detects congestion only when it detects packet loss and then it throttles back.  Because we control both ends of the transfer, we can actually measure the single-trip time between when the packet is sent and a packet arrives.  (Not round-trip-times but single-trip-times…)  We have essentially, an ability to monitor single trips over the internet across millions and millions of terminals, and we built an algorithm around that to do things like eliminate the discrepancy between the stock clocksettings on different terminals – to identify where there is actual – very fine grain, down to milliseconds – changes in the speed of which packets are arriving….” 

“The way that prioritization policies are set [by network operators] is extremely varied, and so, it’s possible that [uTP-based] BitTorrent traffic will get a higher prioritization, but only in cases where it’s not causing any type of congestion at all… [uTP] will never trample over UDP based latency sensitive traffic, nor TCP-based traffic.  [UDP is] designed to throttle back if there’s anything else on the line. 

“I mean, it’s essentially designed to be – a term that we use internally is – a “scavenger protocol.”  It scavanges and uses bandwidth that is not being used by other applications at all, and it’s designed to throttle back very very quickly in case there is any type of congestion on the line.

“Unfortunately the way that TCP works is – just profoundly broken as a method of control congestion on the Internet.  Especially when there are applications out there that are designed to get the most out of network bandwidth – like BitTorrent.  People have tried to make TCP better, but the problem is that it’s such a huge implementation task to make it happen, because you need to upgrade all of the Web servers and all of the terminals.  Now, the insight here is that in most manycases, we have control of both ends of the [communication].  So we can actually take the right steps in the direction of solving this problem. 

Those concerned about BitTorrent (either classic or newfangled) traffic on their networks might want to check out a solution designed to monitor and track the types of traffic going on the network, including information about what applications are transmitting and receiving what amounts of traffic. 

, , , , ,

7 Responses to BitTorrent over UDP: End of the World or just End of the Beginning?

  1. Richard Bennett December 1, 2008 at 6:20 pm #

    Don’t assume that intentions always translate directly into effects. The effects of moving half the traffic on the Internet from TCP to UDP will be considerable, and go far beyond the state of affairs imagined by the programmers at BitTorrent, Inc. When the other implementations of the BitTorrent protocol, such as Vuze, get their hands on user-definable congestion avoidance parameters, some very interesting things are going to happen, and I would expect a vendor of network equipment to know that.

  2. Patrick Farling December 1, 2008 at 8:25 pm #

    Mr. Bennett, some of us in the world actually see the automatic downshifting of throughput over TCP in the event of packet loss as quite the annoyance – especially those of us with connections that aren’t exactly “top-of-the-line.” The fact is, TCP has never been effectively expanded upon as the size and scale of TCP/IP-based networking has grown. Now, it’s quite impractical to do so.

    Change can be good, and good change often starts with good intentions. So before you say that a popular P2P service has “declared war” on gamers, people using VoIP, and others that rely on UDP, do some research. Get some test figures. Test it yourself, even. But don’t decry the new because it makes use of something not completely understood by the many. You’ve gone and entered the realm of FUD with that column of yours.

  3. J Kupferman December 1, 2008 at 9:10 pm #

    Excellent article, this answered a lot of questions which I had after reading the uTorrent forum post. It seems like the guys over at BitTorrent Inc. are doing this with the best of intentions. Yes, its UDP based, but there are only two real choices in networking and TCP is a terrible choice for their application, thus they are left with UDP.
    Regarding Richards post, yes the effects will be considerable but the potential benefit here is quite large. It does currently account for somewhere between 30-70% of the Internets traffic (depending on who you ask), but using UDP in an intelligent way can create significantly better performance and use less bandwidth that it would with TCP.
    It seems like they have read their literature on protocols like PCP which show that by using a more intelligent congestion control/avoidance scheme the end result can be a significant increase in performance.
    I wrote an article on my blog about why I think this is a GOOD thing:

  4. Don Marti December 2, 2008 at 9:57 am #

    Even if the current code is 100% perfect and more polite than TCP, there are still two situations where the Bennett scenario might matter.
    1. a dedicated “seedbox” that only does Bittorrent traffic doesn’t have to worry about the user’s games or web surfing.
    2. Since no spec or sample code for the utp protocol is being released, people are going to reverse engineer it, and some of them will get it wrong, or come up with implementations that interact with the official one in unpredictable ways.

  5. Richard Bennett December 3, 2008 at 5:24 pm #

    Incidentally, my article did not say uTorrent over UDP would “end the Internet as we know it and completely congest the network.” If anything is alarmist nonsense, it’s that retarded reading of my article.
    The situation in Canada indicates that the uTorrent over UDP does in fact consume more bandwidth than the TCP version does, and this is taken away from all other Internet uses, including VoIP and gaming.
    While nobody should assume that uTorrent is a deliberate net-killer, nobody should ignore the side-effects of any substantial change to Internet congestion controls.
    TCP is seriously defective as a vehicle for P2P, and I’ve been saying that myself for a long time. But we need to tread carefully in this area because of the Internet’s fundamental lack of resiliency.

  6. Zach Dean December 4, 2008 at 6:22 pm #

    bennett, while you may not have said that uTP will end the internet as we know, that is not far off from what your article reads. in fact, you do call it a “net-killer” which implies the end of the internet. and you do talk about how it will cut latency for most users by 75%. so while he may not have used your exact quote, and in the authors defense he didn’t use quotes. So what you call a retarded reading of your article i found to be quite accurate, stating nearly word for word lines from your article. Maybe you need to reread your own article, and realize this is the internet, everyone gets criticized, and your article creates the alarmist response because you said they are declaring war, an alarmist line.

  7. Richard Bennett December 5, 2008 at 7:58 pm #

    I’ve written a follow-up article explaining the deficiencies in the new BitTorrent system slowly and carefully:
    It’s not getting nearly as much attention as the first one, unfortunately.