BEP:23
Title:Tracker Returns Compact Peer Lists
Version: 5e240cfc7d64fd67b671ed2408c82e71f434546d
Last-Modified:Thu Sep 25 20:35:10 2008 +0000
Author: David Harrison <dave@bittorrent.com>
Status: Accepted
Type:Standards track
Content-Type:text/x-rst
Created:12-May-2008
Post-History:

Abstract

To reduce the size of tracker responses and to reduce memory and computational requirements in trackers, trackers may return peers as a packed string rather than as a bencoded list.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in IETF RFC 2119 [3].

Overview

In BitTorrent as described in BEP 3 [1], peers wishing to transfer a file contact a central tracker. This tracker returns a list of peers that are currently transferring the file. The list of peers is implemented as a list of bencoded dicts. Each dict in the list contains three fields: peer id, ip, and port. The peer id is 20 bytes plus 3 bytes bencoding overhead. The ip is a string containing a domain name or an IP address, and an integer port number. The ip is variable length, but since in its longest form it is a domain name it cannot exceed 255 bytes [2] plus 4 bytes bencoding overhead. Bencoded integers are also variable length but since it is a port number, it cannot be more than 7 bytes including bencoding overhead. Thus,

total peer list length in bytes < n * ( 23 + 259 + 7 )

It is common now to use a compact format where each peer is represented using only 6 bytes. The first 4 bytes contain the 32-bit ipv4 address. The remaining two bytes contain the port number. Both address and port use network-byte order.

It is SUGGESTED that trackers return compact format by default. By including compact=0 in the announce URL, the client advises the tracker that is prefers the original format described in [1], and analogously compact=1 advises the tracker that the client prefers compact format. However the compact key-value pair is only advisory: the tracker MAY return using either format. compact is advisory so that trackers may support only the compact format. However, clients MUST continue to support both.

For example,

GET /announce?peer_id=aaaaaaaaaaaaaaaaaaaa&info_hash=aaaaaaaaaaaaaaaaaaaa
&port=6881&left=0&downloaded=100&uploaded=0&compact=1

The compact format uses the same peers key in the bencoded tracker response, but the value is a bencoded string rather than a bencoded list.

The peer id does not appear in the compact format. The format has been in use for years and the lack of a peer id has posed no problems.

This compact format is supported by BitTorrent mainline, Azureus, libtorrent, uTorrent, and probably most other clients.

References

[1](1, 2) BEP_0003. The BitTorrent Protocol Specification. Cohen. (http://www.bittorrent.org/beps/bep_0003.html)
[2]RFC-1034. DOMAIN NAMES - CONCEPTS AND FACILITIES. Mockapetris, November 1987. (http://tools.ietf.org/html/rfc1034)
[3]RFC-2119. (http://www.ietf.org/rfc/rfc2119.txt)