Good values for piece length?

[Jack Bates]

Can anyone please point me at any best practices for choosing the length of the Metalink <pieces/> element? I looked at RFC 5854 and the discussion group archives, and while they make quite clear how to interpret the length attribute of the <pieces/> element, I found no discussion yet on how to choose good values. Is there any "conventional wisdom" in the Metalink community on choosing good values? Can anyone please offer me any advice?

It seems that you would want to choose a larger value for larger files, to keep the number of </hash> descendant elements from exploding. At the same time, too large a value would make it less efficient for clients to isolate errors

I found many resources that say that the length of pieces in torrent files is almost always a power of two, but I found no explanation of why that is. Is there a good reason why the length of BitTorrent pieces is usually a power of two? Is the length of Metalink <pieces/> also usually a power of two?

Thanks for any advice!

[Neil M.]

The best I can come up with is the Metalink Editor source code which
uses 256k sized chunks unless that would produce more than 255 chunks,
then it limits the number of chunks to 255. I adopted this for Metalink
Checker as well.

As for the power of two I'm not sure, perhaps that just optimizes the
hash calculation.

Neil

> --
> You received this message because you are subscribed to the Google
> Groups "Metalink Discussion" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/metalink-discussion/-/O-k4tlLtSYIJ.
> To post to this group, send email to metalink-...@googlegroups.com.
> To unsubscribe from this group, send email to
> metalink-discus...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/metalink-discussion?hl=en.

[Bram Neijt]

I can recall there was code like that: double the size of the piece till the number of pieces is below some kind of threshold. Can't remember the threshold, but 255 seems ok. Maybe bittorrent uses that as well, which would explain why they always have an even number of pieces.

Greets,

Bram

[Tatsuhiro Tsujikawa]

There seems to be some guidelines to choose piece length in BitTorrent.

Here is the quote from http://wiki.theory.org/BitTorrentSpecification

The piece length specifies the nominal piece size, and is usually

a power of 2. The piece size is typically chosen based on the

total amount of file data in the torrent, and is constrained by

the fact that too-large piece sizes cause inefficiency, and

too-small piece sizes cause large .torrent metadata

file. Historically, piece size was chosen to result in a .torrent

file no greater than approx. 50 - 75 kB (presumably to ease the

load on the server hosting the torrent files).  Current

best-practice is to keep the piece size to 512KB or less, for

torrents around 8-10GB, even if that results in a larger .torrent

file. This results in a more efficient swarm for sharing

files. The most common sizes are 256 kB, 512 kB, and 1 MB.  Every

piece is of equal length except for the final piece, which is

irregular. The number of pieces is thus determined by 'ceil(

total length / piece size )'.  For the purposes of piece

boundaries in the multi-file case, consider the file data as one

long continuous stream, composed of the concatenation of each

file in the order listed in the files list. The number of pieces

and their boundaries are then determined in the same manner as

the case of a single file. Pieces may overlap file boundaries.

Best regards,

Tatsuhiro Tsujikawa

> metalink-discussion+unsub...@googlegroups.com.

> For more options, visit this group at
> http://groups.google.com/group/metalink-discussion?hl=en.

--
You received this message because you are subscribed to the Google Groups "Metalink Discussion" group.
To post to this group, send email to metalink-...@googlegroups.com.

To unsubscribe from this group, send email to metalink-discussion+unsub...@googlegroups.com.