Community Forums

Home » Vuze Forums » Open Development


Thread: Patch for super-seeding


Thread Locked This thread is locked - replies are not allowed.


This question is answered.

Search Forum Search Forum Back to Thread List Back to Thread List

Permlink Replies: 13 - Pages: 1 - Last Post: Sep 2, 2010 9:40 AM Last Post By: mmore1q3
mmore1q3


Posts: 6,194
Registered: 12/02/07
Patch for super-seeding
Posted: Nov 1, 2009 9:21 AM
 
Click to report abuse...
A year ago there was some discussion about the weakness of the current superseeding algorithm in Vuze. The implementation is 4-5 years old, and thus relatively old compared to other codebase. http://forum.vuze.com/message.jspa?messageID=177745#177745

No (major) development on the superseeding code has been done after that, and so I spent a few hours trying to understand what the algorithm actually does.

The main logic, as I understand it:
1) masquerade as an empty peer, and go through pieces in numeric order and announce/assign them individually to peers one by one. When you reach last piece, go to normal seeding mode. For each piece a status number is maintained: Level 0 = unassigned, level 1 = assigned but incomplete, level 2 = complete (sent to someone), no other levels.
2) for selecting to which peer you upload next, keep track of how quickly the peers redistribute the piece they got from us (how long does it take for the piece to appear at a third peer?). for this purpose, when someone tells that he has a new piece (BT_HAVE or bitfield), check the tracking status of that piece. The responsible peer gets his "upload speed info" updated, when then piece appear somewhere. (The "upload speed info" is just time difference between end original sending of piece data and time of new observation.)
3) when there is time to announce/assign a piece, give it to the speediest free peer.

I tested with several torrents, where I was properly the only original seed, and evaluated the results.

Some observations and minor patches:
Both patches include some additional commenting in order to make the logic more understandable in future. The patches can be applied separately, as they touch separate parts of the logic.

1)
The first part is a rather simple algorithm, and offers rudimentary run-through of the pieces. However, the main problem is that there is no sophisticated housekeeping of the swarm status. If an early peer leaves/quits, he takes his "level 2" pieces with him, and others in the swarm may not yet have them. There is logic for returning the "assigned but incomplete" level 1 pieces to the pool (back to level 0) when a peer leaves, but as the pool is only gone through once from 0 to max piece, this gives no actual benefit.

I created a patch for PEPeerControlImpl.java, which uses a local boolean variable to enable the re-run the pool from 0 to max.piece, when the end is reached for the first time. This enables small improvement to the logic, as those pieces which are known to be free/unallocated, get a second try. (Local variable is enough, as the goal is just to check if there are any level 0 pieces left when we reach the end. If there are, the counter gets set to the first found 0-level piece, and continues from there through pieces again.)

More full fix would be to evaluate availability of all pieces, when the last piece is reached (for the first time?). If peers have left the swarm and there are thus unavailable pieces although their level is 2, we would reallocate them to new peers on the second run.

2)
The idea to monitor peers' upload speed and use it for queue decision is also ok, but as far as I understand the source code, the implementation is faulty... Peer's upload/distribution speed info gets calculated each time when the piece appears somewhere. There is no status for "piece already seen as redistributed to a third-party". Thus each time when the piece appears somewhere, the original target peer gets his speed info punished by setting the speed (or actually slowness) to the slower value of current observation - original time, as if this was the first re-distribution of the piece. In practice this means, that the peers' speed info gets jumpy and does not reliably tell about the peer's recent upload speed of new pieces given to him.

I created a patch for SuperSeedPiece.java, which creates a third status level = 3 to mark the piece as known to be redistributed. When the piece has been once seen at a third party, the original peer has thus redistributed the piece, and there is no need to pay attention for the piece in future. This stabilised the speed values of the good and quick peers by removing the punishment for them caused by repeated observations of the same piece getting distributed in the swarm.
Cea

Posts: 1
Registered: 11/23/09
Re: Patch for super-seeding
Posted: Nov 23, 2009 11:19 AM   in response to: mmore1q3 in response to: mmore1q3
 
Click to report abuse...
Hello;
How does one install this patch? The only program it opens with is notepad. However, I haven't any idea on what to do with the text. lol
Your help would be much appreciated.
ps: It seems my downloads 'always download at a B/s speed while uploading is at Kb/s. hmmmm

thankyou kindly
Cea

Edited by: Cea on Nov 23, 2009 11:24 AM
mmore1q3


Posts: 6,194
Registered: 12/02/07
Re: Patch for super-seeding
Posted: Nov 23, 2009 12:35 PM   in response to: Cea in response to: Cea
 
Click to report abuse...
How does one install this patch?
The patch can be installed to Vuze's java source code with proper tools, and then you can compile your own new Vuze version with the modifications included. Yes, the patch is a text file, and just describes the small changes proposed to be made to the source coude.

(The 'patches' forum is meant for programmers, not for casual basic users. It is for providing new ideas to Vuze programmers.)
mmore1q3


Posts: 6,194
Registered: 12/02/07
Re: Patch for super-seeding
Posted: Apr 25, 2010 2:25 AM   in response to: mmore1q3 in response to: mmore1q3
 
Click to report abuse...
I converted the proposed patch to SVN format, in case there is any interest for it.
nolar

Posts: 1,500
Registered: 11/13/07
Re: Patch for super-seeding
Posted: Apr 27, 2010 11:13 AM   in response to: mmore1q3 in response to: mmore1q3
Helpful
Click to report abuse...
Considering the lameness of the current super-seeding code, I wonder if we shouldn't just have your stuff committed in a parallel branch, and if things look good, we can merge it in later.
mmore1q3


Posts: 6,194
Registered: 12/02/07
Re: Patch for super-seeding
Posted: Apr 27, 2010 11:33 PM   in response to: nolar in response to: nolar
 
Click to report abuse...
The current super-seeding implementation is really lame and has a logical error in monitoring the further distribution of the pieces (that is the latter part of my patch there). I tried to explain the faulty logic of the current implementation in the original message, so hopefully both the problem and the solution are clear.

After bitching about the problems for a year in the referenced forum thread, and seeing no development work being done, I wrote the simple patch myself. I compiled it half a year ago and tested in real life with several torrents. Both parts of the patch improve things, but especially the latter part about a better piece status monitoring is the essential part, as the current implementation has logical bug there.

You might check the logic in my patch, and check it in to SVN.

Edited by: mmore1q3 on 14-May-2010 20:39
bump. hopefully you get into cheking it in (to main SVN...).
tyux

Posts: 9
Registered: 07/04/10
Re: Patch for super-seeding
Posted: Jul 5, 2010 12:46 AM   in response to: mmore1q3 in response to: mmore1q3
 
Click to report abuse...
Has this been fixed in 4.4.0.4 or does it still need patching?
mmore1q3


Posts: 6,194
Registered: 12/02/07
Re: Patch for super-seeding
Posted: Jul 6, 2010 8:37 AM   in response to: tyux in response to: tyux
 
Click to report abuse...
To my knowledge, nothing regarding the superseeding logic has been patched yet. Nolar voiced some positive comments in this thread a few months ago, but I think that the code has not been checked in.

Edited by: mmore1q3 on 06-Jul-2010 18:38
tyux

Posts: 9
Registered: 07/04/10
Re: Patch for super-seeding
Posted: Jul 7, 2010 8:29 PM   in response to: mmore1q3 in response to: mmore1q3
 
Click to report abuse...
Thanks. I ask because I have noticed some odd behavior while super-seeding torrents, especially new ones. Often, when I add a new torrent and it begins super-seeing (I set Vuze to do this for all torrents), peers appear on the trackers, but none connect (or they connect and no data gets transfered). While torrents that have been in the list a while (and have many more seeds than peers) super-seed instantly. I am unsure if Vuze is responsible, it could be the tracker or the peer's client (maybe my isp?). Any thoughts?

P.S. Sorry, this is kinda off topic for this thread.
Washii

Posts: 5,261
Registered: 11/14/07
Re: Patch for super-seeding
Posted: Jul 11, 2010 9:57 PM   in response to: tyux in response to: tyux
 
Click to report abuse...
Super-seeding should not be happening if there are 'many more seeds than peers;' it's really only useful for single seed situations, where you are the single seed.
mmore1q3


Posts: 6,194
Registered: 12/02/07
Re: Patch for super-seeding
Posted: Sep 1, 2010 10:02 AM   in response to: mmore1q3 in response to: mmore1q3
 
Click to report abuse...
bump...
nolar

Posts: 1,500
Registered: 11/13/07
Re: Patch for super-seeding
Posted: Sep 1, 2010 2:48 PM   in response to: mmore1q3 in response to: mmore1q3
 
Click to report abuse...
That patch still drop in for 4505 line?
nolar

Posts: 1,500
Registered: 11/13/07
Re: Patch for super-seeding
Posted: Sep 1, 2010 6:22 PM   in response to: nolar in response to: nolar
Correct
Click to report abuse...
Merged. Will be in Vuze_4505-02. Please check things to make sure I didn't bork it :)
mmore1q3


Posts: 6,194
Registered: 12/02/07
Re: Patch for super-seeding
Posted: Sep 2, 2010 9:40 AM   in response to: nolar in response to: nolar
 
Click to report abuse...
I tested in an Eclipse build (with debug print lines activated) and it seemed to work just fine. Thanks.
Legend
Master: 800 - 9999 pts
Expert: 400 - 799 pts
Advanced: 200 - 399 pts
Intermediate: 100 - 199 pts
Beginner: 50 - 99 pts
Newbie: 0 - 49 pts
Vuze Staff Member
Vuze Community Moderator
Helpful Answer (3 pts)
Correct Answer (5 pts)

Point your RSS reader here for a feed of the latest messages in all forums