How to bypass "Force Recheck" when no issue with the data files?

[avelev]

Hi,

I’m using Vuze 5.5.0.0

Is there a way to bypass “Force Recheck” when I know there is no issue with the data files?

I recently migrated several terabytes of torrent data files I’m seeding to a folder on a local NAS and set that folder as a network drive. Great content. Sometimes the connection breaks and the NAS becomes unavailable even if just for a second. Vuze, however, continues to think it has lost the files and shows an error “File not found”. Force Recheck fixes it. However, it takes days to recheck several terabytes of data.

I had the same problem before but much less often when the torrents were on a storage drive right on the PC. Sometimes (rarely), after a reboot, the PC would not see that storage drive and of course in Vuze all of the torrents showed error “File not found”. The fix was the same -- get the drive to be detected by the OS (Win 7) and then force recheck all the torrents. Very annoying but I did not have to do it so often.

The connection issue with the NAS happens a lot more frequently so the massive force rechecking is becoming prohibitive.

There are many different causes for a network drive to become momentarily not available. A common one – power failure is easily fixed with a UPS battery, but there are many others It’s the nature of the beast.

Even if “Force Recheck” could finish fast it’s still a problem especially on files I’m seeding which I just do not monitor actively and I may not be aware of the problem for a long time thus depriving many users out there of this great content. Please help me make the system more robust by:

1. bring it online without a force recheck.
2. When NAS connection reestablished, automatically recognize that the files ARE there and bypass the force recheck.  

Thanks for the help.

[ekstasee]

Moving Files:  
I suggest to upgrade to the latest version 5.6.1.2.  When you right click on a torrent(s) the newly revised context menu will open which has 4 headers with options listed under each header. Under the "Content" header, you will find an option "Move Data Files".  Left-clicking this option will launch a "browse computer" box so that you can select the destination to send the data-files to.  Vuze will do all the work of moving the file and updating the Vuze index to keep track of the file and torrent relationship.  Just ensure that you stop the torrent before moving the file.   I use 4 HDD's for torrents exclusively with one drive being the dafault for all new torrents. When the torrent is completed, I stop the torrent(s) [you can do as many to any single destination as you wish] and move the data file(s) to one of the other hdds.  My drives are assigned different groupings of the alphabet, but obviously, you can arrange your drives as best suits your needs.
NOTE: you can move the data file at any time. It does not have to be finished. It can still be in "leech" mode.  Just ensure that you stop the torrent and then restart it after the file move has  finished.

FORCE RECHECK:
Although I do not recommend it, you can disable the file recheck in OPTIONS > FILES and uncheck the box for "On crash-restart check the entire file for completed pieces"
A manual "Force-Recheck" can still be done at any time.

[parg]

Next beta (B15) will include an 'advanced' option in the torrent context menu to allow you to bypass the recheck process - obviously you need to use this option with care.

[GaryE]

I have not figured out the exact circumstances . . . but sometimes when that happens . . . try stopping the force recheck . . . and then right click and choose "check if files exist".  I know there have been some cases when that has obviated a force recheck for me -- I just cannot remember the details.  Try it . . . it might help -- if not no harm done right?

[parg]

B15's out - give it a go if you get a chance: see http://wiki.vuze.com/w/Beta_Program

[avelev]

FORCE RECHECK:
Although I do not recommend it, you can disable the file recheck in OPTIONS > FILES and uncheck the box for "On crash-restart check the entire file for completed pieces"
A manual "Force-Recheck" can still be done at any time.


 

Thanks ekstasee,
I upgraded to v5.6.1.2 --much better context menus among other improvements.

Thanks GaryE,

"Check files exist" does exactly the trick because I'm doing that on pretty "static" files stored only for seeding. Of course I force re-check manually the "Downloading" ones.

The whole point is to resume quickly.
BTW ekstasee, the "On crash-restart..." options setting you suggested was already unchecked and it was not helping. It is actually a subitem of the "Use Fast Resume" group. I am not sure I fully understand "Use Fast Resume" but I don't think my scenario is the type of crash the "Use Fast Resume" feature addresses. Mine is not a crash at all. It's a disconnection. It can at best be called a pseudo-crash as no file corruption is at issue and that is why the system should have a quick and easy and automated way to bring those torrents back online.

[avelev]

B15's out - give it a go if you get a chance: see http://wiki.vuze.com/w/Beta_Program


 

Thanks parg.

I'm not new to torrenting but new to Vuze. When I get the chance to set up a test-box or VM I'll try the betas. I'm not comfortable playing with a beta on terrabytes of data.

GaryE and ekstasee helped a lot but it looks like the second question in my original post (how to automate resuming) is turning into a feature request. I'd rather run this wish by you before escalting it as a true feature request because I'm too noob and hate to re-discover the wheel, or worse just babble.
 
To start with, the scenario I'm presenting is one where the blip is a pseudo-crash and no files are corrupted on the disk. It is focused on torrents being seeded which are completely "static" on the seeder's storage device. No incoming packets to complicate matters. With those premises in mind there should be a quick AUTOMATIC resumption and then maybe a deeper check in the background. The two are not mutually exclusive.

I cannot constantly look over literally thousands of torrents I'm seeding. So if a bleep occurs causing it to show the file-does-not-exist error, it will likely be unnoticed for a long time. I suggest for Vuze to poll the error-flagged torrents and automatically check-file-exist them, unflag and start them. That is not so resource intensive and the polling frequency can also be configurable. If the premise in my scenario is true (that files are not really missing nor corrupted) then the good torrents would quickly be unflagged and subsequent polling iterations would be over fewer and fewer items thus reducing the overhead of the loop.
 
 I am not sure I fully understand "Use Fast Resume" but I don't think it's the polling feature I'm suggesting. I think "Use Fast Resume" targets torrents crashed while actively downloading.

The reason the feature I'm requesting is important is to more elegantly maintain uninterrupted the availability of content for others.

[parg]

There is by default a low-resource rechecking of pieces for completed files - see

Tools->Options->Files->Perform low resource recheck of pieces when seeding (default = on)

Also, for Vuze-Vuze peer connections there is a message protocol extension that allows a peer to report that a piece it received was corrupt and this triggers (within reason) a more immediate recheck of the potentially bad piece.

[avelev]

There is by default a low-resource rechecking of pieces for completed files - see

Tools->Options->Files->Perform low resource recheck of pieces when seeding (default = on)

Also, for Vuze-Vuze peer connections there is a message protocol extension that allows a peer to report that a piece it received was corrupt and this triggers (within reason) a more immediate recheck of the potentially bad piece.

 

parg

I know. It's exactly this low resource re-check that is causing some of the problem. This background scan is too quick to flag and too slow to unflag the inactive seeding torrents which Vuze would otherwise be oblivious of. Same for the active ones but they get error-flagged even quicker when a peer fails to dl a piece.



IMHO the only real solution is to auto-scan only error-flagged torrents for "check file exist" and do so repeatedly at a configurable interval i.e. polling. I hope the developers take notice of this request as it would bring back online the content on millions of peers' machines out there. Trackers would report much better seeding stats. It could be huge!

[GaryE]

Just an fyi.  The problem you are describing has happened to me a few times (but not often).  What I do is I sort the torrents by tracker status.  If an error occurs with a torrent(s) those errors immediately pop to the top of the list.  Then I am able to see the problem much earlier and deal with it in a more timely manner.

It is not the best solution . . . but it is something that can work better than just waiting to see those error cases by chance!

Or just sort by "tracker status" a few times a day to look for errors or whatever!

[ekstasee]

Just an fyi.  The problem you are describing has happened to me a few times (but not often).  What I do is I sort the torrents by tracker status.  If an error occurs with a torrent(s) those errors immediately pop to the top of the list.  Then I am able to see the problem much earlier and deal with it in a more timely manner.

It is not the best solution . . . but it is something that can work better than just waiting to see those error cases by chance!

Or just sort by "tracker status" a few times a day to look for errors or whatever!

 

Yes, I agree.  GaryE raises an issue that I suspect most people don't give any thought to and that is one of configuring your columns in the torrent display to reflect all the information you need to properly manage your torrents.  If you only ever have a handful of torrents in your client at any one time, the management process is simple, but when you have hundreds or thousands as some of us do, you need to "work" your client and your torrents by having all the information imperative to your decision-making-process and use the clients columns. 

After all, isn't that why you are using the most powerful and configurable torrent client there is?

[avelev]

Just an fyi.  ...  Or just sort by "tracker status" a few times a day to look for errors or whatever!


 

Yes, that works when I'm in the office working on the PC or actively torrenting. But I'm on the road sometimes for days and may not detect the issue. During that time the seeding is stopped.

The NAS I'm using by QNAP has a whole developer community putting out apps using that platform. One of those apps (it's actually a native one) is a simple torrent client. If I don't find a way to elegantly manage the issue in Vuze, I may  just export the completed torrents to that QNAP torrent client. The data is already on the NAS. I will continue to keep the incomplete torrents in Vuze because I really like it. I will lose many stats and searchability in Vuze and my seeding ratio will drop way down. But the seeding will not be dependent on a flaky microsoft network.

[avelev]

After all, isn't that why you are using the most powerful and configurable torrent client there is?


 

Amen to that!

[GaryE]

Just an fyi.  ...  Or just sort by "tracker status" a few times a day to look for errors or whatever!



 

Yes, that works when I'm in the office working on the PC or actively torrenting. But I'm on the road sometimes for days and may not detect the issue. During that time the seeding is stopped.

The NAS I'm using by QNAP has a whole developer community putting out apps using that platform. One of those apps (it's actually a native one) is a simple torrent client. If I don't find a way to elegantly manage the issue in Vuze, I may  just export the completed torrents to that QNAP torrent client. The data is already on the NAS. I will continue to keep the incomplete torrents in Vuze because I really like it. I will lose many stats and searchability in Vuze and my seeding ratio will drop way down. But the seeding will not be dependent on a flaky microsoft network.

 

The torrent client built into most NAS's is Transmission which is a buggy piece of junk.  Also some tracker's have banned that client and or severely limit which versions they allow/recommend, other trackers have it listed as a "use at your own risk" client.  It has a well documented history going back about 10 - 12 years of problems accurately reporting stats to the tracker.  In every independent test I have seen Transmission finishes at or near the bottom when correctly reporting download amounts (usually the only time it does not finish dead last is when there is a client which has a newly introduced bug).  It typically over reports download amounts by roughly 3% - 10%.  If you are on trackers where you care about your ratio's I strongly recommend that you do not use Transmission.

If you must use a client on your NAS4Free or FreeNAS setup I strongly recommend rTorrent or the ruTorrent/rTorrent combination if you must use a pure unix client.  Both NAS4Free and FreeNAS are FreeBSD based and they have support for rTorrent.  There might be a way to get Vuze working on FreeBSD . . . but I have no idea -- I can not recall reading about getting Vuze working on FreeBSD so I have no concept if it is possible.  Again if you care about your ratio and or your have difficulty maintaining your ratio do not use Transmission . . . it seriously sucks.  The time you spend to get rTorrent and/or the ruTorrent/rTorrent combination working is time well spent.