Help with mass retargeting of Torrents

[GaryE]

Ok I finally got my new NAS put together and working and the files transferred over.

So I just tried to do the bulk retargeting and it failed to work.  Here is what I did:

all of my .config files are in

MacintoshHD/Users/username/Library/Application Support/Vuze

So I converted downloads.config into downloads.json

I edited downloads.json and replaced the "save_dir" from:
\/Volumes\/External\/etc etc

\/Volumes\/Name Of NAS dir\/etc etc

I saved my changes
I made a backup copy of downloads.json
I converted downloads.json back to downloads.config

I stopped Vuze

I replaced 
MacintoshHD/Users/username/Library/Application Support/Vuze/downloads.config
With the file I just edited and converted
Then I restarted Vuze

I then started getting a lot of errors saying words to the effect of "Can not find file at /Volumes/External/etc etc"  (if it is important I can get you the exact text of the error . . . but by the time I started typing all of this I forgot it LOL).

This makes no sense to me LOL

I would understand if the errors were saying words to the effect of "Can not find file at /Volumes/Name of NAS dir/etc etc"  But that is not the error I am getting.

The exact replacement I am doing is:
replace:
\/External\/

with:
\/Name of NAS dir\/

for the replacement I am actually finding one of the two torrents whose "save dir" is seeding fine from the data on the NAS copying that data and pasting it into the replace field to make sure I am not making some stupid typo LOL

I am going to try changing the query-replace so that I am not replacing the '\/' to see if that makes a difference.

Oh I am doing the query replace in emacs not using 'TextEdit' or whatever.  I could use vi or 'TextEdit' or vim if anyone thinks that is a problem.

Any and all help will be appreciated!

[GaryE]

Ok I tried it without replacing the \/ and that made no difference.

The actual text of the error I am getting is:

Error: Data file missing: /Volumes/External/etc etc etc

[ekstasee]

Howdy GaryE.  sorry, I have not read all of the posts (time constraints) and I have no experience with Mac OS. but as with yourself, I am running 2461 torrents comprising 12 TB data spread over 4 x 4 TB drives.

I am using the "move data files" feature every day.  Sometimes it is just to move files from the "Torrents Leeching" folder to "Torrents Seeding" within the same HDD which is a fast process as no data is actually moved, just folder re-assignment as you know, but on other occasions I move files to folders that are similarly named ("seeding" & "Leeching") on another HDD which necessitates moving the files/folders (data) applicable to that torrent. No retargeting is necessary as Vuze does all the work in keeping track of the files and torrents.

I am not using a NAS as I have a large box with the capacity to hold 10 HDD.  As you have/intend to use a NAS, I would suggest that you ensure the OS of the NAS can run a torrent client and as you are building this yourself, I assume you are going to use a common OS such as Linux,OSx, Win. so installing a torrent client to your server should not be a problem.  That should simplify Vuze configuration for you running it all on the server (NAS) which can be left running unattended when you are busy with "real" life. 

My pc runs 24/7.  I use WD Black Caviar & Seagate NAS HDDs. I tried the Seagate as I was able to save some $$ and although I have not had any drive failure, I do intend to use WD NAS RE in the future, and probably will go SAS instead of SATA.  I use one drive as my "leech" drive doing all the hard work and then when it is near to full, I move the files to one of the other 3 HDDs so as to keep the files in some semblance of alphabetical order. i.e.,  numbers & A - G, H - P, R - Z.  When those fill up, I install another and so-on. Actually, I can see myself going to server storage in the next 6-12 months.  One of the reasons I move files to another HDD is to minimise fragmentation of the drives that are seeding. When I get a new HDD, I distribute completed files to the alphabetical HDDs for seeding, then move the files that are still leeching from the existing "high load leeching drive" to the new drive so as to maintain maximum reliability of the "high workload" leech drive (which will now be the new drive).  As I don't always retain everything I download, the "leech" drive can become fragmented so one advantage of transferring all the leeching torrents to a new drive is that it creates an unfragmented drive (data copy, not cloning).  As you would know, these large HDDs can take hours and sometimes days for defragmentation, data-recovery etc. so my drive rotation method is my system of maintaining minimum fragmentation and reliability without having to put the machine under extreme load if I were to defrag while torrenting, or stop the torrent client.  I have never been a big fan of the defrag process of moving data to a temporary place on the drive and then for the data to be moved back again. Double handling of data at least and the same data might even be moved around several times before the defrag algorithm is "happy".  Less stress just to move data in one move.

Anyway, if I haven't helped, perhaps I have given you something to consider.  Best of wishes with your problem.

[GaryE]

Move data files is not going to work for me.  If my content was better organized it might be an option but for many reasons my content is not organized in a way that I can select say any 300 torrents and do a mass redirection.  I would have to do it one torrent at a time.  That is one thing when you are moving the data for 5 or 10 or 20 torrents a day . . . it is another thing if you need to move the data for 3,149 more or less all at once.

For torrents that have only one or two or five data files it would not be a huge deal.  For torrents which have 300 + plus data files . . . it is a big deal.  Not to mention that a lot of the torrents I have were for example initially uploaded in say 30 parts.  Then some time later (often several years) I would combine those 30 torrents into one huge torrent and upload it elsewhere while still seeding the originals at the original torrent site.  That gets very complicated very very quickly.

This is why I asked for the ability to edit the downloads.config file in the first place.

Yes I can run a torrent client on the NAS but I have no desire to do that for several reasons which I am not going to into all of the details.  The main reasons are that I much prefer running a client on my Desktop and I did not design the NAS for torrenting . . . I would probably have to upgrade the CPU's and the memory to do that and that is simply money I do not want to spend.  Not to mention that running a torrent client on the NAS accomplishes nothing regarding redirecting files.  I would still have to add the .torrent files into the client on the NAS one at a time.  Then point the client to the correct existing location of the data.   And then wait for the torrent client to hash check the existing files . . . which again is not a big deal for a small torrent which only has 5 or so files . . . but it takes a long time when you are talking about roughly 35,000 - 50,000 files in total.

The easiest solution is to just edit a config file and be done with it.

I think there is just an extra step I am missing . . . but I can not figure out what it might be.

Hopefully one of the Vuze folks can tell me:

"Gary you idiot you have to edit the XXX file first and change the YYY paramter and then it will work!"  LOL

There are some other things I want to try . . . to try to clarify the situation . . . hopefully I will get to that sometime today.

Thanks for the response however.

[GaryE]

Ok I figured it out . . . kind of.

There are "only" about 90 torrents which are not responding to my mass retargeting.  It seems those are all torrents which I retargeted after I downloaded them for whatever reason.  So I only need to manually move the files for those 90 torrents.  I do not know how long that will all take but it is manageable.

So the approach I outlined above does work for the vast majority of my torrents.  There are a few exceptions but as long as you do not do a lot of combining of torrents and or cross-seeding this system works great.  For those torrents moved about a lot after the torrent is completed or renamed so you can cross seed a torrent with files named this.is.a.long.name on one site while the identical file is named "this is a long name" on the other site . . . those kinds of things . . . those torrents need to be manually retargeted/moved.

[parg]

Ah, yes, your 're-targeting' operation has resulted in another level of crud being stored in the .dat files in the 'active' directory (these are actually just .torrent files with some extra data stored in them such as file linkages)

[GaryE]

Yeah that makes sense.  No big deal.  Took me about 3 - 4 hours of manual work and I am not sure how much real time (I went out for dinner with my wife and a trip to the shops in the middle of the operation).  I would guess about 2 additional hours of waiting for stuff to hash check.

Much better than any alternative LOL

Thanks again for implementing the conversion option in the program parg!  It saved a huge amount of time!

[parg]

No problem matey :)

[GaryE]

Just a bit of extra information.

I will make a very very very long story short.

I discovered that I still had about 200 torrents that were not correctly re-targeted.  I could not re-target them manually (from within the Vuze) until I hooked up my old External drive again (it was actually in pieces and its drives were about to be reformatted in my NAS so I was very fortunate that I was checking and double checking everything and found the problem when I could still fix it).

For some reason Vuze will not let you re-target torrents if the destination directory is currently unreachable/non-existent.  It was blind luck that I figured this out.  LOL

I just wanted to let everyone know in case someone else runs into this problem in the future.

This might be a Mac only thing or it might be a Vuze thing I am not certain . . . but again I wanted to let everyone know who might stumble upon this thread in the future.

[ekstasee]

Just a bit of extra information.

I will make a very very very long story short.

I discovered that I still had about 200 torrents that were not correctly re-targeted.  I could not re-target them manually (from within the Vuze) until I hooked up my old External drive again (it was actually in pieces and its drives were about to be reformatted in my NAS so I was very fortunate that I was checking and double checking everything and found the problem when I could still fix it).

For some reason Vuze will not let you re-target torrents if the destination directory is currently unreachable/non-existent.  It was blind luck that I figured this out.  LOL

I just wanted to let everyone know in case someone else runs into this problem in the future.

This might be a Mac only thing or it might be a Vuze thing I am not certain . . . but again I wanted to let everyone know who might stumble upon this thread in the future.


 

HMM. I think it's a Vuze thing.  A few months back, someone who thinks they know Vuze inside-out and whom will remain nameless, wasn't being attentive and clicked the delete button and lost a whole 4TB HDD which required data-recovery.  Anyway the long and short of it is that while most torrents (~700) could be re-targeted after data ressurection, there were some (~60) that had other ideas and required re-loading and re-targeting via the torrent-load window.  As you might imagine, that whole saga was a few hours of work and brain-drain and I never did come to a conlcusion as to why most did and some didn't.

The need to be ever-vigilant!

E

[megatron]

I'm editing downloads.json now, but I think this page should be updated to reflect the converter's existance within vuze:
http://wiki.vuze.com/w/Copy_Vuze_to_new_computer