Help with mass retargeting of Torrents

[GaryE]

Ok this is going to be a bit of a story so please be prepaired to read.  LOL

I have an External RAID 0/1 (formatted RAID1) connected to my Mac via FireWire800 which contains all of the data for all of the roughly 1,800 torrents I am actively seeding and about 700 other torrents which are not actively seeding (because they are dead or the sites I got them from are dead).  The vast majority of the .torrent files I downloaded are on that external as well (but I do not think that matters) but the Vuze directories are all on my computer's HD.

Most of the data is in one single directory but not all of the data is in that directory -- some of it is spread out amoung 3 or 4 other directory trees of that Hard Drive.

My external is full and I am in the process of building a NAS (which is very very expandable) which will now house my torrent data.

What I want to avoid is having to manually re-target each of my roughly 2,500 torrents which comprimises about 3.4 TB of data.  That would take many many many days of work.

In another forum someone mention that there is a single file you can edit in uTorrent which contains the location of the data for each individual torrent.  Well I thought that Vuze might have a similar file.

Well after a bit of investigation I think that Vuze actually stores the data in an individual ascii/binary file for each torrent.  On my computer at least that data is located in the following directory:

/Applications/Vuze Config Backups/YYYY-MM-DD/active

in that directory there are two files and one folder for each torrent (or very close to that . . . I think the actual number is off by about 6 or 10 but I suspect that is because I have downloaded or uploaded about 6 or 10 files that particular day.

Those file names are (these are actual example names):

FFF60D5A324F00D5845061CBCBFA4A29D3B9B58F
FFF60D5A324F00D5845061CBCBFA4A29D3B9B58F.dat
FFF60D5A324F00D5845061CBCBFA4A29D3B9B58F.dat.bak

The first one is a directory.
The second file seems to contain the location of the data (amongst other things I can not decipher because they are in binary)
The third file seems to be a backup copy of the second file.

The first two lines of that second file look like this:

1) d8:announce68:http://tracker address and port/passkey/announce10:attributesd39:Plugin.<internal>.DDBaseTTTorrent::sha140:99780D0999D1BB82E3932D1B5D37F60CFD033B3134:Plugin.<in\
ternal>.azsubs.subs_infod2:lci1429534574503ee63:Plugin.azextseed.no-ext-seeds-ExternalSeedReaderFactoryGetRighti1e62:Plugin.azextseed.no-ext-seeds-ExternalSeedReaderFactoryWebSeedi1e28:Plugin.az\
rating.globalRating3:0.08:badavaili0e11:canosavedir125:/Volumes/External/unwatched/name of data file

2) /Volumes/External/unwatched/name of data file8:category6:femdom11:file.expandi0e14:filedownloadedd10:downloadedli738097518eee5:flagsi8e14:mdinfodictsizei14296e8:networksl6:Publi\
ce10:parametersd8:dndflagsi2e25:stats.download.added.timei1420911984230e29:stats.download.completed.timei1420912575673e34:stats.download.file.completed.timei1420912575619ee11:peersourcesl7:Track\
er8:Incominge4:pkdoi0e4:pkupi951080e14:primaryfileidxi0e9:reordermbi10e14:resumecompletei2e11:scrapecachei38654705664e7:secretsd2:p120:\350\246G\3331Ys\331\355\207\311B\321\271\303\254D\224\2374\
e7:sr.progi6140162293779933944e10:storetypesl1:Re11:timesincedli-1e11:timesinceuli404735e11:timestoppedi1428300767337e7:versioni1ee26:azureus_private_propertiesde18:azureus_propertiesd7:Contentd\
ee7:comment43:link to torrent on torrent site created by14:uTorrent/3.4.213:creation datei1420876160e8:encoding5:UTF-84:infod13:file-durationli3526ee10:file-mediali0ee6:lengthi7380\
97518e4:name34:name of data file12:piece lengthi1048576e6:pieces14080:\350\246G\3331Ys\331\355\207\311B\321\271\303\254D\224\2374\333\210\342\370VS\266^S\255y^W<\232hW\231u\337\\
2627\320\330\241f&D^X\222\247\300\213Y\315\316[\372\376\247B\360M\275^P%\223\330\317\344\316\240G\323^U\235\255G{^F\237Zp\206^T^Sj<\274\243p^Sm7\252[\355my\274fPJ\305\365\300\360w\357[\202\261\2\
56X\315h\376\234\330\352X\305f\362\304x^\347\345\303`\237,^^FnNb\234\245\246\333^F\360d^H^G^K\272\213`\237F\222\3546\341\322=N\227}\333o^[\247\322\353h\354\241U`g\241\372^Q\376\316fs_\255\270\22\
73=\236\254acdC7a^X\322\214\313^\DB5\370^^0\256=\212\!\356\242^_\230t\326^Q^\\261\232\270\220d\247<^L^Y\313\267^\!\212\227.QY_\341^C00\377\305\232\366T^M5\34423\261\211\225\223@f\232J2SS8w^A\234\
)\372\347\206Y\311\220\337^T\252o\224\314^]\277^]\254\257k!\202-\302\331fo\312^V\222)\246z\366P^Wsx\253:I

So here comes the question.  If I write a program (or perl script or shell script or whatever) that would open up each of the .dat files and replaced /Volumes/External/ on both the first and second line with the address of my NAS . . . the next time Vuze started . . . would it just read the data from that new location and start seeding?

It seems to me that it would.

Or do I need to replace /Volumes/External only on line two?  Or only on line one?

Or is there a better way to mass retarget the torrents?  Mabye there is already a plugin which does this?  Or there is a simpler way?  If there is I can not figure it out.

I might be able to create a symbolic link between /Volumes/External and the NAS but I have no idea if that will work across logical volumes.  I will try that first once I get all the pieces of my NAS in the next week or so and I get it built and configured in the next week or so after that.  But I want to get a jump on this before that day comes.

There are other ways of doing this (like redownloading all of the .torrent files, removing the torrent files from Vuze, reloading all of the torrent files into Vuze, clicking ok roughly 1,800 times, then when asked if it is ok to check the existing files and overwrite them if necessary clicking ok again 1,800 times, and then letting them hash check)  but that idea , like all the others I have heard or came up with, are not significantly different than manually retargeting each torrent.  Which when you have some 2,500 torrents is going to take a very long time.  Also that idea does not take into account the 700 files that I can not get new .torrent files for because those torrents have been deleted from the site in question, or the site in question has gone away or whatever.  I know that some of you might be asking if the torrents are dead why do I keep them . . . well the answer is that I use Vuze as a rudimentary database to organize my data so it is important that I can find the data for the dead torrents from within Vuze (yes there are probably better ways to do that but that is a different story -- this is my prefered way to do that)!

Thanks in advance for any and all assistance.
 

[ArronM]

If you have not moved your files to the NAS yet, you can select them all, right click and do "Content"->"Move Data Files..".  It should only ask you once for the directory, and move everything into it (single file torrents to the base of the selected directory, multi-file torrents into subdirs based on their torrent name)

[ArronM]

If you have moved your files already, it takes an extra step (which it shouldn't and someday might get fixed):

1) select all your torrents
2) right click, choose "Content"->"Check Files Exist".  This should put them all into error state
3) right click, choose "Change Data Directory", select your NAS dir.  Hopefully all or some of the torrents will go back to non-error state.

Currently, skipping step 2 unfortunately results in step 3 doing nothing!

[ArronM]

For actual manual editing of the vuze config (which I don't recommend), the config files are actually at

%appdata%\Vuze

There's a downloads.config, which contains a "save_dir" key.

There's also all those dat files in %appdata%\Vuze\Active which have that canosavedir you found.  I don't know much about how that works (ie. I don't know if downloads.config overrides the value in Active).  In either case, if you were changing the directory manually, ensure you update the string length.  The files are in bencode, so changing something like "canosavedir1:/" to "/foo" would make it "canosavedir4:/foo" (note the 1 changed to a 4)

[GaryE]

If you have not moved your files to the NAS yet, you can select them all, right click and do "Content"->"Move Data Files..".  It should only ask you once for the directory, and move everything into it (single file torrents to the base of the selected directory, multi-file torrents into subdirs based on their torrent name)

 

I have not moved them yet.  I do not even have all of the parts for the NAS yet . . . much less have I assembled everything.  LOL

Ok I just tried that and it did not work.  It did not move the files from one volume (my External) to another volume . . . my iMac's internal drive.  Maybe it works when you are moving the files from location A on one volume to location B on that same volume . . . I do not know!

All I know is that I did it by selecting two torrents (two of the dead ones) I did Content->Move Files . . . it prompted me for a new location . . . I choose the root directory of my internal drive and nothing happened.  No error message, no completion message, no are your sure . . . nothing.

I thought it could be a permissions error so I tried again using a folder at a deeper level.

It did work in that the files were moved but it did not preserve the directory structure.  That is a problem for me.

----- Begin Edit
Let me be more clear.  I will not use the actual names but this is kind of how it was.  One of the torrents was a single file torrent it was located:
/Volumes/External/TV/filename.wmv

When I "moved" it to /Volumes/Macintosh HD/fubar
the resulting file was:
/Volumes/Macintosh HD/fubar/filename.wmv
Where as I wanted to see:
/Volumes/Macintosh HD/fubar/TV/filename.wmv

The other torrent I moved was a multi file torrent that had its' own directory.  The whole directory was moved . . . as you said . . . but it too was inside of another folder . . . and that folder was not recreated either.
For the sake of provideing an example let us say the torrent had the following structure:
NameOfBook/book.mobi
NameOfBook/book.epub
NameOfBook/book.pdf
etc

NameOfBook was inside another directory:
/Volumes/External/Books/NameOfBook

so after moving it that parent folder was not re-created.  The new directory structure was:

/Volume/Macintosh HD/fubar/NameOfBook

Again that is a problem for me.  That entails me figuring out which torrent is in which directory before I move it.  Yeah most of my torrents are in the same directory but there are a few exceptions . . . and I am making it sound a lot less complicated than it really is.  Again the parent directory names I used were pure fiction.  But even if they were not . . . it would not be simple to identify which torrent was in which directory without manually looking at each torrent.  Essentially that is no different than selecting one torrent at a time and re-targeting it then selecting another etc etc etc * roughly 2,500  LOL

----- Edit End
Thanks for the response though!

 

[GaryE]

If you have moved your files already, it takes an extra step (which it shouldn't and someday might get fixed):

1) select all your torrents
2) right click, choose "Content"->"Check Files Exist".  This should put them all into error state
3) right click, choose "Change Data Directory", select your NAS dir.  Hopefully all or some of the torrents will go back to non-error state.

Currently, skipping step 2 unfortunately results in step 3 doing nothing!

 

Ok this sounds more promissing . . . but the words "hopefully all or some" does not give me a lot of confidence.  I mean if some means 98% that might be a viable alternative but then you say hopefully.  LOL

Can you quantify that a bit?

[GaryE]

For actual manual editing of the vuze config (which I don't recommend), the config files are actually at

%appdata%\Vuze

There's a downloads.config, which contains a "save_dir" key.

There's also all those dat files in %appdata%\Vuze\Active which have that canosavedir you found.  I don't know much about how that works (ie. I don't know if downloads.config overrides the value in Active).  In either case, if you were changing the directory manually, ensure you update the string length.  The files are in bencode, so changing something like "canosavedir1:/" to "/foo" would make it "canosavedir4:/foo" (note the 1 changed to a 4)

 

Ugh bencode LOL that would be doable it would just make the whole thing more complex.  It would necessitate doing it in perl or some other real programming languge ( which honestly is what I was most likely to do anyhow).  But it does add a layer of complexity and make it more likely to go wrong LOL

Also I have never had to mess with bencode so there would be a steeper learning curve.

That downloads.config sounds like a more viable alternative but I am not sure about that.

First off it looks like it is bencode also.  Can you confirm or deny that?
Secondly, I just counted the number of lines in that file and there are only 212 lines.  So unless I am missing somthing that is far from all of the torrents I am currently running.

Well I investigated that a little further there are the right number of occurances of save_dir in that file . . . more or less . . . I think it was within about 14 which when we are talking about 2,800 torrents (I actually double checked the number at it is exactly 2,800 right now) getting 2786 of them is close enough . . . I should be able to figure out which are the 14 which didn't work pretty easily.  Nevermind I picked an older version of downloads.config . . . I have downloaded exactly 14 torrents since the file I looked at was created LOL

That is much more promissing.  I could in theory just open downloads.config in a bencode editor and do a search and replace.  That would be awesome!!!!!  We are talking about a 30 second fix (once I move the files over) if that is the case!

I know I am asking a lot here . . . but is there any way you could verify that with a coder?

Thanks again for all of the responses!

[GaryE]

Of course the problem I am having now is that I can not find a simple text editor with bencode support which is OS X native.  Heck I can not even find one which I can use in Darwin/freeBSD.

This might mean the only way to edit downloads.config is:
by writing some code to do it in perl or python or C
by sending the file to someone with a Windows machine
by hand and doing the bencoding by hand (this is the most promising believe it or not LOL)

[parg]

Or I could write a utility to convert bencoded data to/from json (which is much simpler to edit in a text editor...)

[GaryE]

that sounds good LOL

I was thinking about doing that . . . it would not be terribly hard to just convert the whole file.  But I was hoping to avoid that extra step LOL.

 

 But if you were able to do that easily I would be forever grateful!

[parg]

5601 B30 (probably out later today) has Help menu -> Advanced options to convert between the two formats

[GaryE]

You are awesome!  Thanks mate!

 

[parg]

B30's out - tell me how it works out!

[fields]

Hi all, its correct, user now can convert downloads.config to JSON for editing in the text editor, say, change pathes in it?

[parg]

that was the idea:

export azureus.config into json
edit it in text editor
convert it into bencoded azureus.config.temp
close Vuze
copy azureus.config.temp to azureus.config (make a backup of azureus.config first to be sure)
start Vuze
hope

[GaryE]

I will try it when I get to that point I am having issues with the NAS I am putting together (a few missing parts and one part which was delivered (in three pieces) but did not fit together LOL).

So it is going to probably be about a week before I am ready to go.  LOL