Community Forums

Home » Vuze Forums » Community Open Support


Thread: Cannot add torrents via magnet after upgrade to Windows 7


Thread Locked This thread is locked - replies are not allowed.


This question is answered.

Search Forum Search Forum Back to Thread List Back to Thread List

Permlink Replies: 21 - Pages: 1 - Last Post: Dec 4, 2009 9:08 AM Last Post By: parg
BACON

Posts: 3
Registered: 11/23/09
Cannot add torrents via magnet after upgrade to Windows 7
Posted: Nov 23, 2009 3:12 PM
 
Click to report abuse...
For several years I've run Azureus/Vuze on 32-bit Windows XP Professional and had no issues adding torrents via .torrent files or magnet URLs. About a month ago I did a complete reinstall with 32-bit Windows 7 Professional and since then I cannot add torrents via magnet, whether I click the link in my web browser or manually paste the URL into the Open Location dialog. I've tried numerous torrents with thousands of peers from various sites with no success. In every case the Operation in Progress dialog appears and in the Details window it says "Downloading From :magnet:?xt=urn:btih:<HASH>" and then it just sits there indefinitely. Back when it worked I remember it would start listing the peers it was contacting, but now it just does absolutely nothing until I cancel the operation.

I copied my %AppData%\Azureus directory directly from my Windows XP to my Windows 7 user profile, so all of my settings are exactly as they were before. Vuze is configured to use port 56881 for TCP and UDP. My router (pfSense v1.2.2) is configured to forward both protocols on that same port, and I have completely disabled Windows Firewall. All peer sources and networks are enabled. Transport encryption and IPv6 are enabled, though I've also tried it with those settings disabled. The Distributed DB plugin is enabled. If I run the NAT/Firewall Test it is successful. In the status bar I have a green light that says "NAT OK" and the Distributed Database always shows over a million users. The support information from the About Vuze dialog is as follows:

Java 1.6.0_17
Sun Microsystems Inc.
SWT v3550, win32
Windows 7 v6.1, x86
V4.3.0.4/4 az2

Any ideas what else I should look at? As best I can tell, everything else works as it did with Windows XP, it's just magnet that's the only problem. This doesn't seem to be quite the same issue others have reported with recent versions not handling certain magnet links properly, since I've yet to across one that worked at all and, well, I thought v4.3.0.4 addressed that issue. Note that I was likely running v4.2.0.8 when I first installed Windows 7, so it isn't just the v4.3 series that's given me this problem. Thanks for the help.
Washii

Posts: 5,261
Registered: 11/14/07
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Nov 23, 2009 7:55 PM   in response to: BACON in response to: BACON
 
Click to report abuse...
How about posting one of these magnet links that is failing for you?
BACON

Posts: 3
Registered: 11/23/09
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Nov 23, 2009 8:31 PM   in response to: Washii in response to: Washii
 
Click to report abuse...
Well, for example, let's try the install CD for 32-bit Ubuntu Desktop 9.10. MiniNova has a magnet link of magnet:?xt=urn:btih:TDC4GYOQXZPSUB7KR6SQKLS2USAJPZ7W&tr=http://torrent.ubuntu.com:6969/announce and ThePirateBay has magnet:?xt=urn:btih:98c5c361d0be5f2a07ea8fa5052e5aa48097e7f6&dn=ubuntu-9.10-desktop-i386.iso&tr=http%3A%2F%2Ftracker.openbittorrent.com%2Fannounce . I've tried both of those just now and the Operation In Progress windows just sit there. Vuze is not frozen, however, and all of my existing torrents continue to run. I can also download the .torrent file from each site and that works, too.

Another torrent you can try is this one for Gentoo Linux that gave me the same result: magnet:?xt=urn:btih:F7J66AGKGP6IN73DP3SDQRDP2V6K7VQW&tr=http://torrents.gentoo.org/tracker.php/announce

That magnet link is from MiniNova. Both torrents currently have close to 4,000 peers each.

Edited by: BACON on Nov 23, 2009 10:49 PM
mmore1q3


Posts: 6,194
Registered: 12/02/07
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Nov 24, 2009 7:20 AM   in response to: BACON in response to: BACON
 
Click to report abuse...
All three magnet links worked for me just fine.
The three links mentioned by the previous writer, can be found from following pages:
http://thepiratebay.org/torrent/5140002/
http://www.mininova.org/tor/3096822
http://www.mininova.org/tor/697697

(If Vuze was already running, all magnet links worked quickly. If vuze was not running, the the first magnet link remained open for a rather long time (several minutes?), while DHT DB got initialized.

Java 1.6.0_17 Sun Microsystems Inc.
SWT v3550, win32
Windows Vista v6.0, x86
V4.3.0.5_B01/4 az3

Edited by: mmore1q3 on 24-Nov-2009 17:32
Just tested also from Windows7. works ok.
I guess there is something in your system.
Java 1.6.0_17 Sun Microsystems Inc.
SWT v3550, win32
Windows 7 v6.1, x86
V4.3.0.4/4 az2
dobri

Posts: 13
Registered: 11/30/09
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Nov 29, 2009 7:06 PM   in response to: mmore1q3 in response to: mmore1q3
 
Click to report abuse...
I have the same problem - magnet links do not work on my Windows 7 machine, but they work on XP.
I tried the same version of the code on both machines and the latest beta - did not make a difference).

I did some debugging myself and found this:

1. The operation seems to involve Azureus talking to itself over the network. The log shows:
MagnetURI: bound on 45100
It does not log the address, but that is hardcoded to be 127.0.0.1
The actual code is in com.aelitis.net.magneturi.impl.MagnetURIHandlerImpl

2. That same class can be run as an entry point from Eclipse, producing similar behaviour

3. There is another class, which seems to be the client side and this is com.aelitis.net.magneturi.impl.MagnetURIHandlerClient
This one can be executed by itself too and assuming that you run either the whole Azureus application or just the class above you get the following output:

AZMSG=AZMSG;1;torrent;is-ready -> false
AZMSG=AZMSG;1;torrent;load-torrent;{"url":"http://www.vuze.com/download/VCCBRHY5GYNGFKPJSYQID4GB3XPTYGIG.torrent?referal=jws","play-now":true} -> false

I cannot judge if this is the expected outcome or not, but seems that to get "true" you need a response of 134 bytes, while you actually get only 94:
66, 77, 94, 0, 0, 0, 0, 0, 0, 0, 54, 0, 0, 0, 40, 0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 40, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

When I do this my firewall shows connect to port 45100 and Azureus log shows:

21:44:33.350 0 net MagnetURIHandler: processing 'GET /setinfo?name=AZMSG&value=AZMSG;1;torrent;is-ready HTTP/1.1'
21:44:33.354 0 net MagnetURIHandler: processing 'GET /setinfo?name=AZMSG&value=AZMSG;1;torrent;load-torrent;{"url":"http://www.vuze.com/download/VCCBRHY5GYNGFKPJSYQID4GB3XPTYGIG.torrent?referal=jws","play-now":true} HTTP/1.1'

4. When I try to open a magnet link from Azureus itself (File->Open Torrent File->Add from Clipboard) I get a progress bar showing 0% (staying at that forever), the firewall shows a connection to port 45100 and nothing happens. When I press the "Cancel the operation" button the Azureus log shows:

21:51:26.524 0 logger A torrent file download has been successfully cancelled by a user action: magnet:?xt=urn:btih:VLEOGWDMY6L7CYD2KWTMKWQN6SH6QPJN
21:51:26.524 1 net MagnetURIHandler: connect from '127.0.0.1': no data read

Which according to the code means nothing was read from the socket.

I suspect the actual call is made from org.gudy.azureus2.core3.util.protocol.magnet.MagnetConnection, but that class does not have logging statements and it is hard to confirm.

I don't mind doing further investigation on the topic, but I need a few pointers - how to setup a development environment (I took the latest code as in a ZIP file, but there a missing libraries, I prefer Eclipse as a tool) and which classes are the real client code for the magnet. I am really confused, wasn't this supposed to be a distributed search against DHT (I would guess this is where the MagnetURIHandlerListener comes) and why does Azureus communicate to itself through sockets?

Java 1.6.0_17
Sun Microsystems Inc.
SWT v3550, win32
Windows 7 v6.1, x86
V4.3.0.4/4 az2

BTW I am running the 64 version on Windows with 32 bit Java.
dobri

Posts: 13
Registered: 11/30/09
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Nov 30, 2009 9:41 PM   in response to: dobri in response to: dobri
 
Click to report abuse...
Tried to disable IPv6 in the Advanced options to level the field with XP, but that did not help much. Did not go as far as disabling it on the Windows level. May be I should.

Then fully debugged the com.aelitis.net.magneturi.impl.MagnetURIHandlerImpl for a while and that seems to work when I use Firefox to send the magnet URL (something like http://127.0.0.1:45100/....) - apart from getting an double slash at the beginning, everything is fine. Removing that extra slash in the debugger and everything else is working.

The I started from the other end - from the dialogue box where you get the url from the clipboard and up to org.gudy.azureus2.core3.torrentdownloader.impl.TorrentDownloaderImpl. I went through the part it communicates with MagnetURIHandlerImpl and it got back response 200 with reasonably looking other parameters. Made it pass the STATE_INIT and I gave up before the START_STATE. I don't understand if it is about to start resolving the magnet or it is going to start downloading the actual torrent. I don't know what the protocol is at that time, it is getting tough to debug further.
dobri

Posts: 13
Registered: 11/30/09
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Dec 1, 2009 6:58 PM   in response to: BACON in response to: BACON
Correct
Click to report abuse...
ROOT CAUSE found: ESET Smart Security interfering with the communication between org.gudy.azureus2.core3.util.protocol.magnet.MagnetConnection and com.aelitis.net.magneturi.impl.MagnetURIHandlerImpl
In a couple of words the second class never receives the information from the first one.
The longer explanation goes like that: the communication is established successfully and the second class attempts to read from the newly created connection, but it does not get even a single line. That is why when you finally hit Cancel you get a message that nothing was received - that is the case, the second class gets an exception only (socket closed).

WORKAROUND that proves the case: open the ESET configuration and under Setup, Antivirus and Antispyware disable "Web Access Protection". After that the magnets work normally. Enable the option again and they stop working again.
Clearly ESET has no business interfering with the network traffic inside our own machine, but I will not make them change it. More likely I will change the security suit that I am using.

*SOLUTION*? I have not tried yet, but I will. The root cause here is that Azureus should not be talking to itself over the network. MagnetConnection is already using MagnetURIHandler.getSingleton().getPort() to obtain the port where the other component is, so they are coupled by design and deploying the 2 classes on separate machines is not likely. I will make them talk to each other directly, through sendRequest and retrieveResponse methods. It is a bit complicated because the current code is multithreaded and allows multiple MagnetConnections to execute in parallel, but it is attainable. We probably don't need that, but I will try to preserve it since it is already there.
BACON

Posts: 3
Registered: 11/23/09
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Dec 1, 2009 7:37 PM   in response to: dobri in response to: dobri
 
Click to report abuse...
Great find, dobri. I use Eset's NOD32 Antivirus, and after disabling Web Access Protection as you noted, Magnet links work once again. Thanks!

Curiously, I'd used the 4.0.x series of NOD32 on Windows XP for the better part of this year and had no conflicts with Azureus, so I wonder if it's something in Windows 7 or the more recent versions of NOD32 that introduced this problem. I'm not crazy about the workaround (because that seems to be quite a major component of the virus protection to be disabling), but at least it identifies the problem.
dobri

Posts: 13
Registered: 11/30/09
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Dec 2, 2009 3:02 PM   in response to: BACON in response to: BACON
 
Click to report abuse...
Further analysis shows a very quick and painless bugfix:

In MagnetConnection.java append a second pair of \r\n to the end of the string that is sent into the stream.
I have tested the change and it works.

The reason it works is because the communication looks like a HTTP connection to the firewall/antivirus, but it violates the HTTP protocol. In Active mode, NOD32 waits for the complete HTTP message, inspects it and then forwards it, which as is already know breaks a lot of applications. Therefore they invented a Passive mode, which means it is not buffering the complete message, but portions of it. The HTTP request contains an empty line that is the end of the header of the request and the start of the body, which is optional. But the empty line is mandatory. In this case our server side uses the first line only, so technically we don't need the empty line. But the filtering software does not have a reason to forward this portion of the message just yet - since the server is supposed to wait for the empty line the filter can wait for the empty line too.

If you look at the test code, which works with or without the antivirus, the string ends with \r\n\r\n.

I don't have a developer access to the source code yet and I am not familiar with the change management process, so if someone from the existing developers comes across this post please go ahead with the patch.
parg

Posts: 2,698
Registered: 11/13/07
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Dec 2, 2009 3:13 PM   in response to: dobri in response to: dobri
 
Click to report abuse...
Darn it! I've just implemented a new MagnetConnection2 that uses a direct connection to the magnet uri handler to do the job, but I see what you mean about the missing \r\n. I'm wondering whether to revert the code and add your fix or just go ahead with my new approach - avoiding the socket connection entirely does seem the better approach and I'll build it into the next snapshot (B4) - perhaps you could test it?
dobri

Posts: 13
Registered: 11/30/09
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Dec 3, 2009 4:19 AM   in response to: BACON in response to: BACON
 
Click to report abuse...
Direct connection is much better - the less we depend on the OS and firewall behavior - the better. It is better for the resource utilization too, not that anyone cares about it these days.

I don't mind testing it, but as I said - I am new to this and don't know what exactly the process is. Is it the latest CVS build so I can just download that and try it? How do I get the code and the build scripts, preferably straight from CVS into eclipse to look at them - I cannot seem to find anything that is equivalent to the ZIP file that I downloaded? How do you normally debug SWT application from Eclipse? Is there a "setting up development environment for azureus" thread or wiki somewhere?
parg

Posts: 2,698
Registered: 11/13/07
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Dec 3, 2009 10:26 AM   in response to: dobri in response to: dobri
 
Click to report abuse...
Details for setting up eclipse to run off the CVS repository are here - http://www.azureuswiki.com/index.php/Using_Eclipse.

The fix is in the latest CVS build too - B04 from http://azureus.sourceforge.net/index_CVS.php

Thanks
dobri

Posts: 13
Registered: 11/30/09
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Dec 3, 2009 4:12 PM   in response to: parg in response to: parg
 
Click to report abuse...
I confirm that the fix for the magnet download in CVS works with ESET Smart Security - I am able initiate a download using a magnet URL. And it seems that it is faster than before.

However the progress indicator is broken - you don't see the progress, it stays at 0% until it is done and then just closes. And that is even when it takes 10 to 15 seconds to complete. I am not sure this is a bug, it seems that it is a feature - the indicator seems to show the process of transferring the torrent from MagnetURLHandlerImpl into the MagnetConnection. It used to be slow and that is where the progress bar came handy. Now that process is instantaneous and therefore the progress bar is useless and should be completely removed. I guess it cannot be used to show the process of the DHT search, since nobody really knows how long that is going to take and therefore you cannot show percentages.

Thanks for the fix.
Washii

Posts: 5,261
Registered: 11/14/07
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Dec 3, 2009 4:20 PM   in response to: dobri in response to: dobri
 
Click to report abuse...
I always thought it showed the progress of actually getting the pieces off the DHT. When getting stuff from Swarm Discovery, expanding the details showed the progress of requesting pieces.
parg

Posts: 2,698
Registered: 11/13/07
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Dec 3, 2009 5:35 PM   in response to: dobri in response to: dobri
 
Click to report abuse...
oops, well spotted - fixed in next beta, thanks :)
dobri

Posts: 13
Registered: 11/30/09
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Dec 3, 2009 6:12 PM   in response to: Washii in response to: Washii
 
Click to report abuse...
When you expand Details you get a list that looks like this:

Downloading From :magnet:?xt=urn:btih:TDC4GYOQXZPSUB7KR6SQKLS2USAJPZ7W
Waiting for DDB initialization...
Searching...
Found 2105FDD2...
Found 7A9B7867...
Found D6AB8EB3...
Found D0A7DF59...
Found D25A2D70...
Found 814EF5C0...

However during this time the progress bar stays at 0, both in the new and in the old version of the code.

The progress is calculated on line 593 in org.gudy.azureus2.core3.torrentdownloader.impl.TorrentDownloaderImpl
like this:

this.percentDone = (100 * this.readTotal) / size;

The IO is done in chunks since the buf size is 1020 and the torrent is about 45K. So technically the status should update 45 times. But I used to see 4 to 5 steps in the progress bar, sometimes even less. But now we are reading it from memory (PipedInputStream) and writing it to a File (which is memory again, since the OS is going to delay writing it to the actual file. And the Antivirus is not in the chain to add latency, so it is very likely that we can get to 100% and then close the window before it has an opportunity to refresh even once.
dobri

Posts: 13
Registered: 11/30/09
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Dec 3, 2009 8:48 PM   in response to: parg in response to: parg
 
Click to report abuse...
Now both the magnets and the scroll bar work.
However I noticed 2 things:
1. The operation now takes much longer than compared to the previous version (with the broken scrollbar). This may be perception, since I did not actually measure it, but seems significantly slower.
2. I am getting very consistent slow response on this one: magnet:?xt=urn:btih:TDC4GYOQXZPSUB7KR6SQKLS2USAJPZ7W
It goes to 10%, then hangs there for like 30 seconds, then it says (in the details) that it re-requests it and then it continues. Once it had to re-request it twice. I did this consistently 3 times in a row - way to consistent to be a network issue. It is true that I am behind a 2 NAT routers without any ports open on them, but it used to be much faster with the same torrent several hours ago.
Washii

Posts: 5,261
Registered: 11/14/07
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Dec 3, 2009 9:39 PM   in response to: dobri in response to: dobri
 
Click to report abuse...
Worked right-off for me. How long between tries were you doing this?
dobri

Posts: 13
Registered: 11/30/09
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Dec 4, 2009 5:20 AM   in response to: Washii in response to: Washii
 
Click to report abuse...
It goes to 10% in the first 15 seconds, than stays there for another 50, than it says requesting the whole thing from certain place, than it says re-requesting from the same place, and then it seems to get it from a different place. End-to-end is 80 seconds.
I abort at this point and retry from the beginning. This time it takes 17 seconds, it still pauses at 10% on the 10th second.
On the third try it pauses at 10% on the 10th second, retries around 40 seconds later, total time 57 seconds.
During these long pauses it shows a lot of dead things (assuming places/peers).
parg

Posts: 2,698
Registered: 11/13/07
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Dec 4, 2009 7:17 AM   in response to: dobri in response to: dobri
 
Click to report abuse...
Magnet downloads are done via UDP from clients found in the DHT. It looks up peers, selects one that seems to be alive, then tries to download from it. If it fails then it either retries or moves on to another peer if there are lots available. The first 10% progress shown corresponds to findng some peers (as opposed to actually downloading the torrent). The remaining 90% corresponds to actual download.

Also there may be an attempt to perform NAT traversal to the peer if it is firewalled before downloading.

All this leads to non-deterministic behaviour for small swarms - if you're lucky and hit a fast, non NAT-d peer, then it'll download quickly. If its a small swarm with firewalled/recently shutdown peers, then it will in general take longer.

Of course I could request from multiple peers in parallel, but this is a complexity I have so far ignored.
dobri

Posts: 13
Registered: 11/30/09
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Dec 4, 2009 7:56 AM   in response to: parg in response to: parg
 
Click to report abuse...
I don't know how exactly it works and why is it happening, but I tried to go back to the version with the broken progress bar. In that version this magnet is loaded consistently in about 15 seconds total and the other one that I was trying comes in 2 seconds or less. Even the list of nodes found in the Details view comes at much faster rate, if that gives you any clue.
parg

Posts: 2,698
Registered: 11/13/07
Re: Cannot add torrents via magnet after upgrade to Windows 7
Posted: Dec 4, 2009 9:08 AM   in response to: dobri in response to: dobri
 
Click to report abuse...
ah, yes, its a UI/CPU thing - I fixed stuff up to process events correctly (before it was skipping a load of updates) but the UI implementation for displaying these sucks - I'll take a look - thanks
Legend
Master: 800 - 9999 pts
Expert: 400 - 799 pts
Advanced: 200 - 399 pts
Intermediate: 100 - 199 pts
Beginner: 50 - 99 pts
Newbie: 0 - 49 pts
Vuze Staff Member
Vuze Community Moderator
Helpful Answer (3 pts)
Correct Answer (5 pts)

Point your RSS reader here for a feed of the latest messages in all forums