com.limegroup.gnutella.downloader
Class ManagedDownloader

java.lang.Object
  extended bycom.limegroup.gnutella.downloader.ManagedDownloader
All Implemented Interfaces:
BandwidthTracker, Downloader, java.io.Serializable
Direct Known Subclasses:
MagnetDownloader, RequeryDownloader, ResumeDownloader

public class ManagedDownloader
extends java.lang.Object
implements Downloader, java.io.Serializable

A smart download. Tries to get a group of similar files by delegating to HTTPDownloader objects. Does retries and resumes automatically. Reports all changes to a DownloadManager. This class is thread safe.

Smart downloads can use many policies, and these policies are free to change as allowed by the Downloader specification. This implementation provides swarmed downloads, the ability to download copies of the same file from multiple hosts. See the accompanying white paper for details.

Subclasses may refine the requery behavior by overriding the nextRequeryTime, newRequery(n), allowAddition(..), and addDownload(..) methods. MagnetDownloader also redefines the tryAllDownloads(..) method to handle default locations, and the getFileName() method to specify the completed file name.

Subclasses that pass this RemoteFileDesc arrays of size 0 MUST override the getFileName method, otherwise an assert will fail.

This class implements the Serializable interface but defines its own writeObject and readObject methods. This is necessary because parts of the ManagedDownloader (e.g., sockets) are inherently unserializable. For this reason, serializing and deserializing a ManagedDownloader M results in a ManagedDownloader M' that is the same as M except it is unconnected. Furthermore, it is necessary to explicitly call initialize(..) after reading a ManagedDownloader from disk.

See Also:
Serialized Form

Field Summary
protected static java.lang.String UNKNOWN_FILENAME
          The value of an unknown filename - potentially overridden in subclasses
 
Fields inherited from interface com.limegroup.gnutella.Downloader
ABORTED, COMPLETE, CONNECTING, CORRUPT_FILE, COULDNT_MOVE_TO_LIBRARY, DOWNLOADING, GAVE_UP, HASHING, QUEUED, REMOTE_QUEUED, SAVING, WAITING_FOR_CONNECTIONS, WAITING_FOR_RESULTS, WAITING_FOR_RETRY, WAITING_FOR_USER
 
Constructor Summary
ManagedDownloader(RemoteFileDesc[] files, IncompleteFileManager ifc)
          Creates a new ManagedDownload to download the given files.
 
Method Summary
 boolean acceptDownload(java.lang.String file, java.net.Socket socket, int index, byte[] clientGUID)
          Accepts a push download.
 boolean addDownload(RemoteFileDesc rfd, boolean cache)
          Attempts to add the given location to this.
protected  boolean addDownloadForced(RemoteFileDesc rfd, boolean cache)
          Like addDownload, but doesn't call allowAddition(..).
protected  boolean allowAddition(RemoteFileDesc other)
          Returns true if 'other' should be accepted as a new download location.
 boolean conflicts(java.io.File incFile)
          Returns true if this is using (or could use) the given incomplete file.
 boolean conflicts(RemoteFileDesc other)
          Returns true if 'other' could conflict with one of the files in this.
 void discardCorruptDownload(boolean delete)
          either treats a corrupt file as normal file and saves it, or discards the corruptFile, depending on the value of delete.
 void finish()
          Cleans up information before this downloader is removed from memory.
 java.lang.String getAddress()
          Returns the last address that this tried to connect to, or null if it hasn't tried any.
 int getAmountRead()
          Return the amount read.
 float getAverageBandwidth()
          returns the summed average of the downloads
 RemoteFileDesc getBrowseEnabledHost()
          Returns a browse-enabled Endpoint instance for this Downloader.
 int getBusyHostCount()
           
 Endpoint getChatEnabledHost()
          Returns a chat-enabled Endpoint instance for this Downloader.
 int getContentLength()
          Returns the size of this file in bytes, i.e., the total amount to download.
 java.io.File getDownloadFragment()
          If this download is not yet complete, returns a copy of the first contiguous fragment of the incomplete file.
protected  long[] getFailedState(boolean deserialized, long timeSpentWaiting)
          This method is called when 1) all downloads sources failed 2) there are no busy hosts 3) there is no room for a requery Subclasses should override this method if they want to enforce special behavior before going to the GAVE_UP state.
 java.lang.String getFileName()
          Returns the name of the current or last file this is downloading, or null in the rare case that this has no more files to download.
 java.util.Iterator getHosts()
          Returns the locations from which this is currently downloading, as an iterator of Endpoint.
 float getMeasuredBandwidth()
          Returns the throughput of this in kilobytes/sec (KB/s) between the last two calls to measureBandwidth, or 0.0 if unknown.
 int getNumberOfAlternateLocations()
          Returns the number of alternate locations that this download is using.
 int getNumDownloaders()
           
 int getPossibleHostCount()
          Returns the amount of other hosts this download can possibly use.
protected  int getQueryCount(boolean deserializedFromDisk)
          We need to offer this to subclasses to override because they might have specific behavior when deserialized from disk.
 int getQueuedHostCount()
           
 java.lang.String getQueuePosition()
          Returns the position of the download on the uploader, relavent only if the downloader is queueud.
 int getRemainingStateTime()
          Returns an upper bound on the amount of time this will stay in the current state, in seconds.
 int getRetriesWaiting()
          Returns the number of retries this is waiting for.
 int getState()
          Accessors that delegate to dloader.
 java.lang.String getVendor()
          Returns the vendor of the last downloading host.
 boolean hasBrowseEnabledHost()
          Returns whether or not there is a browse-enabled host available for this Downloader.
 boolean hasChatEnabledHost()
          Returns whether or not there is a chat-enabled host available for this Downloader.
protected  boolean hasRFD()
          Certain subclasses would like to know whether we have at least one good RFD.
 void initialize(DownloadManager manager, FileManager fileManager, ActivityCallback callback, boolean deserialized)
          Initializes a ManagedDownloader read from disk.
protected  void initializeIncompleteFile(java.io.File incFile)
          If incompleteFile has already been set, i.e., because a download is in progress, does nothing.
 void measureBandwidth()
          Measures the data throughput since the last call to measureBandwidth.
protected  QueryRequest newRequery(int numRequeries)
          Returns a new QueryRequest for requery purposes.
protected  boolean pauseForRequery(int numRequeries, boolean deserializedFromDisk)
          This dictates whether this downloader should wait for user input before spawning a Requery.
 boolean resume()
          Resumes this.
protected  boolean shouldInitAltLocs(boolean deserializedFromDisk)
          Subclasses should override this method when necessary.
 void stop()
          Stops this.
protected  void tryAllDownloads(boolean deserializedFromDisk)
          Actually does the download, finding duplicate files, trying all locations, resuming, waiting, and retrying as necessary.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

UNKNOWN_FILENAME

protected static final java.lang.String UNKNOWN_FILENAME
The value of an unknown filename - potentially overridden in subclasses

See Also:
Constant Field Values
Constructor Detail

ManagedDownloader

public ManagedDownloader(RemoteFileDesc[] files,
                         IncompleteFileManager ifc)
Creates a new ManagedDownload to download the given files. The download does not start until initialize(..) is called, nor is it safe to call any other methods until that point.

Parameters:
files - the list of files to get. This stops after ANY of the files is downloaded.
ifc - the repository of incomplete files for resuming
Method Detail

initialize

public void initialize(DownloadManager manager,
                       FileManager fileManager,
                       ActivityCallback callback,
                       boolean deserialized)
Initializes a ManagedDownloader read from disk. Also used for internally initializing or resuming a normal download; there is no need to explicitly call this method in that case. After the call, this is in the queued state, at least for the moment.

Parameters:
deserialized - True if this downloader is being initialized after being read from disk, false otherwise.

initializeIncompleteFile

protected void initializeIncompleteFile(java.io.File incFile)
If incompleteFile has already been set, i.e., because a download is in progress, does nothing. Otherwise sets incompleteFile and commonOutFile. Subclasses may override this to force the initial progress to be non-zero.


conflicts

public boolean conflicts(RemoteFileDesc other)
Returns true if 'other' could conflict with one of the files in this. In other words, if this.conflicts(other)==true, no other ManagedDownloader should attempt to download other.


conflicts

public boolean conflicts(java.io.File incFile)
Returns true if this is using (or could use) the given incomplete file.


newRequery

protected QueryRequest newRequery(int numRequeries)
                           throws CantResumeException
Returns a new QueryRequest for requery purposes. Subclasses may wish to override this to be more or less specific. Note that the requery will not be sent if global limits are exceeded.

The default implementation includes all non-trivial keywords found in all RemoteFileDesc's in this, i.e., the INTERSECTION of all file names. A keyword is "non-trivial" if it is not a number of a common English article (e.g., "the"), a number (e.g., "2"), or the file extension. The query also includes all hashes for all RemoteFileDesc's, i.e., the UNION of all hashes. Since there are no more AUTOMATIC requeries, subclasses are advised to stop using createRequery(...). All attempts to 'requery' the network is spawned by the user, so use createQuery(...) . The reason we need to use createQuery is because DownloadManager.sendQuery() has a global limit on the number of requeries sent by LW (as IDed by the guid), but it allows normal queries to always be sent.

Parameters:
numRequeries - the number of requeries that have already happened
Returns:
a new QueryRequest for making the requery
Throws:
CantResumeException - if this doesn't know what to search for

getQueryCount

protected int getQueryCount(boolean deserializedFromDisk)
We need to offer this to subclasses to override because they might have specific behavior when deserialized from disk. for example, RequeryDowloader should return a count of 0 upon deserialization, but 1 if started from scratch.


pauseForRequery

protected boolean pauseForRequery(int numRequeries,
                                  boolean deserializedFromDisk)
                           throws java.lang.InterruptedException
This dictates whether this downloader should wait for user input before spawning a Requery. Subclasses should override with desired behavior as necessary.

Parameters:
numRequeries - The number of requeries sent so far.
deserializedFromDisk - If the downloader was deserialized from a snapshot. May be useful for subclasses.
Returns:
true if we the pause was broken because of new results. false if the user woke us up.
Throws:
java.lang.InterruptedException

shouldInitAltLocs

protected boolean shouldInitAltLocs(boolean deserializedFromDisk)
Subclasses should override this method when necessary. If you return false, then AltLocs are not initialized from the incomplete file upon invocation of tryAllDownloads. The true case can be used when the partial file is being shared through PFS and we've learned about AltLocs we want to use.


allowAddition

protected boolean allowAddition(RemoteFileDesc other)
Returns true if 'other' should be accepted as a new download location.


addDownload

public boolean addDownload(RemoteFileDesc rfd,
                           boolean cache)
Attempts to add the given location to this. If rfd is accepted, this will terminate after downloading rfd or any of the other locations in this. This may swarm some file from rfd and other locations.

This method only adds rfd if allowAddition(rfd). Subclasses may wish to override this protected method to control the behavior.

Parameters:
rfd - a new download candidate. Typically rfd will be similar or same to some entry in this, but that is not required.
Returns:
true if rfd has been added. In this case, the caller should not offer rfd to another ManagedDownloaders.

addDownloadForced

protected final boolean addDownloadForced(RemoteFileDesc rfd,
                                          boolean cache)
Like addDownload, but doesn't call allowAddition(..). If cache is false, the RFD is not added to allFiles, but is added to the appropriate bucket. If the RFD matches one already in allFiles, the new one is NOT added to allFiles, but IS added the appropriate bucket if and only if a matching RFD is not currently in the bucket. If the file is ultimately added to buckets, either reqLock is released or this is notified. This ALWAYS returns true, because the download is either allowed or silently ignored (because we're already downloading or going to attempt to download from the host described in the RFD).


acceptDownload

public boolean acceptDownload(java.lang.String file,
                              java.net.Socket socket,
                              int index,
                              byte[] clientGUID)
                       throws java.io.IOException
Accepts a push download. If this chooses to download the given file (with given index and clientGUID) from socket, returns true. In this case, the caller may not make any modifications to the socket. If this rejects the given file, returns false without modifying this or socket. If this could has problems with the socket, throws IOException. In this case the caller should close the socket. Non-blocking.

Throws:
java.io.IOException

stop

public void stop()
Description copied from interface: Downloader
Stops this. If the download is already stopped, does nothing.

Specified by:
stop in interface Downloader

resume

public boolean resume()
               throws AlreadyDownloadingException
Description copied from interface: Downloader
Resumes this. If the download is GAVE_UP, tries all locations again and returns true. If WAITING_FOR_RETRY, forces the retry immediately and returns true. If some other downloader is currently downloading the file, throws AlreadyDowloadingException. If WAITING_FOR_USER, then launches another query. Otherwise does nothing and returns false.

Specified by:
resume in interface Downloader
Throws:
AlreadyDownloadingException

getDownloadFragment

public java.io.File getDownloadFragment()
Description copied from interface: Downloader
If this download is not yet complete, returns a copy of the first contiguous fragment of the incomplete file. (The copying helps prevent file locking problems.) Returns null if the download hasn't started or the copy failed. If the download is complete, returns the saved file.

Specified by:
getDownloadFragment in interface Downloader
Returns:
the copied file fragment, saved file, or null

getFailedState

protected long[] getFailedState(boolean deserialized,
                                long timeSpentWaiting)
This method is called when 1) all downloads sources failed 2) there are no busy hosts 3) there is no room for a requery Subclasses should override this method if they want to enforce special behavior before going to the GAVE_UP state. NOTE: Only the following states are can be preemptively woken up due to new results - WAITING_FOR_RETRY, WAITING_FOR_RESULTS, and GAVE_UP.

Parameters:
deserialized - true if this downloader was initialized from disk, false if it is brand new.
timeSpentWaiting - the millisecond time that the downloader has spent in the failed state.
Returns:
two longs - long[0] is the state the downloader should go in. long[1] is the time the downloader should spend in state long[0]. if long[1] < 1, this return value is ignored.

finish

public void finish()
Cleans up information before this downloader is removed from memory.


tryAllDownloads

protected void tryAllDownloads(boolean deserializedFromDisk)
Actually does the download, finding duplicate files, trying all locations, resuming, waiting, and retrying as necessary. Also takes care of moving file from incomplete directory to save directory and adding file to the library. Called from dloadManagerThread.


getNumberOfAlternateLocations

public int getNumberOfAlternateLocations()
Returns the number of alternate locations that this download is using.

Specified by:
getNumberOfAlternateLocations in interface Downloader

getPossibleHostCount

public int getPossibleHostCount()
Returns the amount of other hosts this download can possibly use.

Specified by:
getPossibleHostCount in interface Downloader
Returns:
the number of possible hosts for this download

getBusyHostCount

public int getBusyHostCount()
Specified by:
getBusyHostCount in interface Downloader
Returns:
the number of hosts we tried which were busy. We will try these later

getQueuedHostCount

public int getQueuedHostCount()
Specified by:
getQueuedHostCount in interface Downloader
Returns:
the number of hosts we are remotely queued on.

discardCorruptDownload

public void discardCorruptDownload(boolean delete)
Description copied from interface: Downloader
either treats a corrupt file as normal file and saves it, or discards the corruptFile, depending on the value of delete.

Specified by:
discardCorruptDownload in interface Downloader

getState

public int getState()
Accessors that delegate to dloader. Synchronized because dloader can change.

Specified by:
getState in interface Downloader

getRemainingStateTime

public int getRemainingStateTime()
Description copied from interface: Downloader
Returns an upper bound on the amount of time this will stay in the current state, in seconds. Returns Integer.MAX_VALUE if unknown.

Specified by:
getRemainingStateTime in interface Downloader

getFileName

public java.lang.String getFileName()
Description copied from interface: Downloader
Returns the name of the current or last file this is downloading, or null in the rare case that this has no more files to download. (This might happen if this has been stopped.)

Specified by:
getFileName in interface Downloader

hasRFD

protected boolean hasRFD()
Certain subclasses would like to know whether we have at least one good RFD.


getContentLength

public int getContentLength()
Description copied from interface: Downloader
Returns the size of this file in bytes, i.e., the total amount to download.

Specified by:
getContentLength in interface Downloader

getAmountRead

public int getAmountRead()
Return the amount read. The return value is dependent on the state of the downloader. If it is corrupt, it will return how much it tried to read before noticing it was corrupt. If it is hashing, it will return how much of the file has been hashed. All other times it will return the amount downloaded. All return values are in bytes.

Specified by:
getAmountRead in interface Downloader

getAddress

public java.lang.String getAddress()
Description copied from interface: Downloader
Returns the last address that this tried to connect to, or null if it hasn't tried any. Useful primarily for CONNECTING.

Specified by:
getAddress in interface Downloader

getHosts

public java.util.Iterator getHosts()
Description copied from interface: Downloader
Returns the locations from which this is currently downloading, as an iterator of Endpoint. If this is swarming, may return multiple addresses. Result meaningful only in the DOWNLOADING state.

Specified by:
getHosts in interface Downloader

getChatEnabledHost

public Endpoint getChatEnabledHost()
Description copied from interface: Downloader
Returns a chat-enabled Endpoint instance for this Downloader.

Specified by:
getChatEnabledHost in interface Downloader

hasChatEnabledHost

public boolean hasChatEnabledHost()
Description copied from interface: Downloader
Returns whether or not there is a chat-enabled host available for this Downloader.

Specified by:
hasChatEnabledHost in interface Downloader
Returns:
true if there is a chat-enabled host for this Downloader, false otherwise

getBrowseEnabledHost

public RemoteFileDesc getBrowseEnabledHost()
Description copied from interface: Downloader
Returns a browse-enabled Endpoint instance for this Downloader.

Specified by:
getBrowseEnabledHost in interface Downloader

hasBrowseEnabledHost

public boolean hasBrowseEnabledHost()
Description copied from interface: Downloader
Returns whether or not there is a browse-enabled host available for this Downloader.

Specified by:
hasBrowseEnabledHost in interface Downloader
Returns:
true if there is a browse-enabled host for this Downloader, false otherwise

getQueuePosition

public java.lang.String getQueuePosition()
Description copied from interface: Downloader
Returns the position of the download on the uploader, relavent only if the downloader is queueud.

Specified by:
getQueuePosition in interface Downloader

getNumDownloaders

public int getNumDownloaders()

getVendor

public java.lang.String getVendor()
Description copied from interface: Downloader
Returns the vendor of the last downloading host.

Specified by:
getVendor in interface Downloader

getRetriesWaiting

public int getRetriesWaiting()
Description copied from interface: Downloader
Returns the number of retries this is waiting for. Result meaningful on in WAIT_FOR_RETRY state.

Specified by:
getRetriesWaiting in interface Downloader

measureBandwidth

public void measureBandwidth()
Description copied from interface: BandwidthTracker
Measures the data throughput since the last call to measureBandwidth. This value can be read by calling getMeasuredBandwidth.

Specified by:
measureBandwidth in interface BandwidthTracker

getMeasuredBandwidth

public float getMeasuredBandwidth()
Description copied from interface: BandwidthTracker
Returns the throughput of this in kilobytes/sec (KB/s) between the last two calls to measureBandwidth, or 0.0 if unknown.

Specified by:
getMeasuredBandwidth in interface BandwidthTracker

getAverageBandwidth

public float getAverageBandwidth()
returns the summed average of the downloads

Specified by:
getAverageBandwidth in interface BandwidthTracker