fngrab.py version 0.0.3

Copyright Rob Cakebread
Released under the GNU Public License

##############
What it is:
##############

fngrab.py is a replacement for the official Freenet project's
freenet_request(Un*x) and frequest (Windows). It has additional
features such as retrying failed requests with incrementel hops to live.

There is also better support for in-Freenet key index searching.
The official Freenet client doesn't let you specify a range or
set the threshold for the number of fails to quit before giving up.
If a range isn't specified, fngrab.py starts searching the index
at 1 and continues until the number of fails equals the threshold.

You can also specify a file containing a list of keys to download.
The file format is simply a text file containing a key on each line.
When you download keys listed in a file, it derives the filenames
from the keys:

freenet:KSK@/mp3/britney/I_was_born_to_make_Rob_happy.mp3
would be saved as I_was_born_to_make_Rob_happy.mp3

freenet:KSK@test
would simply be saved as test


fngrab.py is also great for verifying a list of keys. It aborts after
downloading a few kilobytes, but NOTE: because of the way Freenet works
internally your node will continue to download the files into your local
node. This isn't necessarily a bad thing, because you can go back and download
the goods one much quicker. A large list of keys will obviously consume
a large amount of bandwidth. The only way to avoid downloading the data
to your node would be to actually kill your freenet node after each
download starts. I may implement this feature in the future.

fngrab.py uses XML-RPC, available in Freenet version 0.3.9.1 or later. 

By default it uses localhost and port 6690, which are the standard
values in .freenetrc(Un*x) and freenet.ini (Windows).

#######################################################################
TODO:
      Multiple simultaneous requests via threading

      Allow naming of files in batch mode with -f:
         KSK@test // saturday.txt
         CHK@aJ3IWErjweoirjOIJ // groovy.mp3

      Smarter filenames from keys (remove more funky characters)

      Add option to specify output directory

      Installation script

#######################################################################


##############
Examples:
##############
(Note for Windows users: You may need to preface fngrab with 'python'
 in these examples)

fngrab.py test

  Requests freenet:KSK@test from localhost port 6690, prints it to stdout

fngrab.py -o test.txt test

  Requests freenet:KSK@test from localhost port 6690 and saves it in a
  file named test.txt

fngrab.py -f keys.txt -d /tmp

  Opens file named keys.txt and attempts to download each key and save
  the files in /tmp. The filenames are derived from each key

fngrab.py -f keys.txt -r 4 -i 20 -h 10

  Requests all keys in keys.txt, retrying failed downloads up to 4 times,
  incrementing the htl by 20 each time, with a starting htl of 10

fngrab.py -t -f keys.txt -o keys.out
  Requests all keys in file keys.txt and tests to see if they exist.
  Writes results of test in file keys.out. Each file is not saved,
  although your node will keep downloading them.

fngrab.py -x freegle -g 4,42 -l 5
  Request keys from in-Freenet incremental index named 'freegle-n', with a range
  of 4 through 42, giving up after 5 failures. Keys are dumped to stdout.

fngrab.py -x snarfoo -z 3
  Request keys from in-Freenet date based index named 'snarfoo', from three
  days ago. Omit the -z option to get index for current day.

########################
Usage:
########################

fngrab.py [options] key

Options:

-d, --dir directory    Saves files in directory specified. Use with -f
-f, --file filename    Download all keys in file. One key per line, plain text
-g, --range n,n        Used with -index to specify range: 4,15
-h, --htl n            Hops to Live. Default: 20
-i, --increment n      Number to increase htl on retry (-r, --retry)
-l, --threshold n      Total nbr of fails before quitting (-x) default: 3
-n, --number n         Number of simultaneous requests (not implemented)
-o, --output filename  Write to file with filename -o name
-p, --port n           XML-RPC port. Default: 6690
-r, --retry n          Number of attempts to retry on fail
-s, --server server    Server address. Default: 127.0.0.1
-t, --test             Test only, good for verifying lists of keys with -f
-v, --verbose          Verbose output, shows filesize in chunks as received.
-V, --version          Prints version, copyright info, then exits.
-x, --index name       Requests range (-g) of keys from in-Freenet index

If an output file is not specified (-o, --output) the program will
dump the contents to stdout.

The -t test option aborts the download after beginning.