fngrab.py version 0.0.3 Copyright Rob Cakebread Released under the GNU Public License ############## What it is: ############## fngrab.py is a replacement for the official Freenet project's freenet_request(Un*x) and frequest (Windows). It has additional features such as retrying failed requests with incrementel hops to live. There is also better support for in-Freenet key index searching. The official Freenet client doesn't let you specify a range or set the threshold for the number of fails to quit before giving up. If a range isn't specified, fngrab.py starts searching the index at 1 and continues until the number of fails equals the threshold. You can also specify a file containing a list of keys to download. The file format is simply a text file containing a key on each line. When you download keys listed in a file, it derives the filenames from the keys: freenet:KSK@/mp3/britney/I_was_born_to_make_Rob_happy.mp3 would be saved as I_was_born_to_make_Rob_happy.mp3 freenet:KSK@test would simply be saved as test fngrab.py is also great for verifying a list of keys. It aborts after downloading a few kilobytes, but NOTE: because of the way Freenet works internally your node will continue to download the files into your local node. This isn't necessarily a bad thing, because you can go back and download the goods one much quicker. A large list of keys will obviously consume a large amount of bandwidth. The only way to avoid downloading the data to your node would be to actually kill your freenet node after each download starts. I may implement this feature in the future. fngrab.py uses XML-RPC, available in Freenet version 0.3.9.1 or later. By default it uses localhost and port 6690, which are the standard values in .freenetrc(Un*x) and freenet.ini (Windows). ####################################################################### TODO: Multiple simultaneous requests via threading Allow naming of files in batch mode with -f: KSK@test // saturday.txt CHK@aJ3IWErjweoirjOIJ // groovy.mp3 Smarter filenames from keys (remove more funky characters) Add option to specify output directory Installation script ####################################################################### ############## Examples: ############## (Note for Windows users: You may need to preface fngrab with 'python' in these examples) fngrab.py test Requests freenet:KSK@test from localhost port 6690, prints it to stdout fngrab.py -o test.txt test Requests freenet:KSK@test from localhost port 6690 and saves it in a file named test.txt fngrab.py -f keys.txt -d /tmp Opens file named keys.txt and attempts to download each key and save the files in /tmp. The filenames are derived from each key fngrab.py -f keys.txt -r 4 -i 20 -h 10 Requests all keys in keys.txt, retrying failed downloads up to 4 times, incrementing the htl by 20 each time, with a starting htl of 10 fngrab.py -t -f keys.txt -o keys.out Requests all keys in file keys.txt and tests to see if they exist. Writes results of test in file keys.out. Each file is not saved, although your node will keep downloading them. fngrab.py -x freegle -g 4,42 -l 5 Request keys from in-Freenet incremental index named 'freegle-n', with a range of 4 through 42, giving up after 5 failures. Keys are dumped to stdout. fngrab.py -x snarfoo -z 3 Request keys from in-Freenet date based index named 'snarfoo', from three days ago. Omit the -z option to get index for current day. ######################## Usage: ######################## fngrab.py [options] key Options: -d, --dir directory Saves files in directory specified. Use with -f -f, --file filename Download all keys in file. One key per line, plain text -g, --range n,n Used with -index to specify range: 4,15 -h, --htl n Hops to Live. Default: 20 -i, --increment n Number to increase htl on retry (-r, --retry) -l, --threshold n Total nbr of fails before quitting (-x) default: 3 -n, --number n Number of simultaneous requests (not implemented) -o, --output filename Write to file with filename -o name -p, --port n XML-RPC port. Default: 6690 -r, --retry n Number of attempts to retry on fail -s, --server server Server address. Default: 127.0.0.1 -t, --test Test only, good for verifying lists of keys with -f -v, --verbose Verbose output, shows filesize in chunks as received. -V, --version Prints version, copyright info, then exits. -x, --index name Requests range (-g) of keys from in-Freenet index If an output file is not specified (-o, --output) the program will dump the contents to stdout. The -t test option aborts the download after beginning.