mafintosh / peerwiki Goto Github PK

all of wikipedia on bittorrent

License: MIT License

JavaScript 100.00%

peerwiki's Introduction

peerwiki

BROWSE ALL OF WIKIPEDIA USING BITTORENT

npm install -g peerwiki

Usage

peerwiki 9090 # starts a peerwiki server on port 9090

To get more debug output do

DEBUG=peerwiki peerwiki 9090

This can useful the first time you start browsing since it needs to some download some shared static assets which can result in a added latency

To use a pregenerated index (will speed up page load) do

peerwiki 9090 --use-index

And open a browser on http://localhost:9090/BitTorrent

Cache

When downloading articles they are cached on your local file system in ./peerwiki.

Programmatic usage

var peerwiki = require('peerwiki')
var wiki = peerwiki(function() {
  // wiki is ready

  // fetch the BitTorrent article metadata from other peers
  wiki.findEntryByUrl('html/B/i/t/T/BitTorrent', function(err, result) {
    console.log(result)
  })

  // fetch the actual article from other peers
  wiki.findBlobByUrl('html/B/i/t/T/BitTorrent', function(err, buf) {
    console.log(buf)
  })

  wiki.listen(9091) // listen for p2p connections on port 9091
})

License

MIT

peerwiki's People

Contributors

Stargazers

Watchers

Forkers

kesla wizek eiriklv ashumeow addaleax mariotristan imclab jhonnrodr jordietyahii rnbwd packetlost ecoblockchain hexagon6 hummans suryatmodulus

peerwiki's Issues

What's this ip?

https://github.com/mafintosh/peerwiki/blob/master/server.js#L48

request('http://178.62.147.171/'+url, {json:true}, function(err, response) {

npm ERR! [email protected] install: node-gyp rebuild
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is most likely a problem with the xz package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node-gyp rebuild
npm ERR! You can get their info via:
npm ERR! npm owner ls xz
npm ERR! There is likely additional logging output above.
npm ERR! System Darwin 13.4.0

Python >= v2.5.0 & < 3.0.0 requirement

As with this Stack Overflow Question, I got the following error when trying to install this with Python v3.4.2:

gyp ERR! stack Error: Python executable "python" is v3.4.2, which is not supported by gyp.
gyp ERR! stack You can pass the --python switch to point to Python >= v2.5.0 & < 3.0.0.

I installed python 2.7.3 and got around this, but it would be useful if this was mentioned in the README.

The README for graylog2-web-interface does something similar.

dependency on coffee-script

Also it doesn't seem to be needed - the xy-dependency doesn't include any .coffee-files from what I can tell - so what's up with that?

Nothing to see at http://localhost:9090

Hi,

I am trying to test it out...
I successfully installed the app globally via: npm install -g peerwiki and then run via: DEBUG=peerwiki peerwiki 9090 or DEBUG=peerwiki peerwiki 9090 --use-index in either way I do not see anything in Google Chrome at: http://localhost:9090/BitTorrent or http://localhost:9090
Getting Error code: ERR_CONNECTION_REFUSED.

Here is the log:

EBUG=peerwiki peerwiki 9090
Verifying previously downloaded data...
  peerwiki engine is ready +0ms
  peerwiki fetching required wikipedia header... +2ms
  peerwiki connected to 0/0 (0 B/s) peers +5s
  peerwiki connected to 0/0 (0 B/s) peers +5s

and so on...

[Feature Request] Make on-demand downloading an option

I noticed that my ./peerwiki folder keeps growing even if I'm not browsing wiki pages through peerwiki.

Is this because all of wikipedia is being download in the background?

If that is the case, would it be possible to have on-demand downloading of only the requested pages + resources?

The eager/lazy behaviour could perhaps be controlled through a flag: --on-demand

Not for Windows

I'm on Windows 7 64-bit with python v2.7.4 and npm v1.4.9

After calling npm install -g peerwiki, it gets some of the way but gives the following error because obviously Windows (native DOS prompt) doesn't understand what the sh extension is for. So could you mention that its not for Windows (yet) in the README.

'sh' is not recognized as an internal or external command,
operable program or batch file.
gyp: Call to 'sh liblzma-config.sh C:\ProgramData\chocolatey\lib\nodejs.commandline.0.12.0\tools\node_modules\peerwiki\node_modules\lzma-native/build C:\ProgramData\choco
latey\lib\nodejs.commandline.0.12.0\tools\node_modules\peerwiki\node_modules\lzma-native/deps/xz-5.0.6.tar.bz2' returned exit status 1. while trying to load binding.gyp