Git Product home page Git Product logo

cachebrowser's Introduction

Merged Into MassBrowser

CacheBrowser has been merged into our new censorship circumvention tool called MassBrowser. Please visit the MassBrowser project's website and its git repository.

CacheBrowser

The current CacheBrowser software is a research prototype implementation, and is not yet ready for end-users. If you are a researcher or developer check it out and give us feedback, otherwise please be patient while we work on an end-user version of the code. Thanks for your understanding and support!

https://cachebrowser.net

Installation

pip install cachebrowser

Running CacheBrowser

To run the CacheBrowser process simply enter the cachebrowser command

cachebrowser

You can then use CacheBrowser by using the CacheBrowser plugin for your browser or by setting the HTTP and HTTPS proxy on your browser to localhost:8080

Bootstrapping

In order to be able to browse a page using CacheBrowser, the page must first be bootstrapped. What this means is that CacheBrowser must first find out what CDN the page is hosted on and also obtain some edge server addresses for that CDN.

CacheBrowser supports local and remote bootstrapping sources. Local bootstrapping sources are files in YAML format which provide bootstrapping information for hosts and CDNs. Remote bootstrapping sources are RESTful services which provide bootstrapping information through HTTP requests.

Configuring Bootstrapping Sources

Any number of local or remote bootstrapping sources may be defined in the CacheBrowser configuration file. When bootstrapping a host or host, CacheBrowser will go through the provided sources in the specified order until the information is found.

Bootstrapping sources can be specified in the configuration file using the bootstrapper_sources key, for example:

boostrapper_sources:
	- type: local
	  path: local_bootstrapper.yaml
	- type: remote
	  url: https://bootstrap.cachebrowser.net/api

Local Boostrapping Files

A local bootstrapping file is a YAML file containing a list of Host or CDN information.

For example:

- type: cdn
  id: akamai
  name: Akamai
  edge_servers:
    - 23.208.91.198
    - 23.218.210.7

- type: host
  name: www.nbc.com
  ssl: false
  cdn: akamai

- type: host
  name: www.bloomberg.com
  ssl: false
  cdn: akamai

For each CDN provided in the bootstrapping file the following information should be provided:

Key Value
type cdn
id A unique ID to be used for the CDN
name The CDN Name
edge_servers List of edge server addresses for the CDN

For each Host provided in the bootstrapping file the following information should be provided:

Key Value
type host
name The hostname
cdn The ID of the CDN which provides this host
ssl Whether the pages supports HTTPS connections or not

CacheBrowser Commands

You could also run a command with CacheBrowser:

cachebrowser <command>

These are the valid commands you could give:

Command Description
bootstrap Bootstrap
get url [target] Retrieve url using CacheBrowser. If target is specified CacheBrowser uses the given target. If not, CacheBrowser uses the bootstrapped target if the host has been bootstrapped or makes a normal request to the host if not.
host list Lists the bootstrapped hosts
host add Add host info
cdn list Lists the bootstrapped CDNs
cdn add Add host info

Example

cachebrowser bootstrap www.nbc.com
cachebrowser get http://www.nbc.com

cachebrowser get https://www.istockphoto.com 69.31.76.91

cachebrowser's People

Contributors

ha-d avatar houmansadr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cachebrowser's Issues

Establishing secure connection Error

Hi

i just installing chrome extension and i can open youtube fine but after i close my chrome each time i try to open youtube. or fb.com or other site using https it gives me this error when it try to establishing connection :

"SSL connection error

ERR_SSL_PROTOCOL_ERROR
Hide details
Unable to make a secure connection to the server. This may be a problem with the server, or it may be requiring a client authentication certificate that you don't have."

i clear my cache and try again. but still the same.

i even install python version and run it through terminal in my mac and use 127.0.0.1:8080 as proxy but still the same

How to uninstall?

Thank you so much for coding this program。
How to uninstall it if i have installed?

python setup.py install doesn't work

C:\Users\xxx\Desktop\cachebrowser-master>python setup.py install
File "setup.py", line 72
os.chmod(chromeexec, 0755)
^
SyntaxError: invalid token

ImportError: No module named 'main'

Traceback (most recent call last):
File "/usr/bin/cachebrowser", line 4, in
import('pkg_resources').run_script('cachebrowser==0.1.0', 'cachebrowser')
File "/usr/lib64/python3.4/site-packages/pkg_resources/init.py", line 735, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib64/python3.4/site-packages/pkg_resources/init.py", line 1659, in run_script
exec(script_code, namespace, namespace)
File "/usr/lib64/python3.4/site-packages/cachebrowser-0.1.0-py3.4.egg/EGG-INFO/scripts/cachebrowser", line 2, in
# EASY-INSTALL-SCRIPT: 'cachebrowser==0.1.0','cachebrowser'
File "/usr/lib64/python3.4/site-packages/cachebrowser-0.1.0-py3.4.egg/cachebrowser/init.py", line 1, in
ImportError: No module named 'main'

Traceback when running from command line on linux

I cloned the code from GitHub, and used sudo python setup.py install to install. But when I set browser's proxy to localhost:8080 and ran cachebrowser it didn't seem to work properly. A traceback was given everytime I wanted to open a new web page.

I am on Fedora 23 x86_64, Firefox 42.0, Python 2.7.10.
uname -a gives Linux localhost 4.2.6-301.fc23.x86_64 #1 SMP Fri Nov 20 22:22:41 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

$ cachebrowser
('SELECT name FROM sqlite_master WHERE type = ? ORDER BY name;', ('table',))
('SELECT name FROM sqlite_master WHERE type = ? ORDER BY name;', ('table',))
Cachebrowser running...
Waiting for connections...
Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/cachebrowser/common.py", line 19, in inner
    return func(*args, **kwargs)
  File "build/bdist.linux-x86_64/egg/cachebrowser/proxy.py", line 45, in on_local_data
    self._schema = schema(self, self._buffer)
  File "build/bdist.linux-x86_64/egg/cachebrowser/proxy.py", line 173, in __init__
    self._start_upstream()
  File "build/bdist.linux-x86_64/egg/cachebrowser/proxy.py", line 201, in _start_upstream
    Host.get(url=host)
  File "/usr/lib/python2.7/site-packages/peewee-2.7.3-py2.7.egg/peewee.py", line 4392, in get
    sq = sq.filter(**kwargs)
  File "/usr/lib/python2.7/site-packages/peewee-2.7.3-py2.7.egg/peewee.py", line 2524, in filter
    query, joins = self.convert_dict_to_node(piece.query)
  File "/usr/lib/python2.7/site-packages/peewee-2.7.3-py2.7.egg/peewee.py", line 2500, in convert_dict_to_node
    model_attr = getattr(curr, piece)
AttributeError: type object 'Host' has no attribute 'url'

(The same trackback appeared again and again so I just skipped them here)

Document the remote bootstrapper protocol

For those looking to experiment with implementing their own remote bootstrappers some documentation would be great to have. I've noticed the RemoteBootstrapSource has these two methods lookup_host and lookup_cdn - I could throw something in the README about making these work with a custom host.

cachebrowser is not recognized as an internal or external command

I've successfully installed cachebrowser on WIN7 (64 bit) after installed Visual C++ for Python 2.7
but cachebrowser cannot be used as a command.
After installation, I can find a cachebrowser file (without extention) under the path D:\stat-tools\Python27\Scripts, I tried run the command under this folder, it doesn't work.

I also installed the plug-in for chrome, it doesn't work as well.

Implement remote bootstrapper

To be able to browse a webpage using CacheBrowser, it needs to bootstrap for that site to know what CDN the page is hosted on.

For testing and development purposes, the bootstrapping is currently done using a hard-coded dictionary in the code. To be able actively extend and update the database, a remote bootstrapper needs to be implemented, though in a way to avoid being blocked by censors.

The method we have in mind is to host the bootstrap server on a CDN and contact the the server the same way CacheBrowser uses to retrieve webpages.

get command not found

We have installed cachebrowser from the Master branch. The example in the README uses cachebrowser get command to request pages from the cli. However, the cachebrowser -help command indicates that there is no get command available.

Could you please let us know if we are missing anything or possible solutions to fix this issue. It will be really helpful to move further with our research if you can provide some insights on this one ASAP.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.