Git Product home page Git Product logo

zester's Introduction

Zester

Zester is a library that makes it easier to develop Python clients for websites without APIs.

No lxml, no XPath, just javascript.

Let's make a client library for Hacker News by saving the following code in a file named hnclient.py:

from zester import MultipleClient, Attribute

class HNClient(MultipleClient):
    url = "http://news.ycombinator.com/"
    title = Attribute(selector="$('.title a')", modifier="$(el).html()")
    link = Attribute(selector="$('.title a')"), modifier="$(el).attr('href')")
    points = Attribute(selector="$('.subtext span')", modifier="$(el).html().replace(' points', '')")

Now, let's use the client we just made. Open a python shell:

>>> from hnclient import HNClient
>>> client = HNClient()
>>> stories = client.process()
>>> stories[0]
HNClientResponse(points=u'200', link=u'http://daltoncaldwell.com/what-twitter-could-have-been', title=u'What Twitter could have been')
>>> print stories[0].title
What Twitter could have been
>>> print stories[0].link
http://daltoncaldwell.com/what-twitter-could-have-been
>>> print stories[0].points
56

We subclassed MultipleClient there because we were planning on returning multiple results. If we wanted to make a client for something like Weather.gov that returned a single result, we could do something like this:

from zester import SingleClient, Attribute

class WeatherClient(SingleClient):
    url = "http://forecast.weather.gov/MapClick.php?lat={lat}&lon={lng}"
    temperature = Attribute(selector="$('.myforecast-current-lrg').html()")
    humidity = Attribute(selector="$('.current-conditions-detail li').contents()[1]")
    heat_index = Attribute(selector="$('.current-conditions-detail li').contents()[11]")

    def __init__(self, lat, lng, *args, **kwargs):
        super(WeatherClient, self).__init__(*args, **kwargs)
        self.url = self.url.format(lat=lat, lng=lng)

This also demonstrates how you can allow arguments to be taken:

>>> from weather_client import WeatherClient
>>> client = WeatherClient(lat=40.7143528, lng=-74.0059731)
>>> curr_weather = client.process()
>>> curr_weather
WeatherClientResponse(heat_index=u'82\xb0F (28\xb0C)', temperature=u'80\xb0F', humidity=u'58%')
>>> print curr_weather.temperature
80°F
>>> print curr_weather.humidity
58%
>>> print curr_weather.heat_index
82°F (28°C)

Installation

Zester is dependant upon Ghost.py. You must install it before installing Zester. Ghost.py will also require the installation of either PyQt or PySide.

After Ghost.py is installed, to install zester: :

$ pip install zester

zester's People

Contributors

spulec avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

zester's Issues

RuntimeError: A QApplication instance already exists.

I was trying to create a tox.ini for tox for this project (I already created a working tox.ini for Ghost.py) but I'm getting this weird error. Any ideas? (I'm not too familiar with PyQt, PySide, etc.)

marca@ubuntu:~/dev/git-repos/zester$ tox
GLOB sdist-make: /home/marca/dev/git-repos/zester/setup.py
py27 sdist-reinst: /home/marca/dev/git-repos/zester/.tox/dist/zester-0.0.3.zip
py27 runtests: commands[0]

(python2.7:10617): Gtk-WARNING **: Unable to locate theme engine in module_path: "pixmap",

(python2.7:10617): Gtk-WARNING **: Unable to locate theme engine in module_path: "pixmap",

(python2.7:10617): Gtk-WARNING **: Unable to locate theme engine in module_path: "pixmap",

(python2.7:10617): Gtk-WARNING **: Unable to locate theme engine in module_path: "pixmap",
.EException AttributeError: "'Ghost' object has no attribute 'app'" in <bound method Ghost.__del__ of <ghost.ghost.Ghost object at 0xa7ebd2c>> ignored

======================================================================
ERROR: test_single_client (test_zester.ZesterTestSuite)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/marca/dev/git-repos/zester/tests/test_zester.py", line 34, in test_single_client
    url=weather_snapshot)
  File "/home/marca/dev/git-repos/zester/tests/fixtures/weather_client.py", line 13, in __init__
    super(WeatherClient, self).__init__(*args, **kwargs)
  File "/home/marca/dev/git-repos/zester/zester/client.py", line 15, in __init__
    self._ghost = Ghost()
  File "/home/marca/dev/git-repos/zester/.tox/py27/local/lib/python2.7/site-packages/ghost/ghost.py", line 184, in __init__
    self.app = QApplication(['ghost'])
RuntimeError: A QApplication instance already exists.

----------------------------------------------------------------------
Ran 2 tests in 7.581s

FAILED (errors=1)
ERROR: InvocationError: '/home/marca/dev/git-repos/zester/.tox/py27/bin/nosetests'
__________________________________________________________________ summary __________________________________________________________________
ERROR:   py27: commands failed

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.