Git Product home page Git Product logo

tentacles's Introduction

Dependencies Status

An octocat is nothing without her tentacles

Tentacles is a Clojure library for working with the Github v3 API. It supports the entire Github API.

This library is the successor to my old clj-github library. clj-github will no longer be maintained.

Usage

This is on clojars, of course. Just add [tentacles "0.5.1"] to your :dependencies in your project.clj file.

CODE!

The library is very simple. It is a very light wrapper around the Github API. For the most part, it replaces keywords with properly formatted keywords, generates JSON for you, etc. Let's try out a few things.

user> (user-repos "amalloy")
; Evaluation aborted.
user> (repos/user-repos "amalloy")
[{:fork false, :pushed_at "2010-12-10T07:37:44Z", :name "ddsolve", :clone_url "https://github.com/amalloy/ddsolve.git", :watchers 1, :updated_at "2011-10-04T02:51:53Z", :html_url "https://github.com/amalloy/ddsolve", :owner {:avatar_url "https://secure.gravatar.com/avatar/1c6d7ce3810fd23f0823bf1df5103cd3?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-140.png", :url "https://api.github.com/users/amalloy", :gravatar_id "1c6d7ce3810fd23f0823bf1df5103cd3", :login "amalloy", :id 368685}, :language "Clojure", :size 1704, :created_at "2010-08-18T16:37:47Z", :private false, :homepage "", :git_url "git://github.com/amalloy/ddsolve.git", :url "https://api.github.com/repos/amalloy/ddsolve", :master_branch nil, :ssh_url "[email protected]:amalloy/ddsolve.git", :open_issues 0, :id 846605, :forks 1, :svn_url "https://svn.github.com/amalloy/ddsolve", :description "Double-dummy solver for contract bridge"} ...]

I cut out most of the output there. If you try it yourself, you'll notice that it produces a ton of output. How can we limit the output? Easily!

user> (repos/user-repos "amalloy" {:per-page 1})
[{:fork false, :pushed_at "2010-12-10T07:37:44Z", :name "ddsolve", :clone_url "https://github.com/amalloy/ddsolve.git", :watchers 1, :updated_at "2011-10-04T02:51:53Z", :html_url "https://github.com/amalloy/ddsolve", :owner {:avatar_url "https://secure.gravatar.com/avatar/1c6d7ce3810fd23f0823bf1df5103cd3?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-140.png", :url "https://api.github.com/users/amalloy", :login "amalloy", :gravatar_id "1c6d7ce3810fd23f0823bf1df5103cd3", :id 368685}, :language "Clojure", :size 1704, :created_at "2010-08-18T16:37:47Z", :private false, :homepage "", :git_url "git://github.com/amalloy/ddsolve.git", :url "https://api.github.com/repos/amalloy/ddsolve", :master_branch nil, :ssh_url "[email protected]:amalloy/ddsolve.git", :open_issues 0, :id 846605, :forks 1, :svn_url "https://svn.github.com/amalloy/ddsolve", :description "Double-dummy solver for contract bridge"}]

This time we actually did get just one item. We explicitly set the number of items allowed per page to 1. The maximum we can set that to is 100 and the default is 30. We can get specific pages of output the same way by using the :page option. Additionally, :all-pages true can be passed, which will return a lazy seq of all items on all pages.

This also introduces an idiom in tentacles: options are a map passed to the last parameter of an API function. The options map also contains authentication data when we need it to:

user> (repos/repos {:auth "Raynes:REDACTED" :per-page 1})
[{:fork true, :pushed_at "2011-09-21T05:37:17Z", :name "lein-marginalia", :clone_url "https://github.com/Raynes/lein-marginalia.git", :watchers 1, :updated_at "2011-11-23T03:27:47Z", :html_url "https://github.com/Raynes/lein-marginalia", :owner {:login "Raynes", :avatar_url "https://secure.gravatar.com/avatar/54222b6321f0504e0a312c24e97adfc1?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-140.png", :url "https://api.github.com/users/Raynes", :gravatar_id "54222b6321f0504e0a312c24e97adfc1", :id 54435}, :language "Clojure", :size 180, :created_at "2011-11-23T03:27:47Z", :private false, :homepage "", :git_url "git://github.com/Raynes/lein-marginalia.git", :url "https://api.github.com/repos/Raynes/lein-marginalia", :master_branch nil, :ssh_url "[email protected]:Raynes/lein-marginalia.git", :open_issues 0, :id 2832999, :forks 0, :svn_url "https://svn.github.com/Raynes/lein-marginalia", :description "A Marginalia plugin to Leiningen "}]

Default options can be specified via with-defaults.

If an API function has no options and authentication would have no uses for that particular call, the options map is not a parameter at all. For API calls that can do different things based on whether or not you are authenticated but authentication is not required, then the options map will be an optional argument. For API calls that require authentication to function at all, the options map is a required argument. Any data that is required by an API call is a positional argument to the API functions. The options map only ever contains authentication info and/or optional input.

Authentication is supported by Github user authentication :auth <username:password> as demonstrated above, or by oauth or oauth2. For oauth use :oauth-token <token> instead of :auth in the options map. Likewise, for oauth2, include :client-id <client_id> :client-token <client_token> in the options map.

You can access useful information returned by the API such as current rate limits, etags, etc. by checking the response with core/api-meta. You can then use this to perform conditional requests against the API. If the data has not changed, the keyword :tentacles.core/not-modified will be returned. This does not consume any API call quota.

user> (core/api-meta (repos/readme "Raynes" "tentacles" {}))
{:links {nil nil}, :etag "\"f1f3cfabbf0f98e0bbaa7aa424f92e75\"", :last-modified "Mon, 28 Jan 2013 21:13:48 GMT", :call-limit 60, :call-remaining 59}

user> (repos/readme "Raynes" "tentacles" {:etag "\"f1f3cfabbf0f98e0bbaa7aa424f92e75\""})
:tentacles.core/not-modified

user> (repos/readme "Raynes" "tentacles" {:if-modified-since "Mon, 28 Jan 2013 21:13:48 GMT"})
:tentacles.core/not-modified

Similarly, you can set an User-Agent to make your requests more friendly and identifiable.

user> (repos/readme "Raynes" "tentacles" {:user-agent "MyPhoneApp"})

The Github API is massive and great. I can't demonstrate every API call. Everything is generally just as easy as the above examples, and I'm working hard to document things as well as possible, so go explore!

Here are some lovely Marginalia docs. I also wrote a demonstrational blog post about Tentacles that I intend to keep updated with future releases.

If you run into something that isn't documented well or you don't understand, look for the API call on the Github API docs. If you feel like it, please submit a pull request with improved documentation. Let's make this the most impressive Github API library around!

Hacking

Running the tests

In order to run the tests, you need to create a testinfo.clj in the root of the checkout with some info required for the tests to run properly. This file is ignored by git, so don't worry about committing auth info. This file should contain a Clojure map like the following:

{:user "" ;; Github username
 :pass "" ;; Github password
 :follows ""} ;; Username of a person that this user follows.

As more tests are written this information may grow.

License

Copyright (C) 2011 Anthony Grimes

Distributed under the Eclipse Public License, the same as Clojure.

tentacles's People

Contributors

arohner avatar atroche avatar bcambel avatar benashford avatar brandonbloom avatar cbilson avatar d2fn avatar danburkert avatar danfertev avatar dlobatog avatar etoews avatar fcuny avatar gilbertw1 avatar gpg0 avatar grepory avatar hugoduncan avatar jakemcc avatar jeffreypalmer avatar jeluard avatar joelittlejohn avatar jszakmeister avatar kipz avatar mcfunley avatar milt avatar ngrunwald avatar raynes avatar thearthur avatar vjc avatar whostolebenfrog avatar yosiat avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tentacles's Issues

Custom User-Agent?

Is there a simple way to add a custom User-Agent header so that my application can be a good GitHub citizen? If not, I'm happy to add it. Any pointers on where it would most easily fit?

Thanks.

Make `core` more generic so it can be used for other APIs

I'm implementing an app that needs to use REST APIs for a number of services. I'm implementing a connector for Pivotal Tracker right now, and it looks to me like tentacles.core could be a good basis for this. I'd have to make it a bit more generic though (e.g. the API base URL has to be a parameter and not hardcoded, and there needs to be some way to add arbitrary headers).

Would you consider a pull request along these lines?

Add support for rate-limiting

Once current user rate limit threshold is crossed all calls will return a 403 status code.
Currently this code is ignored by safe-parse thus cannot be handled by caller.

Is there any other way to catch those errors?

defaults are not added to request headers

Hi @Raynes, loving your work 😃 I've been having a problem writing to the GitHub API and I think it's caused by my defaults not being added to the request correctly.

In tentacles.core/make-request we build up a request, adding headers if necessary. After adding headers, the defaults are merged into the query map. This process means that defaults can appear in a request body (for put/post/delete) or in the query string (for get etc), but they cannot go into headers.

I'm trying to use GitHub Trees API and I'm finding that the :oauth-token value I have in my tentacles defaults is ignored. It appears to be because this API does not allow the token to be supplied in the body, it has to be in the Authorization header.

I think we should merge defaults in before deciding which headers to add. Does this sound sensible? If so I'm happy to submit a PR for this.

Publish the new version to Clojars

Hi @Raynes,

Would you be kind enough to publish the new version to Clojars so we can use the new update in the project.clj.

Currently the last publish version was at 0.5.1.
Thanks again for your great contribution.

edit-issues 404?

I am a Clojure noob, so there is a higher than normal probability of this being user error.

However, while I can successfully use functions like specific-issue, I find that with edit-issue, I get a response from Github that indicates a 404 error.

I notice that the Github issues API docs indicate that the edit endpoint should use PATCH, however edit-issues appears to be using POST.

Output with redacted user/repo names

user=> (issues/edit-issue "my-user" "my-repo" 1 {:body "new issue body"})
{:orig-content-encoding "gzip", :trace-redirects ["https://api.github.com/repos/my-user/my-repo/issues/1"], :request-time 638, :status 404, :headers {"Access-Control-Expose-Headers" "ETag, Link, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval", "Server" "GitHub.com", "Content-Type" "application/json; charset=utf-8", "Access-Control-Allow-Origin" "*", "X-Content-Type-Options" "nosniff", "X-Frame-Options" "deny", "Strict-Transport-Security" "max-age=31536000; includeSubdomains; preload", "X-RateLimit-Limit" "60", "X-RateLimit-Remaining" "46", "X-RateLimit-Reset" "1416809234", "Connection" "close", "Transfer-Encoding" "chunked", "Status" "404 Not Found", "X-GitHub-Request-Id" "redacted-request-id", "X-GitHub-Media-Type" "github.v3; format=json", "Date" "Mon, 24 Nov 2014 05:46:02 GMT", "Access-Control-Allow-Credentials" "true", "X-XSS-Protection" "1; mode=block", "Content-Security-Policy" "default-src 'none'"}, :body {:message "Not Found", :documentation_url "https://developer.github.com/v3/issues/#edit-an-issue"}}

lazy-seq over pages

<TeXnomancy> Raynes: reading about tentacles, have you considered the idea of
             a generalized rest-pagination lazy-seq?                    [17:17]
<Raynes> TeXnomancy: No, but that's a fantastic idea.
<TeXnomancy> the only hitch is that you would need a lazy seq that could
             perform no-ops when it was being forced from a drop call
<Raynes> Bleh.                                                          [17:18]
<TeXnomancy> certainly not insurmountable, but worth pondering
*** jimmy1980 ([email protected]) has quit: Ping timeout: 240 seconds
                                                                        [17:19]
<Raynes> TeXnomancy: I'll make an issue for that. Great idea, so thanks.

This is a test issue.

I made this issue using the library, because I needed to test this particular API call. Ignore me.

ClojureScript and caching support

I'm currently finishing up a half-baked fork with the codebase ported to CLJX so as to support ClojureScript. The motivation was a) to have cljs support; and b) to allow pluggable HTTP clients, which in turn allows caching to be performed at the request level so users (i.e. me) don't have don't have to wrap the whole API.

I'm considering submitting a PR but don't want to spend time tidying this up if the idea is a non-starter for you.

Thoughts?

Tentacles broken with Clojure 1.8.0

Hey there, I'm running into an issue trying to use tentacles with clojure 1.8.0.

I get the following error when requiring the tentacles.pulls namespace:

CompilerException java.lang.NoClassDefFoundError: IllegalName: compile__stub.clj_http.headers.clj-http.headers/HeaderMap, compiling:(clj_http/headers.clj:105:1) 

Thanks for the sweet tool.

Collection responses have an extra empty map as the last element

Is there a reason for responses returning a collection of github objects to include an empty element at the end or is that a bug?

(pulls/pulls "Raynes" "tentacles" {:auth "cddr:M24thbdinF"
                                                     :per-page 1})
=> [{:created_at "", :assignee nil.....} {}]

Would you accept a PR fixing it or would that be considered a breaking change?

Head branches cannot be passed into reference(s)

Motivation

https://developer.github.com/v3/git/refs/

In the above link there's discussion about pulling a specific branch from GH requiring a heads prefix to be passed into the references api call. While Tentacles does have support for pulling all branches defined here, pulling a single branch requires more work via the references endpoint.

Unfortunately, if you try to use said endpoint herein, you get the following:

(tdata/reference username reponame
                     (str "head/" branch-name)
                     {:oauth-token github-access-token
                      :throw-exceptions true
                      :all_pages true})

=> CompilerException clojure.lang.ExceptionInfo: clj-http: status 404 {:status 404, :headers ..., :body "{\"message\":\"Not Found\",\"documentation_url\":\"https://developer.github.com/v3/git/refs/#get-a-reference\"}", :request-time 203, :trace-redirects ["https://api.github.com/repos/CircleCITestOrg/circle-dummy-branch-repo/git/refs/head%2Fabranchname], :orig-content-encoding "gzip"}, compiling: ...

Note the url encoded / which comes from the heads/abranchname.

I'm not really sure what to do to provide this functionality in this repo as forward slashes are legal ref names...

Suggested Musical Pairing

https://soundcloud.com/misterwives/coloring-outside-the-lines-rac

Requests error out when the body is not json

Evaluate the following:

(tentacles.pulls/specific-pull "Malabarba" "lazy-map-clojure" 1 {:accept "application/vnd.github.VERSION.patch"})

You'll immediately get an exception because the :body is a patch file, and parse-safe tries to parse it as json.

Thanks for this great package!

`with-defaults` isn’t working

Hey @Raynes,

First off: this project is great.

I’m still very new to clojure so I may be doing something wrong, but with-defaults doesn’t seem to be working for me. (Side note: the first entry in every API response is always an empty hash for me...)

Example:

(ns project.repl
  (:require [tentacles.core :as t]
            [tentacles.orgs :as orgs]))

(def defaults { :oauth-token "LOOK_ITS_MAH_TOKEN"
                  :per-page 100 })

;; I assume this should automatically merge these defaults with any provided options for every API request
(t/with-defaults defaults)

;; so this request should have my authorization token and return private teams
(defn teams [org]
  (orgs/teams org))

(teams "skookum") ;; -> only public teams

;; so I have to manually pass in the options object to get private teams
(defn teams-manual-defaults [org]
  (orgs/teams org defaults))

(teams-manual-defaults "skookum") ;; -> all teams

Empty contents throws a base64 exception

Hi,

I am calling to empty content file and I am getting an error.

Code:

(tentacles.repos/contents "USER" "REPO" "EMPTY_FILE" {:str? true})

0.2.7:

ArrayIndexOutOfBoundsException -1 clojure.data.codec.base64/pad-length (base64.clj:53)

master:
I get the next error, even If I specify {:str? false}
ArityException Wrong number of args (0) passed to: core$byte-array clojure.lang.AFn.throwArity (AFn.java:437)

Write tests

The whole thing is completely testless at the moment. Still figuring out what the best way to go about tests would be.

Cannot run tests

'lein test' or 'lein check' throw

Exception in thread "main" java.lang.IllegalArgumentException: Parameter declaration str should be a vector

I tried running the tests with java 1.7.0_35, and 1.6.0_27 in Ubuntu 13.04 and 13.10.
I created a file "testinfo.clj" at the root containing a map similar to the one described on the README with my personal data.

How to handle rate-limits?

When an error occurs like rate-limit exceeded, the response is unparsed (safe-parse not applied?), thus it's not clear that an error occur from the client side, because keys like :status or :trace-redirects or :body could be part of a valid response. Maybe the response should be a nil or an exception should be thrown (which I like much less). Then the client can check for nil and do the api-meta call to see what went wrong.

Btw, the api-meta seems not to work for me. I guess it's because the metadata gets lost after the call and assigning it to a variable. But is necessary for inspection and decision, e.g.

(defn succeeded? [meta]
  (and meta (not= (403 (:status meta)))))

(let [res (repos/languages "Raynes" "tentacles")]
  (when (succeeded? (core/api-meta res)))
    res)
;=> nil

Typo in the README

While I was reading your README file, I noticed a misspelling (highlighted in bold-italics below)

We can get specific pages of output the same way by using _hte_ :page option. Additionally, :all-pages true can be passed, which will return a lazy seq of all items on all pages.

Use (default) options for all calls

I try to implement a small server application where I would like to use the 5000 requests/h rate-limit when using basic authentication or oauth. When testing, I get:

(tentacles.core/with-defaults {:client-id "foobar" :client-token "foobar"}
  (tentacles.core/rate-limit))

Thus, I assume e.g. tentacles.core/rate-limit ignores the authentication credentials, as described in the documentation?

If an API function has no options and authentication would have no uses for that particular call, the options map is not a parameter at all.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.