looker / looker-sdk-ruby Goto Github PK

The following Swagger resource (looker-sdk-ruby/test/looker/swagger.json) seems to capture most of the user-related methods. Is there something similar for all of the query-related methods?

[Archived]

For docs: to hit an endpoint like /users/<id>/user_roles you need to be in the right role.

https://rubygems.org/gems/looker-sdk/versions/0.0.5 but the latest tag available in git is 0.0.3

[Archived]

I was taking a look at this .md file and it looks like it suggests hitting

https://mycompany.looker.com/login

to login but the Looker API docs have this endpoint listed as

https://mycompany.looker.com/api/<version here>/login

Can we update the instructions since hitting /login without the /api/3.1/ will yield a 403?

I've never been a fan of much of the work the Sawyer layer does in this SDK. I'm considering an option to completely bypass it - and perhaps replace some of the benefit we do get from it with simpler alternatives.

The main annoyances for me are:

1. The date/time deserialization based on field names https://github.com/lostisland/sawyer/blob/master/lib/sawyer/serializer.rb#L123-L125
   This has been a terrible fit for Looker's API and has gotten in the way a few times.

2. The funky Sawyer:: Resource results that often get in the way - and require one to call to_attrs (or map(&:to_attrs) if the result is an array!) in order to do any non-trivial work with a request result.

The thing is, while some of the deserialization stuff can really get in the way, some the other serialization stuff is important; e.g. converting times using Time.iso8601.

An alternative might be options that leave Sawyer in place, but give some specific control over its behavior; e.g. an option to not deserialize dates/times and a separate option to return results as hashes rather than as Sawyer:: Resource. Note that we previously added the 'raw_responses' option that skips all deserialization and returns raw json instead.

@dthorpe @conrizzle

Provide some hints in the docs about how to hit end points like /users//user_roles etc.

[Archived]

[Archived]

[Archived]

LookerSDK.client. merge_content_type_if_body fails to delete options[:headers] (as for instance LookerSDK.client.parse_query_and_convenience_headers correctly does) so the caller must supply content_type in order for the call to work.

E.g I'd expect the following to pass both "my-header" and "content_type" headers to the server:

sdk.create_some_resource(
  {resource_name: "foo", an_attr: "bar"},
  {headers: {"my-header" => "something"}},
)

instead it only passes "my-header" and so the caller must do the following to get it to work:

sdk.create_some_resource(
  {resource_name: "foo", an_attr: "bar"},
  {headers: {content_type: "application/json", "my-header" => "something"}},
)

Looks like Ruby SDK is sending api key and secret via querystring, it should be sent via body instead.
We used gzr(https://github.com/looker-open-source/gzr) to import dashboard which is using this sdk underlying, and we are seeing the secret being logged in our firewall logs, and server logs. Basically, anyone who looks at this request (MITM style) can see our auth credentials without having to ssl decrypt the request.

We are not able to use this SDK with our applications because our gems require higher versions of Faraday.

Hi there,

Sawyer v0.6 is being enforced in the gemspec which itself is very strict on the addressable GEM version it uses. The next version 0.7 relaxes that constraint, so I could fork the repo and test with that version and authentication + a run_look were working fine.

Going for the last version would be even better for everybody, since in between versions were only relaxing the dependencies: https://github.com/lostisland/sawyer/commits/master

I can write a PR for either:

• s.add_dependency 'sawyer', '<= 0.8.1'
• s.add_dependency 'sawyer', '~> 0.8'

Let me know what you think and thanks for the 💎
Thanks

Perhaps the Ruby version could be given in the spec; and be looser.

@jbandhauer
Not sure that this is an SDK issue as much as it is with the Looker API but I'll start here. I get a JSONParse error when there is a DB (Redshift/Postgres in this case) error thrown, when running a look. Judging by the error message the SDK throws the JSONParse error when it gets an unexpected response format from the looker instance. My thought is that an UnexpectedResponse exception of sorts should get thrown and that the API shouldn't be returning something that isn't in JSON format. This is how the API reports other errors it knows how to handle.

{
  "error": "Could not load model \"thelook\".",
  "error_detail": [
    "Model is not configured"
  ]
}

#<JSON::ParserError: unexpected token at 'SQL Error: ERROR: table 806669 dropped by concurrent transaction]'>

json/ext/Parser.java:251:in `parse'
/usr/local/lib/jruby-1.7.12/lib/ruby/shared/json/common.rb:155:in `parse'
/usr/local/lib/jruby-1.7.12/lib/ruby/shared/json/common.rb:334:in `load'
org/jruby/RubyMethod.java:128:in `call'
/var/www/looker/panopticon/releases/20150919190326/bundle/vendor/jruby/1.9/gems/sawyer-0.6.0/lib/sawyer/serializer.rb:62:in `decode'
/var/www/looker/panopticon/releases/20150919190326/bundle/vendor/jruby/1.9/gems/sawyer-0.6.0/lib/sawyer/agent.rb:125:in `decode_body'
/var/www/looker/panopticon/releases/20150919190326/bundle/vendor/jruby/1.9/bundler/gems/looker-sdk-ruby-a72323386700/lib/looker-sdk/sawyer_patch.rb:24:in `data'
/var/www/looker/panopticon/releases/20150919190326/bundle/vendor/jruby/1.9/bundler/gems/looker-sdk-ruby-a72323386700/lib/looker-sdk/client.rb:230:in `request'
/var/www/looker/panopticon/releases/20150919190326/bundle/vendor/jruby/1.9/bundler/gems/looker-sdk-ruby-a72323386700/lib/looker-sdk/client.rb:137:in `paginate'
/var/www/looker/panopticon/releases/20150919190326/bundle/vendor/jruby/1.9/bundler/gems/looker-sdk-ruby-a72323386700/lib/looker-sdk/client/dynamic.rb:96:in `invoke_remote'
/var/www/looker/panopticon/releases/20150919190326/bundle/vendor/jruby/1.9/bundler/gems/looker-sdk-ruby-a72323386700/lib/looker-sdk/client/dynamic.rb:63:in `method_missing'
/var/www/looker/panopticon/releases/20150919190326/lib/looker/permissions_api.rb:114:in `set_accounts_lists'
/var/www/looker/panopticon/releases/20150919190326/lib/looker/permissions_api.rb:80:in `update'
/var/www/looker/panopticon/releases/20150919190326/lib/looker/permissions_api.rb:32:in `initialize'
org/jruby/RubyKernel.java:1501:in `loop'
/var/www/looker/panopticon/releases/20150919190326/lib/looker/permissions_api.rb:31:in `initialize'

Since the update of Sawyer to version 0.8, the Looker Ruby SDK has failed to run on Ruby 1.9.3 due to Sawyer v0.8 dependencies that require Ruby >=2.1. Specifically, the Addressable 2.5.2 gem which requires public_suffix 2.0.2, which requires Ruby >=2.1.

This is now causing the rake test:all task and the Travis CI PR tests to fail for the Ruby 1.9.3 and JRuby 1.7 platforms.

Since a gem can't refer to different versions of dependencies for different Ruby platforms, it appears the only way to fix this is to drop support for Ruby 1.9.3. Anyone who needs the Looker Ruby SDK to run in the Ruby 1.9.3 environment should use Looker Ruby SDK version 0.0.5.

[Archived]

1. users = sdk.all_user() - you mean sdk.all_users()
2. get_credentials_email doesn't work - /opt/boxen/rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/looker-sdk-ruby-10ba1ca56f57/lib/looker-sdk/client/dynamic.rb:62:inmethod_missing': undefined method get_credentials_email' for #<LookerSDK::Client:0x007f8212854210> (NoMethodError) from basic.rb:38:in'`

A weak readme reduces confidence in the code/api.

Hello,

I noticed some strange behavior with the client sdk.

I am configuring the client as:

  def client
    @_client ||= LookerSDK::Client.new(
      client_id: Settings.looker.client_id,
      client_secret: Settings.looker.client_secret,
      api_endpoint: Settings.looker.api_endpoint,
      connection_options: {
        request: {
          timeout: 360,
          open_timeout: 30,
        },
      },
    )
  end

when using the client, I sometimes get

NoMethodError: undefined method run_inline_query' for #<LookerSDK::Client:0x0000000130cbd0> from looker-sdk/client/dynamic.rb:62:in method_missing'

I have not been able to find a better stack trace for this (even adding breakpoints in your client was not enough to figure this out). I was able to narrow down the failure to this test case:

    it "throws an error" do
      stub_request(:post, /caring.looker.com:19999\/login/).to_return(status: [500, "Internal Server Error"])
      stub_request(:get, /caring.looker.com:19999\/api\/3.0\/swagger.json/).to_return(status: [500, "Internal Server Error"])

      expect {
        client.run_inline_query('json', {})
      }.to raise_exception(NoMethodError)
    end

How should I handle this? My understanding is that a 500 during the login or when fetching swagger.json would make @swagger be nil, propping find_entry(method_name) in dynamic.rb return false. The 500 during either the login or fetching the config json is something to be expected, but I would expect a LookerSDK::ServerError instead of a NoMethodError

rake test works, but with rake test:all I run into Could not find rake-10.4.2 in any of the sources issues. Fiddling with which rake version to specify did not fix the problem. There are lots of stackoverflow questions about similar problems, but no answered resolved this for me. This used to work.

If one wants to leverage this in one's own gem: At present you can't specify it in a .gemspec 'cos it isn't published.

Using the example in the README, with netrc configuration:

/Users/xavier/.gem/ruby/2.3.1/gems/looker-sdk-0.0.5/lib/looker-sdk/authentication.rb:39:in `[]': no implicit conversion of Symbol into Integer (TypeError)
	from /Users/xavier/.gem/ruby/2.3.1/gems/looker-sdk-0.0.5/lib/looker-sdk/authentication.rb:39:in `set_access_token_from_params'
	from /Users/xavier/.gem/ruby/2.3.1/gems/looker-sdk-0.0.5/lib/looker-sdk/authentication.rb:32:in `block in authenticate'
	from /Users/xavier/.gem/ruby/2.3.1/gems/looker-sdk-0.0.5/lib/looker-sdk/authentication.rb:17:in `without_authentication'
	from /Users/xavier/.gem/ruby/2.3.1/gems/looker-sdk-0.0.5/lib/looker-sdk/authentication.rb:29:in `authenticate'
	from /Users/xavier/.gem/ruby/2.3.1/gems/looker-sdk-0.0.5/lib/looker-sdk/authentication.rb:10:in `ensure_logged_in'
	from /Users/xavier/.gem/ruby/2.3.1/gems/looker-sdk-0.0.5/lib/looker-sdk/client.rb:252:in `request'
	from /Users/xavier/.gem/ruby/2.3.1/gems/looker-sdk-0.0.5/lib/looker-sdk/client.rb:72:in `get'
	from /Users/xavier/.gem/ruby/2.3.1/gems/looker-sdk-0.0.5/lib/looker-sdk/client.rb:193:in `alive'
	from test.rb:9:in `<main>'

This seems weird because:

• Documentation claims it's not even trying to authenticate, but stack trace looks like it is.
• I know my supplied credentials are invalid, I'd expect an error message that suggests as much, rather than the inscrutable no implicit conversion error.

We noticed that when Sawyer::Resource attempts to parse a resource (.data / .to_attrs) it can mis-parse certain fields.

One of the SerDes in Sawyer attempts to auto convert json dates/timestamps to ruby dates/timestamps. It does this by checking to see if the key matches _at, _on, _date, or date and if so, tries to parse the value as a date. If the value is a string, it then tries to do a Time.parse of the value.

Queries (for example) have a filters parameter which is a json hash of the filter key (lookml_field) /value (query string) pairs. If you have a field that ends in _date with a value that resembles a date in some way (or parses to a date/timestamp) then sequel will automatically convert it to a timestamp. This can lead to some really crazy behavior if you aren't careful.

what we saw happening:
{"users.created_date" => "30 days"} would get parsed as {"users.created_date" => 2016-08-30 00:00:00 -700"}

If you check out https://github.com/lostisland/sawyer/blob/master/lib/sawyer/serializer.rb#L101-L113 you can see that Time.parse("30 days") gets parsed as the current timestamp at midnight (or perhaps the 30th of the current month at midnight).

Suggestion: be less permissive of date values and only automagically parse dates that are in the correct iso8601 format because that is the only format that the Looker API will return for timestamp values.

Any JSON response member with a time-like key name whose value is NOT in iso8601 format will be left alone.

Hey,

I must be missing something so would like to clarify. I'm following this quick start guide. It says once I create a looker client I can simply call looker.look(<look_id>) to get data back however I am not seeing this.

The method look doesn't exist. Neither does run_look. Is this guide out of date? I ran looker.methods in rails console and can't see them. I only see the usual get post put http methods.

looker.alive returns 200, so that part worked and the path I'm calling is this:

https://<company_name>.eu.looker.com/:19999/api/3.0

Would appreciate being pushed in the right direction. Thanks.

clientSDK.inspect includes the entire swagger payload in its output - something like 79k of info. We can live without that.

Per customer request:

There is a conflict in the current Looker Gem dependencies with the Google Ruby Gem. The current master Looker Gem has that dependency updated but it has not been released. We've been asked to directly link to the master branch on Github but that is not good form. We'd rather use the released Gem.

Per our internal discussion, this change might require a major version.
cc: @dthorpe

Hello,

We are trying to upgrade the elasticsearch gem to v7.6.0 which depends on Faraday ~> 1.0.0

The gem looker-sdk has a dependency on Faraday ['>= 0.9.0', '< 1.0']. We aren't able to upgrade elasticsearch because of this dependency.

Could you please review and try to fix this issue?

Thanks in advance!

Just an idea . . .

I have recently noticed that in my code I want to fetch all users (so they are cached - rather than calling for each user). But then I want roles for each user.

It would be quite convenient to be able to include roles during these calls.

(Ideally this would be done in the server-side API rather than in the client.)

The use case is for analyzing user lists for compliance.

Hi, is there a way to set query string to create endpoint like http://LOOKER:19999/api/3.0/queries/run_async?format=csv? thanks!

So I was working with a customer using the API who was struggling because the errors were not explanatory. I tried their call in the API web page and got back a detailed useful error message. I recommended that the user change their code to catch the error and display a useful error message.

This kind of pattern should be modeled in the examples.

begin
  req = {
    :name => group_name,
    :label => group_name,
    :type => 'string',
    :default_value => group_name,
    :user_can_view => false,
    :user_can_edit => false
  }
  attribute = sdk.create_user_attribute(req)
rescue LookerSDK::Error => e
  puts e.message
  puts e.errors if e.errors
  raise
end

[Archived]

Can your dynamic wrapper do the v3 /users/{user_id}/credentials_email ? (answer seems to be: no.)

The client code inherited a scheme for plucking out query headers that seem like they are intended to be in the headers.

That is done in parse_query_and_convenience_headers at
https://github.com/looker/looker-sdk-ruby/blob/master/lib/looker-sdk/client.rb#L458-L462

The specific headers it supports are:
CONVENIENCE_HEADERS = Set.new([:accept, :content_type])

It isn't clear that this is doing anyone any good. And, there have been a couple known instances of this being a pain for people.

Because Looker API designers were not precluded from using these strings as query param names, there are some endpoints in Looker that use content_type as a param. Those endpoints are thus confusingly difficult to use from the SDK.

The workaround is to explicitly declare the query parts like:

Instead of:

sdk.foo({content_type: 'bar', baz: 1})

Use:

sdk.foo({query: {content_type: 'bar', baz: 1}})

The SDK allows for explicitly saying which params are for the query part and which are intended to be headers. So, one can still do:

sdk.foo({query: {content_type: 'bar', baz: 1}, headers:{content_type: 'json'}})

Nevertheless, my proposal is to simply remove the CONVENIENCE_HEADERS stuff so that the only way to have hash members end up in the headers is to do it explicitly.

Yet another case of the 'helpful' name-specific handling getting in our way more than helping us.