algolia / algoliasearch-client-ruby Goto Github PK

In our application, we make requests to both the search API and the analytics API using this client. Unfortunately, there seems to be a pretty major multithreading issue where the wrong host gets picked for requests. For example, we've had indexing PUT requests go to the analytics API, and analytics GET requests attempt to use the indexing API.

From my light exploration, the problem seems to lie with the fact that hosts are stored in a global per-thread state, even if you manually initialize the client with Algolia::Client.new.

We use this library in conjunction with algoliasearch-rails, so we have models that are being indexed with that gem, as well as some controller code for fetching analytics. We initialize the analytics client with:

@algolia ||= Algolia::Client.new(
  application_id: ENV['ALGOLIA_APP_ID'],
  api_key: ENV['ALGOLIA_API_KEY'],
  hosts: ["analytics.algolia.com"]
)

Let me know if there's any more info you need, or if you would like me to take a crack at this issue.

LoadError: cannot load such file -- algolia/analytics

• Algolia Client Version: 1.23.0
• Language Version: ruby 2.5.1

I've tried running a bundle install and even a manual gem install algoliasearch-rails and bundler doesn't seem to want to touch this new gem. I have some funky stuff going on with Algolia and I want to see if this is the issue. If anyone can explain why bundler would not update this gem, let me know.

Here is the relevant part of my Gemfile:

# Search
gem 'algoliasearch', '~> 1.6', '>= 1.6.1'
gem "algoliasearch-rails"

I am trying to delete an object from my Algolia index in ruby.

Bot::Candidates.index.delete_object(candidate_id)

I get the following error:

Cannot DELETE to https://MYID.algolia.net/1/indexes/candidates/: {"message":"Method not allowed with this API key","status":403}

My API key has the following ACL:

search,addObject,deleteObject

When I try the same operation from the REST API, it works without a glitch:

curl -X DELETE -H "X-Algolia-API-Key: <sameapikeyasabove>" -H "X-Algolia-Application-Id: MYID" "https://MYID.algolia.net/1/indexes/candidates/<objectID>"

after checking the task status, I get:

{"status":"published","pendingTask":false}

What is it working with the REST api and not with the ruby wrapper ? Is the ruby wrapper performing another operation than simply delete ?

• Algolia Client Version: 1.25.0
• Language Version: 2.5

Description

Loading algoliasearch fails with the error cannot load such file -- algolia/insights (LoadError).

This also impacts users of jekyll-algolia (cf algolia/jekyll-algolia#108)

I have this:

irb(main):003:0> query
=> [{"query"=>"", "hitsPerPage"=>40, "facets"=>"*", "facetFilters"=>["centre_id:chatswood", "country:au"], "page"=>0, "maxValuesPerFacet"=>"300"}]

I query Algolia:

irb(main):004:0> Algolia.multiple_queries query

I get a bunch of results. Good.

Next, I repeat the same query and get an error:

irb(main):004:0> Algolia.multiple_queries query 
Algolia::AlgoliaProtocolError: 400: Cannot POST to https://XXXXXXXX-dsn.algolia.net/1/indexes/*/queries?strategy=none: {"message":"Null not allowed near line:1 column:30","status":400} 
(400) 
from /app/vendor/bundle/ruby/2.2.0/gems/algoliasearch-1.6.1/lib/algolia/client.rb:406:in perform_request' 
from /app/vendor/bundle/ruby/2.2.0/gems/algoliasearch-1.6.1/lib/algolia/client.rb:340:in block in request' 
from /app/vendor/bundle/ruby/2.2.0/gems/algoliasearch-1.6.1/lib/algolia/client.rb:333:in times' 
from /app/vendor/bundle/ruby/2.2.0/gems/algoliasearch-1.6.1/lib/algolia/client.rb:333:in request' 
from /app/vendor/bundle/ruby/2.2.0/gems/algoliasearch-1.6.1/lib/algolia/client.rb:356:in post' 
from /app/vendor/bundle/ruby/2.2.0/gems/algoliasearch-1.6.1/lib/algolia/client.rb:106:in multiple_queries' 
from /app/vendor/bundle/ruby/2.2.0/gems/algoliasearch-1.6.1/lib/algolia/client.rb:488:in multiple_queries' 
from (irb):4 

If I restart my irb session, I can get success again - the first time

List of things to change (if possible)

• index.search rename params to searchParameters
• search_for_facet_value rename parameters
• Fix browse (and doc snippets)
• delete_objects pass objectIds list
• wait_task timeBeforeRetry arg ?
• get_settings remove options?
• Algolia.init refactor ENV['xxx'] ?

Docs around

# Iterate with a filter over the index
index.browse({:query => "test", :numericFilters => 'i=42'}) do
    # Do something
end

need work - the block is passed something weird and is not explained

And pretty sure cover-rails is not up-to-date.

It's getting thrown around in my tests a lot and I can't figure out why. This is the line that fails:

@users = User.search(search_string)

It feels like I'm missing something obvious, but I don't know what

gem version 1.19.1

Algolia.delete_index and Algolia.delete_index! raise an ArgumentError
ArgumentError: wrong number of arguments (given 2, expected 1)

It seems that our ruby API client doesn't return the proper "type" in its response. It just defaults to type="all". Also we didn't update the onlyError parameter even though it's deprecated.

https://github.com/algolia/algoliasearch-client-ruby/blob/master/lib/algolia/client.rb#L189
https://github.com/algolia/algoliasearch-client-ruby/blob/master/lib/algolia/protocol.rb#L104

Sometimes our users search for unreasonably large strings and we get back an AlgoliaProtocolError object with a message attribute that looks like this:

"Cannot POST to https://<snip>.algolia.io/1/indexes/*/queries: {\"message\":\"query string too long\",\"status\":400}\n (400)"

There's also a status attribute that is just 400. I want to be able to handle this specific error since in our case it is a user error, but I don't want to catch all algolia errors or all algolia errors with a status code of 400. The best I can do right now is

e.message =~ /query string too long/

But I would rather have a more reliable way to access the API response, which I would expect to be more stable (and hopefully one day documented).

Reference: algolia/algoliasearch-client-python#379

Implement method on the client that lets the user retrieve multiple objects from multiple indices.

Hey folks,

I've mentioned this to @redox in a PR on one of our apps (or rather, a demo app) and have recreated the issue I've been experiencing in this PR

The PR shows 2 commits, one being the initial setup for webmock and webmock/algolia where tests will fail. The second copies all request stubs from the file we required and adds them in the test file itself, making the tests pass.

TL;DR: just using require 'webmock/algolia' doesn't actually appear to register any stubs, and I'm not entirely sure why.

index.browse({
  query: 'test'
})

A slightly modified version of the code from the README. Getting back a {"message":"Invalid value for page parameter","status":400}.

Here's the URL it's trying to hit:
https://FOO.algolia.net/1/indexes/BAR/browse?page=%7B%3Aquery%3D%3E%22test%22%2C+%3Apage%3D%3E1%7D&hitsPerPage=1000

Is it possible to disable Algolia completely when runnings tests? None of my tests depend on it and it only slows my test down.

Calling copy_index!('foo', 'bar') when foo does not exists will result in the method hanging forever.

What happened?

• The API will accept the job, even if foo does not exist (this is because the index might exist in the future, when the job will be processed).
• The API will return a taskID, bound to the foo index.
• When the job will be executed, the job will do nothing, because foo does not exist. The API will move to the next job.
  Note: It is not entirely true that the job will do nothing. It will still create bar actually

From a user standpoint, it means that we receive a taskID, bound to an index that does not exist. Listening for this taskID will result in an 400 error (index foo does not exist), and copy_index! will just wait forever for this to resolve.

How to solve it

Solution 1: Raising the error from the wait_task to the copy_index!

It seems that copy_index! is swallowing the error returned by the API. Maybe it should abort its processing if the check to the taskID returns an error. This might be tricky because as said above, it might be perfectly fine for this call to return an error if the index does not exist yet, but will exist when the job will be processed.

Solution 2: Forcing the API to reject the job if the source index does not exist

This is breaking the asynchronous paradigm of our API, but could be acceptable in that case maybe? What are the use-cases where you would try to copy a non-existent index, knowing that it will exists when the job will be executed?

Solution 3: Checking for the existence of the index before doing the copy

This is what I do in jekyll-algolia: I first do a get_settings to the index to check if it exists before sending a copy_index! job. If the index is not here, I just don't do it. This might be a check we could add to the Ruby API Client as well (but that would mean adding it to all API Clients as well).

cc @ercolanelli-leo @Nagriar @clement-leprovost @adrienjoly

My shopify app needs to connect to multiple algolia instances. Hence, I'm calling Algolia.init each time I'm dealing with something for a specific client.
When I'm looking at the Algolia.client, the info is OK. However, when trying to call Algolia, I got a "403 APP-ID or API key invalid".
I investigated a bit, and found out that the issue was due to the fact that we have a specific client per thread in Thread.current[:algolia_hosts], and that we always get back the same one, even when the client is changed. I belive it should be set during the initialize instead of in thread_local_hosts if not defined.
My solution to this is to do a Thread.current[:algolia_hosts] = nil, but I believe this should be handled differently.

What do you think?

• Algolia Client Version: 1.23.2
• Language Version: 2.4.1

Description

client.delete_index() does nothing

Steps To Reproduce

Hi folks, I found the method to generate secured API key via support but figured it might be worth creating a ticket incase others come looking for the same:

Found the method here:

It would be super helpful to see this in the README, with an example or two.

Thank you!

In the case the distinct is enabled on an index (with distinct > 1), a call to the method delete_by_query returns the following error:

{
  "message": "invalid distinct value, hitsPerPage * distinct should be <= 1000",
  "status": 400
}

We should probably override distinct=false at query time when the method performs the search call.

• Algolia Client Version: 1.20.0
• Language Version:

Description

The PR #285 introduce multi cluster management (MCM). All methods are correct but they should be added to the singleton Algolia.

Hey,

I think the recommended webmock setup in https://github.com/algolia/algoliasearch-client-ruby#mocking does not work in case of multiple test files because require will only load the code once. I think the stubs needs to be loaded in every test. Here is what I am doing instead:

RSpec.describe Event do

  specify do
    load $LOADED_FEATURES.find{|x| x.include?("algolia/version.rb") }.gsub("version.rb", "webmock.rb")

    # code which triggers algolia integration goes here....

    expect(WebMock).to have_requested(:put, "https://application_id.algolia.net/1/indexes/events/#{event.id}").
      with(body: hash_including({
        "objectID": event.id.to_s,
        "name": "dummy event",
        "category": "music",
      }))
  end

end

The key difference is that I do load instead of require. Unfortunately doing load 'algolia/webmock' did not just work so I had to use $LOADED_FEATURES.find{|x| x.include?("algolia/version.rb") }.gsub("version.rb", "webmock.rb") to find the file and load it.

I need to do it in every affected test.

We keep getting this error in our application:

ERROR -- : 0: Cannot reach any host: execution expired, execution expired, execution expired, execution expired
/usr/local/bundle/gems/algoliasearch-1.10.0/lib/algolia/client.rb:351:in `request'
/usr/local/bundle/gems/algoliasearch-1.10.0/lib/algolia/client.rb:359:in `post'
/usr/local/bundle/gems/algoliasearch-1.10.0/lib/algolia/index.rb:548:in `batch'
/usr/local/bundle/gems/algoliasearch-1.10.0/lib/algolia/index.rb:271:in `save_objects'

When this happens, the application will stay in this state forever (e.g. getting this exact error in every 80 seconds). Sometimes this error occurs just a few minutes after the application started, sometimes it takes weeks to occur. Couldn't find any pattern.
Calling Algolia.destroy then Algolia.init does not solve the problem. The only solution we have so far is to restart the application.
Why is this happening and what can we do to avoid this error?
Using Ruby 2.3.1, Ubuntu 14.04, kernel 4.4.0. Thanks.

When trying to save a synonym with a numeric objectID, such as save_synonym(12345, {...}), it throws the following error:

TypeError: no implicit conversion of Fixnum into String

It occurs in protocol.rb in this function:

def Protocol.synonym_uri(index, object_id)
  "#{synonyms_uri(index)}/#{CGI.escape(object_id)}"
end

Looks like it just needs to be updated to CGI.escape(object_id.to_s), like how it's done in the ::to_query method.

Not a huge deal since I can call to_s on the ID beforehand, just wanted to log this!

From hannesfostie/big_articles#1

My first try was to use index.browse['nbHits']. But it gets 1000 records.
Currently, I am using index.search('', { hitsPerPage: 0 })['nbHits'].

It would be nice to add a method index.number_of_records or index.records_count in this gem.

When requiring algolia/webmock, it will only stub algolia.(net|io) requests. (Source)

However, the client sometimes makes requests to algolianet.com. Are those intentionally not stubbed ? If it is a mistake, happy to do a PR.

cc @Nagriar

index.delete_by_query(nil)

End result is 0 records in the relevant index.

Algolia.init(application_id: 'YourApplicationID', api_key:        'YourAPIKey')

This implies that only one application can be managed at the same time.
What should i do if i need to manage multiple apps?

The Algolia::Client post, put, and delete requests all make a network call to Algolia. Recently, we have started noticing performance issues where put calls are taking ~300ms apiece. It seems like an easy solution (that we are currently monkey patching successfully) is to make post requests async (which HTTPClient supports) and optimistically assume the update will succeed. Is this a feature you could add so we can stop monkey patching our gem?

Hello. I am trying to record requests and responses using vcr for my testing purposes.
When there is no request/response for a given test it would pass. But on the second run test will fail.

I've noticed the difference in URI when I am trying to get an object from algolia index.
On the firs run it would look like:
https://atc2cvtc1r-dsn.algolia.net/1/indexes/Gift_test_avbrychak/gift_2626
On the second:
https://ATC2CVTC1R-1.algolianet.com/1/indexes/Gift_test_avbrychak/gift_2695

I think the problem is in the changing part -dsn. Is it possible to use the same URI?

• Algolia Client Version: 1.23.0
• Language Version: all

Description

Fix for issue on Dashboard where the wrong User agent was being displayed (Rails should be listed first instead of Ruby)

algolia_for_ruby_(1.23.2);_ruby_(2.3.1);_algolia_for_rails_(1.20.4);_algolia_for_ruby_(1.23.2)

Hey there, I'm trying to remove dynamically some replicas, but it seems I can't do that, without manually disassociate it from the main index. I couldn't find anything in here, rails client, and algolia api docs that tell me how to do it... is not supported?

Thanks

This is mostly an issue for batch vs send.
Both types of requests use a different connection pool, which means they can end up on different machines (no keep-alive guarantee here).

So if we send a batch immediately followed by a move_index or a delete_index, if they share the same timestamp, the consensus might reorder them.

I don't know how we should fix it. It might be by documenting the fact the move or delete should wait on the previous indexing task.

It might also simply be by adding a sleep 1 to ensure the timestamps are different.

The delete_object and delete_objects methods both have a ! version that will actually wait until the delete has taken place before returning. It appears that the delete_by_query method goes against this and waits by default. Is there any reason that this operation would need to wait?

The method is mentioned in the Geosearch guide to pass X-Forwarded-For to the engine when doing a backend implementation, but the method is not documented at the API client level so it's hard to know it actually exists :)

• Algolia Client Version: 1.23.2
• Language Version: ruby 2.5.0p0 (2017-12-25 revision 61468)

Description

The hits_per_page argument informed on the export_synonyms differ from hitsPerPage expected on search_synonyns causing an issue.

Steps To Reproduce

1. Add index
2. Create 21 synonyms (used 20 here because search_synonyms default hitsPerPage is 20)
3. run the export_synonyms
4. will retrieve only 20 records of the first page (because 20 is < 100)

Workaround

1. run the export_synonyms(20)
2. will retrieve all 21 synonyms (once 20 is not < 20)

Hey guys, I have need of a method that will return the status of a task given the task id. I have created a pull request that makes this change here #198 but in the case that this isn't how you guys would want to go about it I decided to post an issue here. That way if you guys want to go about it a different way there will still be a record of it.

Wanting to generate public search keys scoped by account_id.

Doing this:

generated_public_key = Algolia.generate_secured_api_key("YourSearchOnlyApiKey, "numericFilters=account_id%3D{{account_id}})

And calling it in JS:

var client = algoliasearch('my_application_id', '<%= generated_public_key %>');

And when I get a search the response in the web console is:

{name: "n", message: "Invalid Application-ID or API key"}

I don't get it.

• Algolia Client Version: 1.23.2
• Language Version: 5.1.6

Description

Timeout settings don't work. The gem doesn't throw a timeout exception.

Steps To Reproduce

Configure your client with:

batch_timeout: 120,
connect_timeout: 2,
receive_timeout: 10,
search_timeout: 5,
send_timeout: 10

Hi there

I'm using Algolia for the first time and trying to get reindexing to work locally but it's throwing the error below:

Algolia::AlgoliaError: API not initialized

2016-09-30T06:18:00.071Z 50262 TID-oxqk9ev1w WARN: {"class":"Search::SomeWorker","args":[2,false],"retry":5,"queue":"default","jid":"1519b78812e13d6df5fdb6a9","created_at":1475216280.067702,"enqueued_at":1475216280.067759,**"error_message":"API not initialized","error_class":"Algolia::AlgoliaError"**,"failed_at":1475216280.0707068,"retry_count":0}
**2016-09-30T06:18:00.071Z 50262 TID-oxqk9ev1w WARN: Algolia::AlgoliaError: API not initialized**
2016-09-30T06:18:00.071Z 50262 TID-oxqk9ev1w WARN: /Users/xxx/.rvm/gems/ruby-2.3.1@foro/gems/algoliasearch-1.11.0/lib/algolia/client.rb:657:in `client'
/Users/xxx/.rvm/gems/ruby-2.3.1@foro/gems/algoliasearch-1.11.0/lib/algolia/index.rb:11:in `initialize'
/Users/xxx/.rvm/gems/ruby-2.3.1@foro/gems/algoliasearch-rails-1.15.0/lib/algoliasearch-rails.rb:229:in `new'
/Users/xxx/.rvm/gems/ruby-2.3.1@foro/gems/algoliasearch-rails-1.15.0/lib/algoliasearch-rails.rb:229:in `initialize'
/Users/xxx/.rvm/gems/ruby-2.3.1@foro/gems/algoliasearch-rails-1.15.0/lib/algoliasearch-rails.rb:647:in `new'
/Users/xxx/.rvm/gems/ruby-2.3.1@foro/gems/algoliasearch-rails-1.15.0/lib/algoliasearch-rails.rb:647:in `algolia_ensure_init'
/Users/xxx/.rvm/gems/ruby-2.3.1@foro/gems/algoliasearch-rails-1.15.0/lib/algoliasearch-rails.rb:492:in `block in algolia_index!'
/Users/xxx/.rvm/gems/ruby-2.3.1@foro/gems/algoliasearch-rails-1.15.0/lib/algoliasearch-rails.rb:489:in `each'
/Users/xxx/.rvm/gems/ruby-2.3.1@foro/gems/algoliasearch-rails-1.15.0/lib/algoliasearch-rails.rb:489:in `algolia_index!'
/Users/xxx/.rvm/gems/ruby-2.3.1@foro/gems/algoliasearch-rails-1.15.0/lib/algoliasearch-rails.rb:792:in `algolia_index!'

I had no trouble connecting to the Algolia client as shown below:

[1] pry(main)> Algolia::Client.new(application_id: xxx, api_key: xxx)
=> #<Algolia::Client:0x007fe100329f40
 @api_key="xxx",
 @application_id="xxx",
 @batch_timeout=120,
 @connect_timeout=2,
 @headers=
  {"X-Algolia-API-Key"=>"xxx",
   "X-Algolia-Application-Id"=>"xxx",
   "Content-Type"=>"application/json; charset=utf-8",
   "User-Agent"=>"Algolia for Ruby 1.11.0"},
 @hosts=["xxx.algolia.net", "xxx-1.algolianet.com", "xxx-3.algolianet.com", "xxx-2.algolianet.com"],
 @receive_timeout=30,
 @search_hosts=["xxx-dsn.algolia.net", "xxx-2.algolianet.com", "xxx-1.algolianet.com", "xxx-3.algolianet.com"],
 @search_timeout=5,
 @send_timeout=30,
 @ssl=true,
 @ssl_version=nil>

Any help would be great.

Let me know if required more details.

Thanks

Eric

https://github.com/algolia/algoliasearch-client-ruby/blame/master/README.md#L245

Shouldn't that be operation => instead of operation:?

Hi! I use Algolia search for my application.
There are several filters on my search page. When I try to use parameter filter: 'attribute:name' for
my request, I get a very strange response. It returns empty array in most cases, but sometimes search results are correct.
Please, look at screenshot:

Probably a prerequisite to algolia/algoliasearch-rails#22