breakroom / snap Goto Github PK

I can't seem to get stats aggregations to work. I get back something like

"unit_price_stats" => %Snap.Aggregation{
      buckets: nil,
      doc_count: nil,
      doc_count_error_upper_bound: nil,
      interval: nil,
      sum_other_doc_count: nil,
      value: nil
    }

I think the problem is that elasticsearch doesn't return buckets etc. for stats aggregations - it returns something like

"aggregations": {
    "grades_stats": {
      "count": 2,
      "min": 50.0,
      "max": 100.0,
      "avg": 75.0,
      "sum": 150.0
    }
  }

Hello all,

Let me first thank you for a well written library. It is really easy to read. I also like how easy to follow the tests are.

We, as developers, can't predict the future and will inevitably have to change the document structure in an index. Why is the hotswap api build the way it is instead of using the native Reindex APIs that ostensibly does the same thing? What was the design decision there?

Opensearch: https://opensearch.org/docs/2.1/opensearch/reindex-data/
Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html

I'm not an Opensearch/Elasticsearch expert so are there any gotchas to be aware of?

Looking at the changelog there are 2 entries for 0.8.0.
The docs on hex are for version 0.8.1 and the commit messages also include "Bump to 0.8.1".
Did something perhaps go wrong with the tagging of the latest release?

Hello, I am working on an application where we work with different companies whose data is stored in their own elasticsearch instances. Currently, we roughly have a custom solution to make queries etc as our use cases until now have been quite basic. We are hoping to do more with elasticsearch and are hoping to use a more robust solution. Looking at the docs for this project, it looks like it would work well for a single tenant but I was wondering if you have any advice on how to use this in a multitenanted application.

Hi,

the docs say that :auth is optional in the config, but when omitted I get an error:

** (KeyError) key :auth not found in: [url: "http://localhost:9200"]
    (elixir 1.11.3) lib/keyword.ex:420: Keyword.fetch!/2
    (snap 0.4.1) lib/snap/request.ex:7: Snap.Request.request/7
    (snap 0.4.1) lib/snap/indexes/indexes.ex:85: Snap.Indexes.list/2

Kind regards,
Nicolas

Hey there! Thanks for an excellent library. I was wondering if you could point me to how I can pass "doc_as_upsert": true to Snap.Bulk.Action.Update. This line in the docs is ambiguous to me:

Any other options, such as pipeline: "foo" are passed through as query parameters to the Bulk API endpoint.

Is that how I should do it?

Patch is used to send a JSON patch e.g. OpenSearch API docs and would be a nice addition to Snap.Clusters.

For large/complex JSON responses, Jsonrs is ~20x faster at decoding in our benchmarks. Consider supporting pluggable JSON libraries, so users can switch to this if they want.

Hello!

I am not able to test as it is in the documentation, following the test guide, the application does not start:

** (MatchError) no match of right hand side value: {:error, {:ioms, {{:shutdown, {:failed_to_start_child, Ioms.Cluster, {:undef, [{HTTPClientMock, :child_spec, [[cluster: Ioms.Cluster, url: "http://localhost:9200", username: "username", password: "password", http_client_adap
ter: HTTPClientMock]], []}, {Snap.Cluster.Supervisor, :maybe_initialize_http_client, 2, [file: 'lib/snap/supervisor.ex', line: 36]}, {Snap.Cluster.Supervisor, :init, 1, [file: 'lib/snap/supervisor.ex', line: 24]}, {:supervisor, :init, 1, [file: 'supervisor.erl', line: 330]},
 {:gen_server, :init_it, 2, [file: 'gen_server.erl', line: 423]}, {:gen_server, :init_it, 6, [file: 'gen_server.erl', line: 390]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}]}}}, {Ioms.Application, :start, [:normal, []]}}}}

I have to remove the Cluster from the application, use setup :set_mox_global with async: false (not desired) and on test setup:

setup do
  Mox.stub(HTTPClientMock, :child_spec, fn _config -> :skip end)
  Ioms.Cluster.start_link([])
end

Is there a better way to test?

Hello!

I've been seeing random %Mint.TransportError{reason: :closed} errors when calling Elasticsearch through Snap, and usually retrying the same query works.

It looks like this is related to connection pooling, with the client keeping connections that are closed by Elasticsearch. Have you seen anything similar? Wanted to ask here before digging deeper into the issue.

when following the test instructions the application fails to start during mix test because the child_spec stub is not defined yet because test_helper.exs runs after the application starts and therefore the cluster attempts to start.

** (Mix) Could not start application my_app: MyApp.Application.start(:normal, []) returned an error: shutdown: failed to start child: MyApp.Cluster
    ** (EXIT) an exception was raised:
        ** (UndefinedFunctionError) function MyApp.HTTPClientMock.child_spec/1 is undefined (module MyApp.HTTPClientMock is not available)
            MyApp.HTTPClientMock.child_spec([cluster: MyApp.Cluster, url: "https://localhost:9999", username: "doesnotexist", password: "notarealpassword", http_client_adapter: MyApp.HTTPClientMock])
            (snap 0.10.0) lib/snap/supervisor.ex:45: Snap.Cluster.Supervisor.maybe_initialize_http_client/2
            (snap 0.10.0) lib/snap/supervisor.ex:29: Snap.Cluster.Supervisor.init/1
            (stdlib 5.1) supervisor.erl:330: :supervisor.init/1
            (stdlib 5.1) gen_server.erl:962: :gen_server.init_it/2
            (stdlib 5.1) gen_server.erl:917: :gen_server.init_it/6
            (stdlib 5.1) proc_lib.erl:241: :proc_lib.init_p_do_apply/3

I've tried using Mox's compile-time requirements but it's not possible to define stubs at compile time.

Am I missing something in this approach?

Hi, today i updated snap and a lot of tests broke down.

I traced the error back to these changes here: 51ec88f

I understood the documentation in way that namespaces are optional:

index_namespace - see [Snap.Cluster.Namespace](https://hexdocs.pm/snap/Snap.Cluster.Namespace.html) for details (defaults to nil)

But these filters basically break the hotswap and cleanup calls if no namespaces are used.

First, let me thank you for putting this library together, it's been working great for us!

Recently we had the need to access the matched_queries key in the returned Elasticsearch documents (exposing to our users which criteria had yielded the result). Because of the way the results are wrapped in the Snap.Hit struct, our only option was to fork your repo and extend the struct. While this solves our immediate need, I do think it's a more general problem. I imagine having access to the highlight key is another common use case.

Do you have any suggestion on how to proceed with this? We'd be happy to work on it and provide a PR.

Cheers