Scroll API support about elasticsearch-java HOT 6 CLOSED

Thanks for the feedback @gquintana.

I'm a bit confused by the first part of your question, which is a search request (it accepts a scroll duration but not a scroll_id) whereas the code in the second one is about clearing a scroll context.

For the second part there is indeed an issue in the way Elasticsearch interprets the request that was sent by the client. And this is fixed in the main and 7.16 branches.

If you're interested in the details: the clear scroll request accepts scroll_id both as a path segment and as a body property. In the java client version 7.15, the id is sent in the path with an empty object as the body. When a body is present (even if an empty object), Elasticsearch reads the scroll_id in the body and ignores the one in the path.

We've changed the code generation to prioritize body properties when a parameter exists both on the url and in the body. This solves the issue for this API and also for others. The change will be available in the 7.16.0-SNAPSHOT version starting tomorrow.

We still have a remaining issue with this endpoint: Elasticsearch returns 404 with a valid response when no scroll context is cleared. The client will then try to parse the 404 response as an error and fail to do so. We're working on fixing this too and I'll update this issue.

Thanks again for your feedback!

from elasticsearch-java.

Thanks for you answer @swallez .

I'll try to make myself clearer. AFAIK, there are 3 parts when using scroll API:

1. Run standard query with scroll time out and get a scroll Id along with the first page of results
2. Use the scroll id to get the next page of results, again and again until the end
3. USe the scroll id to close the scroll (usefull for error handling on client side) and free up resources on ES server side

I tried to implement the 3 parts with this API

1. Is alright
2. Couldn't find how to implement it -> first part of the question https://www.elastic.co/guide/en/elasticsearch//reference/7.12/scroll-api.html#scroll-api
3. Raises an exception -> second part of the question

from elasticsearch-java.

So right now, since only a "scroll" duration can be specified when building the search request and no way to set the scrollId after getting the first page of results, we'd have to use the REST client correct?

from elasticsearch-java.

@mindscratch correct, the _search/scroll API endpoint is missing in version 7.15 of the java client. Is has been added in the upcoming 7.16.

Furthermore the issues reported by @gquintana have been solved too (see test).

For the nasty details, this came from the fact that in this API endpoint Elasticsearch will ignore the scroll_id request parameter when there is a body, even if doesn't contain a scroll_id property. And we were sending the request parameter with an empty body, hence this error. We've now send scroll_id as a request body property and Elasticsearch is happy 😉

from elasticsearch-java.

@swallez thanks for the follow-up, is there an ETA for when the 7.16.0 version will be released?

from elasticsearch-java.

@mindscratch I can't give an exact date, but it should be out in the next two weeks!

from elasticsearch-java.