fetchNext #
Fetches the next SearchResult, by triggering a new search/scroll request depending on the options and filters of the SearchResult.
If the previous request was a search or a scroll action which provided a scroll argument, fetchNext will use the scrollId retrieved from the current result to make a new scroll request.
If the previous request was a search action which provided size argument and sort filtering, fetchNext will use Elasticsearch's search_after mechanism, which can efficiently search through a large volume of documents, bypassing internal hard limits[1], but at the cost of reflecting the latest changes of the index, as opposed to using scroll.
If the previous request was a search action which provided from and size arguments, fetchNext will add size to from and make a new search request.
How to process every document of a collection #
The safest way to process all documents in a collection is to fetch them as a batch in order to avoid memory exhaustion and possibly hitting some hard limits[1] on the database layer.
Make sure your first search request includes the size and scroll parameters
[1] Elasticsearch limits the number of documents inside a single page to 10,000 by default.
Usage #
searchResult.fetchNext(new ResponseListener<SearchResult>() {
@Override
public void onSuccess(SearchResult nextSearchResult) {
// called once the fetchNext action has been completed
// nextSearchResult is an instantiated SearchResult object
}
@Override
public void onError(JSONObject error) {
// Handle error
}
});import io.kuzzle.sdk.core.Kuzzle;
import io.kuzzle.sdk.core.Options;
Kuzzle kuzzle = new Kuzzle("localhost");
JSONObject filter = new JSONObject();
Options options = new Options();
options.setFrom((long) 0);
options.setSize((long) 1000);
options.setScroll("30s");
ResponseListener<SearchResult> listener = new ResponseListener<SearchResult>() {
@Override
public void onSuccess(SearchResult searchResult) {
if (searchResult == null) {
return;
}
for (Document doc : searchResult.getDocuments()) {
// do something with the document
// this.processDocument(doc);
}
searchResult.fetchNext(this);
}
@Override
public void onError(JSONObject error) {
// handle errors here
}
};
kuzzle
.collection("collection", "index")
.search(filter, options, listener);