Elasticsearch 7 – query with raw JSON of Kibana

Recently I’ve started using the Elasticsearch (7.6.x) with Java backend (I’m pretty much a beginner in ES) and I have a general question, I’ve searched for that but didn’t find an answer:

My workflow goes like this:

1. I open kibana’s dev tools console and come up with a query to search. Since the syntax is new to me, it takes some time.

2. Then I have RestHighLevelClient in my DAO-s. I’m trying to “translate” the query I’ve created in kibana to numerous builders to build a query with java API. Obviously this also takes some effort.

So I was wondering whether there is a convenient API that allows me to execute the JSON query that I’ve prepared during step 1 directly on ElasticSearch via the Java API. If it exists I could use some template engine to fill the placeholders, and execute the compiled template with applied placeholders resolution. I see a big advantage of this method because it will allow me to bypass my step “2” altogether…

An Example of Query in Kibana:

GET telemetry/_search
{
  
  "aggs": {
    "value": {
      "histogram": {
        "field": "value",
        "interval": 50
      }
    }
  }
} 

So I could store the json file in my src/main/resources or something and since “interval” is a parameter, it could have looked like this:

{
  
  "aggs": {
    "value": {
      "histogram": {
        "field": "value",
        "interval": {{interval_param}}
      }
    }
  }
} 

Then a template engine would compile the query (it can be done only once, but it doesn’t really relate to the question) and the query json could be executed directly:

TemplateEngine engine ...
String query = engine.compile(json_resource).execute(Map.of("interval_param", 50))
API_THAT_IM_LOOKING_FOR.executeQuery(query, "telemetry")

If such an API exists what are the “pro” and “contra” of this approach? Will it be less performant as opposed to Java API? Will it have some security related constaints (stuff like SQL-like injection, etc)?

Answer

You could use the Java Low Level REST Client and pass your query body as string directly via this method:

request.setJsonEntity("{"json":"text"}");

In general, the Low Level REST Client should be a bit faster because it doesn’t serialize requests or parse responses. Drawbacks would be additional code and these query strings might be harder to maintain. That said, i understand that you would prefer passing query bodies directly as strings because the high level rest client is IMHO a bit unintuitive and overengineered.

The Low Level REST Client is providing you with load balancing across the available nodes and failover mechanics, and is used by the High Level REST Client under the hood.

You could also just use any HTTP client to directly talk to your Elasticsearch instance. I once worked on a c# project where we did just that after we realized that the overhead introduced by the high level .NET client was serious due to it using a lot of reflection. You then have to implement your own load balancing or failover strategies.