复杂查询

涉及到复杂的查询语句，采用Java High Level REST Client 构建 SearchSourceBuilder 往往不那么直观，我们可以采用更高效的Low Level方式：

1. 在Kibana调试符合预期的查询语句。
2. 如果查询语句需要传入参数，利用占位符替换产生最终查询语句。
3. 调用 Java Low Level REST Client 调用获取查询结果，采用JSON工具类库解析结果集。

示例

一个稍显复杂的Query，存放在项目的资源目录下 /resource/agent-network.json

{
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "@timestamp": {
              "gte": ${from},
              "lte": ${to},
              "format": "epoch_millis"
            }
          }
        },
        {
          "term": {
            "nginx.access.app_name": {
              "value": "${appname}"
            }
          }
        },
        {
          "exists": {
            "field":"nginx.access.user_agent.original"
          }
        }
      ]
    }
  },
  "aggs": {
    ...
  }
}

该示例带有3个参数 日期类型的 from to 和keyword类型的 appname

占位符填充参数值

public class TemplateFileUtils {
    public static String readFile(String classpathFile) {
        Resource resource = new ClassPathResource(classpathFile);
        try {
            return FileCopyUtils.copyToString(new InputStreamReader(resource.getInputStream(), UTF_8));
        } catch (IOException e) {
            throw new RuntimeException("read file io error:" + classpathFile, e);
        }
    }

    public static String replaceHolder(String source, Map<String, Object> params) {
        StrSubstitutor sub = new StrSubstitutor(params, "${", "}");
        return sub.replace(source);
    }
}

该工具依赖 commons-lang3

<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-lang3</artifactId>
</dependency>

构造 Java Low Level REST Client

RestClientBuilder restClientBuilder() {
    Header[] headers = new Header[]{new BasicHeader(HttpHeaders.CONTENT_TYPE, "application/json")};
    final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
    credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
    RestClientBuilder builder = RestClient.builder(HttpHost.create(server))
            .setDefaultHeaders(headers)
            .setHttpClientConfigCallback(httpClientBuilder -> httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider));
    return builder;
}

调用查询

String template = TemplateFileUtils.readFile("agent-network.json");
Calendar c = Calendar.getInstance();
c.add(Calendar.HOUR, -hour);
Date from = c.getTime();
Date to = new Date();
Map<String, Object> params = Map.of(
        "from", from.getTime(),
        "to", to.getTime(),
        "appname", "foo");
String query = TemplateFileUtils.replaceHolder(template, params);

try (RestClient restClient = restClientBuilder.build()) {
    Request request = new Request("GET", "/_search");
    request.setJsonEntity(query);
    Response response = restClient.performRequest(request);
    ...
}