Java elasticsearch create index

ElasticSearch: JAVA API for ElasticSearch

ElasticSearch 6. X: ElasticSearch 6. X: ElasticSearch 6.

ElasticSearch JAVA API

There are several common ElasticSearch Java API rack packages available in the market. ElasticsearchTemplate, Elasticsearch Bboss, ElasticsearchTemplate, Elasticsearch Bboss, JestClient, SpringBoot integration of SpringData, ElasticsearchTemplate, Elasticsearch Bboss, etc. Except that the SUPPORTED ES version will be lower.

The Java High Level REST Client for ElasticSearch is available on versions 6.x and later. The Java High Level REST Client for ElasticSearch is available on versions 6.x and later. The JDK is required to be above 1.8, which can be well compatible in the large version, and the package itself also contains Java Low Level REST Client methods, which can deal with some special needs for special processing, it is for some commonly used methods encapsulation Restful style, Can be directly corresponding to the operation name call use, support synchronous and asynchronous (Async) call.

Here we can also directly correspond to the previous article in the USE of DSL statements, so that it is very convenient to compare and learn to use.

For Elasticsearch Java High Level REST Client, you need to initialize the connection.

 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost(elasticIp, elasticPort))); Copy the code

It is also very simple to close the client. The client is not empty.

1. New data

ElasticSearch can add data directly, as long as you specify index and type. You can specify the primary key ID when adding it, or you can not specify the primary key ID, which is generated by ElasticSearch. Elasticsearch Java High Level REST Client provides three methods for creating new data.

New data created by jsonString code example 1:

String index = "test1"; String type = "_doc"; // Unique id String ; IndexRequest request = new IndexRequest(index, type, id); String jsonString = ""; request.source(jsonString, XContentType.JSON); IndexResponse indexResponse = client.index(request, RequestOptions.DEFAULT);Copy the code

New data code example 2, created by map, will be automatically converted to JSON data:

String index = "test1"; String type = "_doc"; // Unique id String ; IndexRequest request = new IndexRequest(index, type, id); MapString, Object jsonMap = new HashMap(); jsonMap.put("uid", 1234); jsonMap.put("phone", 12345678909L); jsonMap.put("msgcode", 1); jsonMap.put("sendtime", "2019-03-14 01:57:04"); jsonMap.put("message", "xuwujing study Elasticsearch"); request.source(jsonMap); IndexResponse indexResponse = client.index(request, RequestOptions.DEFAULT);Copy the code

New data code example 3, from the XContentBuilder object to create:

String index = "test1"; String type = "_doc"; // Unique id String ; IndexRequest request = new IndexRequest(index, type, id); XContentBuilder builder = XContentFactory.jsonBuilder(); builder.startObject(); < builder.field("uid", 1234); builder.field("phone", 12345678909L); builder.field("msgcode", 1); builder.timeField("sendtime", "2019-03-14 01:57:04"); builder.field("message", "xuwujing study Elasticsearch"); >builder.endObject(); request.source(builder); IndexResponse indexResponse = client.index(request, RequestOptions.DEFAULT);Copy the code

Of the above three methods, I recommend the second one, which is easier to understand and use.

Create an index library

In the example above, we create data through directly through the library, thus creating the index but not create indexes library generated by ES itself this is not friendly, because it will use the default configuration, the field structure is the text (text data segmentation, at the time of storage will be additional footprint), copy of fragmentation and index using the default, The default values are 5 and 1. The shard number of ES cannot be changed after creation except for reindex, so we will specify the data template for creation. There are three ways to create an index library using the JAVA API, as above, but only one of them is described here.

Example of code for adding an index library:

private static void createIndex() throws IOException < String type = "_doc"; String index = "test1"; // Setting MapString, Object setMapping = new HashMap(); // Number of partitions, number of copies, and cache refresh time setMapping. put("number_of_shards", 10); setmapping.put("number_of_replicas", 1); setmapping.put("refresh_interval", "5s"); MapString, Object keyword = new HashMap(); // Set the type keyword. Put ("type", "keyword"); MapString, Object lon = new HashMap(); // Set type lon. Put ("type", "long"); MapString, Object date = new HashMap(); // Set the type date.put("type", "date"); date.put("format", "yyyy-MM-dd HH:mm:ss"); MapString, Object jsonMap2 = new HashMap(); MapString, Object properties = new HashMap(); // Set the field message properties. Put ("uid", lon); properties.put("phone", lon); properties.put("msgcode", lon); properties.put("message", keyword); properties.put("sendtime", date); MapString, Object mapping = new HashMap(); mapping.put("properties", properties); jsonMap2.put(type, mapping); GetIndexRequest getRequest = new GetIndexRequest(); getRequest.indices(index); getRequest.local(false); getRequest.humanReadable(true); boolean exists2 = client.indices().exists(getRequest, RequestOptions.DEFAULT); If (exists2) CreateIndexRequest request = new CreateIndexRequest(index); Try catch (IOException e) < e.printStackTrace(); >>Copy the code

Note: when creating an index library, you must first check whether the index library exists!! This alias is a good thing to use properly to improve query performance. We will save it for next time.

3. Modify data

When ES provides the API for modifying, there are two ways: one is to modify directly, but if the data does not exist, it will throw an exception; the other is to update the data and insert it if it does not exist. The second one is easier to use than the first one, but not as fast as the first one.

Examples of code for ES modifications:

private static void update() throws IOException < String type = "_doc"; String index = "test1"; // Unique id String ; UpdateRequest upateRequest = new UpdateRequest(); upateRequest.id(id); upateRequest.index(index); upateRequest.type(type); MapString, Object jsonMap = new HashMap(); jsonMap.put("uid", 12345); jsonMap.put("phone", 123456789019L); jsonMap.put("msgcode", 2); jsonMap.put("sendtime", "2019-03-14 01:57:04"); jsonMap.put("message", "xuwujing study Elasticsearch"); upateRequest.doc(jsonMap); // The upsert method adds a new upaterequest.docasupsert (true) if the data does not exist; client.update(upateRequest, RequestOptions.DEFAULT); System.out.println(" Update successful!" ); >Copy the code

Note: the upsert method means that if the data does not exist, then a new one will be created. The default is false.

4. Delete data

The DELETE method is already known from the above operations, so let’s get started.

ES Examples of code to delete by ID:

private static void delete() throws IOException < String type = "_doc"; String index = "test1"; // Unique id String ; DeleteRequest deleteRequest = new DeleteRequest(); deleteRequest.id(id); deleteRequest.index(index); deleteRequest.type(type); / / set the timeout deleteRequest. A timeout (TimeValue. TimeValueMinutes (2)); // Set the refresh policy "wait_for" // keep this request open until the refresh makes the contents of this request searchable. This refresh strategy with high throughput compatible index and search, but it can lead to request waiting for a response, until the refresh deleteRequest. SetRefreshPolicy (WriteRequest. RefreshPolicy. WAIT_UNTIL); DeleteResponse DeleteResponse = client. Delete (deleteRequest, requestOptions.default); >Copy the code

ES deletes according to the condition:

private static void deleteByQuery() throws IOException < String type = "_doc"; String index = "test1"; DeleteByQueryRequest request = new DeleteByQueryRequest(index,type); Request. SetQuery (QueryBuilders. TermsQuery ("uid",1234)); BulkByScrollResponse bulkResponse = Client.deleteByQuery (Request, requestOptions.default); >Copy the code

The test results

The query

Several common query apis are briefly introduced here, and then all query statement codes are presented directly.

Query API

• Term query: QueryBuilders. TermQuery (name,value);
• More value (terms) query: QueryBuilders termsQuery (name, value, value2, value3. ). ;
• Query: QueryBuilders.rangequery (name).gte(value).lte(value);
• (exists) query: QueryBuilders existsQuery (name);
• Fuzzy (wildcard) query: QueryBuilders. WildcardQuery (name,+value+);
• BoolQuery: BoolQueryBuilder BoolQueryBuilder = New BoolQueryBuilder();

Query all code examples

private static void allSearch() throws IOException < SearchRequest searchRequestAll = new SearchRequest(); SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); searchSourceBuilder.query(QueryBuilders.matchAllQuery()); searchRequestAll.source(searchSourceBuilder); SearchResponse searchResponseAll = client.search(searchRequestAll, requestOptions.default); System. The out. Println (" the total number of all the query: "+ searchResponseAll. GetHits () getTotalHits ()); >Copy the code

General query code sample

In fact, it is equivalent query, but in the inside added paging, sorting, timeout, routing and so on Settings, and in the query results added some processing.

private static void genSearch() throws IOException < String type = "_doc"; String index = "test1"; SearchRequest SearchRequest = new SearchRequest(index); searchRequest.types(type); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); // set the query condition sourceBuilder.query(QueryBuilders. TermQuery ("uid", "1234")); Sourcebuilder. from(0); sourceBuilder.size(5); sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS)); Searchrequest. routing("routing"); / / set the index expression library searchRequest. IndicesOptions (indicesOptions. LenientExpandOpen ()); SearchRequest. Preference ("_local"); Sortresortbuilder.sort (new ScoreSortBuilder().order(sortorder.desc)); Sourcebuilder.sort (new FieldSortBuilder("id").order(sortOrder.asc)); / / close suorce query / / sourceBuilder fetchSource (false); String[] includeFields = new String[]; String[] excludeFields = new String[]; / / / / sourceBuilder include or exclude field fetchSource (includeFields excludeFields); searchRequest.source(sourceBuilder); System.out.println(" Plain query DSL statement :"+sourceBuilder.toString()); SearchResponse = client.search(searchRequest, requestOptions.default); RestStatus Status = searchResponse.status(); RestStatus status = searchResponse.status(); TimeValue took = searchResponse.getTook(); Boolean terminatedEarly = searchResponse.isTerminatedEarly(); boolean timedOut = searchResponse.isTimedOut(); / / for the segmentation of the total number of affected by search statistics, as well as the success and failure of segmentation int totalShards. = the searchResponse getTotalShards (); int successfulShards = searchResponse.getSuccessfulShards(); int failedShards = searchResponse.getFailedShards(); For (ShardSearchFailure Failure: The searchResponse. GetShardFailures ()) / / result the searchResponse getHits (). The forEach (hit - < MapString, Object map = hit.getSourceAsMap(); System.out.println(" normal query result :" + map); >); System.out.println("\n=================\n"); >Copy the code

Or query

In fact, this or query is one of those bool queries, where the query statement is equivalent to the SQL statement

SELECT * FROM test1 where (uid = 1 or uid =2) and phone = 12345678919

private static void orSearch() throws IOException < SearchRequest searchRequest = new SearchRequest(); searchRequest.indices("test1"); searchRequest.types("_doc"); SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder(); BoolQueryBuilder boolQueryBuilder2 = new BoolQueryBuilder(); /** * SELECT * FROM test1 where (uid = 1234 or uid =12345) and phone = 12345678909 * */ boolQueryBuilder2.should(QueryBuilders.termQuery("uid", 1234)); boolQueryBuilder2.should(QueryBuilders.termQuery("uid", 12345)); boolQueryBuilder.must(boolQueryBuilder2); boolQueryBuilder.must(QueryBuilders.termQuery("phone", "12345678909")); searchSourceBuilder.query(boolQueryBuilder); System. The out. Println (" or query statement: "+ searchSourceBuilder. ToString ()); searchRequest.source(searchSourceBuilder); SearchResponse = client.search(searchRequest, requestOptions.default); The searchResponse. GetHits (). The forEach (documentFields - ); >Copy the code

Fuzzy query

Equivalent to a like query in an SQL statement.

private static void likeSearch() throws IOException < String type = "_doc"; String index = "test1"; SearchRequest searchRequest = new SearchRequest(); searchRequest.indices(index); searchRequest.types(type); SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder(); /** * SELECT * FROM p_test where message like '%xu%'; * */ boolQueryBuilder.must(QueryBuilders.wildcardQuery("message", "*xu*")); searchSourceBuilder.query(boolQueryBuilder); System. The out. Println (" fuzzy query: "+ searchSourceBuilder. ToString ()); searchRequest.source(searchSourceBuilder); SearchResponse = client.search(searchRequest, requestOptions.default); The searchResponse. GetHits (). The forEach (documentFields - ); System.out.println("\n=================\n"); >Copy the code

Multivalued query

This is the equivalent of an IN query in an SQL statement.

private static void inSearch() throws IOException < String type = "_doc"; String index = "test1"; SearchRequest SearchRequest = new SearchRequest(index,type); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); Query(QueryBuilders. TermsQuery ("uid", 1,2)); /** * SELECT * FROM p_test where uid in (1,2); searchRequest.source(sourceBuilder); System.out.println("in query DSL statement :"+sourceBuilder.toString()); SearchResponse = client.search(searchRequest, requestOptions.default); Searchresponse.gethits ().foreach (hit - ); System.out.println("\n=================\n"); >Copy the code

There is a query

Check whether the field exists. The usage is similar to exist in the SQL statement.

private static void existSearch() throws IOException < String type = "_doc"; String index = "test1"; SearchRequest SearchRequest = new SearchRequest(index); searchRequest.types(type); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); / / set the query conditions sourceBuilder. Query (QueryBuilders. ExistsQuery (" msgcode ")); searchRequest.source(sourceBuilder); System.out.println(" existing query DSL statement :"+sourceBuilder.toString()); SearchResponse = client.search(searchRequest, requestOptions.default); Searchresponse.gethits ().foreach (hit - ); System.out.println("\n=================\n"); >Copy the code

Range queries

In the SQL statement, gt is greater than, LT is less than, GTE is greater than or equal to, and LTE is less than or equal to.

private static void rangeSearch() throws IOException< String type = "_doc"; String index = "test1"; SearchRequest searchRequest = new SearchRequest(index); searchRequest.types(type); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); // Set the query condition sourceBuilder.query(QueryBuilders. RangeQuery ("sendtime").gte("2019-01-01 00:00:00").lte("2019-12-31 ") 23:59:59 ")); searchRequest.source(sourceBuilder); System.out.println(" Range query DSL statement :"+sourceBuilder.toString()); SearchResponse = client.search(searchRequest, requestOptions.default); Searchresponse.gethits ().foreach (hit - ); System.out.println("\n=================\n"); >Copy the code

Regular query

ES can use the re to query, query is also very simple, the code example is as follows:

private static void regexpSearch() throws IOException< String type = "_doc"; String index = "test1"; SearchRequest SearchRequest = new SearchRequest(index); searchRequest.types(type); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); / / set the query conditions sourceBuilder. Query (QueryBuilders. RegexpQuery (" message ", "xu 1")); searchRequest.source(sourceBuilder); System.out.println(" regular query DSL statement :"+sourceBuilder.toString()); SearchResponse = client.search(searchRequest, requestOptions.default); Searchresponse.gethits ().foreach (hit - ); System.out.println("\n=================\n"); >Copy the code

Querying test Results

other

ES official documents: www.elastic.co/guide/en/el.

SpringBoot integration with ElasticSearch and JestClient can be found in this article: SpringBoot integration with ElasticSearch

For ElasticSearch Java API options, JestClient is recommended if the ElasticSearch version is older than 6.x. If you want to upgrade to 7.x after 6.x, you can use the official Java High Level REST Client of ES directly, because the connection mode of Transport Client will be discarded after 7.x. The ES integrated with Spring and SpringBoot currently uses this method (I wonder if it will be adjusted later).

The code for this article has been included in my Java-Study project, so if you are interested, welcome to Star, fork, and Issues. Project address :github.com/xuwujing/ja.

ElasticSearch combat series: ElasticSearch combat series a: ElasticSearch cluster + Kinaba installation tutorial ElasticSearch combat series 2: ElasticSearch DSL statements use tutorial — graphic explanation

Источник

Java elasticsearch create index

Each index created can have specific settings associated with it.

request.settings(Settings.builder() .put("index.number_of_shards", 3) .put("index.number_of_replicas", 2) );

Index mappingsedit

An index may be created with mappings for its document types

request.mapping( "\n" + " >\n" + ">", XContentType.JSON);

The mapping for this type, provided as a JSON string

The mapping source can be provided in different ways in addition to the String example shown above:

Map message = new HashMap<>(); message.put("type", "text"); Map properties = new HashMap<>(); properties.put("message", message); Map mapping = new HashMap<>(); mapping.put("properties", properties); request.mapping(mapping);

Mapping source provided as a Map which gets automatically converted to JSON format

XContentBuilder builder = XContentFactory.jsonBuilder(); builder.startObject(); < builder.startObject("properties"); < builder.startObject("message"); < builder.field("type", "text"); >builder.endObject(); > builder.endObject(); > builder.endObject(); request.mapping(builder);

Mapping source provided as an XContentBuilder object, the Elasticsearch built-in helpers to generate JSON content

Index aliasesedit

Aliases can be set at index creation time

request.alias(new Alias("twitter_alias").filter(QueryBuilders.termQuery("user", "kimchy")));

Providing the whole sourceedit

The whole source including all of its sections (mappings, settings and aliases) can also be provided:

request.source(",\n" + " \"mappings\" : \n" + " >\n" + " >,\n" + " \"aliases\" : \n" + " >\n" + ">", XContentType.JSON);

The source provided as a JSON string. It can also be provided as a Map or an XContentBuilder .

Optional argumentsedit

The following arguments can optionally be provided:

request.setTimeout(TimeValue.timeValueMinutes(2));

Timeout to wait for the all the nodes to acknowledge the index creation as a TimeValue

request.setMasterTimeout(TimeValue.timeValueMinutes(1));

Timeout to connect to the master node as a TimeValue

request.waitForActiveShards(ActiveShardCount.from(2)); request.waitForActiveShards(ActiveShardCount.DEFAULT);

The number of active shard copies to wait for before the create index API returns a response, as an int

The number of active shard copies to wait for before the create index API returns a response, as an ActiveShardCount

Synchronous Executionedit

When executing a CreateIndexRequest in the following manner, the client waits for the CreateIndexResponse to be returned before continuing with code execution:

CreateIndexResponse createIndexResponse = client.indices().create(request, RequestOptions.DEFAULT);

Synchronous calls may throw an IOException in case of either failing to parse the REST response in the high-level REST client, the request times out or similar cases where there is no response coming back from the server.

In cases where the server returns a 4xx or 5xx error code, the high-level client tries to parse the response body error details instead and then throws a generic ElasticsearchException and adds the original ResponseException as a suppressed exception to it.

Asynchronous Executionedit

Executing a CreateIndexRequest can also be done in an asynchronous fashion so that the client can return directly. Users need to specify how the response or potential failures will be handled by passing the request and a listener to the asynchronous create-index method:

client.indices().createAsync(request, RequestOptions.DEFAULT, listener);

The CreateIndexRequest to execute and the ActionListener to use when the execution completes

The asynchronous method does not block and returns immediately. Once it is completed the ActionListener is called back using the onResponse method if the execution successfully completed or using the onFailure method if it failed. Failure scenarios and expected exceptions are the same as in the synchronous execution case.

A typical listener for create-index looks like:

ActionListener listener = new ActionListener() > @Override public void onFailure(Exception e) > >;

Called when the execution is successfully completed.

Called when the whole CreateIndexRequest fails.

Create Index Responseedit

The returned CreateIndexResponse allows to retrieve information about the executed operation as follows:

boolean acknowledged = createIndexResponse.isAcknowledged(); boolean shardsAcknowledged = createIndexResponse.isShardsAcknowledged();

Indicates whether all of the nodes have acknowledged the request

Indicates whether the requisite number of shard copies were started for each shard in the index before timing out

Источник