Internal Errigal Collaboration Wiki

User Tools

Site Tools

Trace: • elasticsearch
development:elasticsearch:elasticsearch

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
development:elasticsearch:elasticsearch [2021/12/13 19:49] – slawrencedevelopment:elasticsearch:elasticsearch [2021/12/16 07:15] (current) – slawrence
Line 1: Line 1:
 __**Re-Index ElasticSearch**__ __**Re-Index ElasticSearch**__
 +
 +REF: https://www.thirdrocktechkno.com/blog/6-steps-to-reindex-elasticsearch-data/
  
 There are potentially several situations where you may need to recreate the current Elastic Search index.  This article is being written to handle one of the most common causes related to field data-type mapping. There are potentially several situations where you may need to recreate the current Elastic Search index.  This article is being written to handle one of the most common causes related to field data-type mapping.
Line 7: Line 9:
 If such an affecting data-type change is released after the first of the month, this can lead to errors in the MDC Tracker log file (~/logs/spring-boot/tracker.log) such as: If such an affecting data-type change is released after the first of the month, this can lead to errors in the MDC Tracker log file (~/logs/spring-boot/tracker.log) such as:
  
 +<WRAP center round box 90%>
 **//org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [parameters.winlink1000OduAirSesState.currentValue] of type [float] in document with id 'HCKGgn0BEcj44NiWqUh6'. Preview of field's value: 'scanning']//** **//org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [parameters.winlink1000OduAirSesState.currentValue] of type [float] in document with id 'HCKGgn0BEcj44NiWqUh6'. Preview of field's value: 'scanning']//**
 +</WRAP>
  
 This error implies that the existing data mapping for the current ElasticSearch index was expecting data of type 'float' for 'parameters.winlink1000OduAirSesState.currentValue', but instead received the String value 'scanning'.   This error implies that the existing data mapping for the current ElasticSearch index was expecting data of type 'float' for 'parameters.winlink1000OduAirSesState.currentValue', but instead received the String value 'scanning'.  
Line 13: Line 17:
 To resolve this situation, the basic process is to look up existing indices, create a new schema with some manually specified data-type mapping, copy the existing index data to the new index, and then delete the old index.  Commands to perform this can be executed in a terminal connected to the 'esk' server instance, or wherever elasticsearch is running for the affected environment. To resolve this situation, the basic process is to look up existing indices, create a new schema with some manually specified data-type mapping, copy the existing index data to the new index, and then delete the old index.  Commands to perform this can be executed in a terminal connected to the 'esk' server instance, or wherever elasticsearch is running for the affected environment.
  
-To begin, log into the server running elasticsearch via SSH in a termanal window (i.e. newtowerqaesk1.err or the like).  Then:+**To begin, log into the server running elasticsearch via SSH in a termanal window (i.e. newtowerqaesk1.err or the like).  Then: 
 +** 
 +**1.  Retrieve existing indices: 
 +** 
 +**RUN: 
 +** 
 +<WRAP center round box 80%> 
 +curl -X GET 'http://localhost:9200/_cat/indices/%2A?v=&s=index:desc' 
 +</WRAP>
  
-1.  Retrieve existing indices: 
- 
-RUN: 
- 
-curl -X GET \'http://localhost:9200/_cat/indices/%2A?v=&s=index:desc' \ 
- 
- 
-GET: 
  
 +**GET:
 +**
 <WRAP center round box 95%> <WRAP center round box 95%>
 health status index                     uuid                   pri rep docs.count docs.deleted store.size pri.store.size health status index                     uuid                   pri rep docs.count docs.deleted store.size pri.store.size
 +
 yellow open   rdf_performance_2021-12   FrgUteO8SwWYAjr_bBzeTA   1   1        638            0        1mb            1mb yellow open   rdf_performance_2021-12   FrgUteO8SwWYAjr_bBzeTA   1   1        638            0        1mb            1mb
  
Line 63: Line 70:
  
  
-2.  Create a new index for the data, here we want to update mapping for:  'parameters.winlink1000OduAirSesState.currentValue' +**2.  Create a new index for the data. In this example, we want to update mapping for:  **'parameters.winlink1000OduAirSesState.currentValue'** and **'parameters.winlink1000OduAirSesState.newValue'**. 
- +** 
-(NOTE:  DOING AGAIN FOR:  parameters.winlink1000OduAirSesState.newValue) +**RUN:  
- +** 
-RUN:  +<WRAP center round box 90%> 
- +curl -X PUT \http://localhost:9200/rdf_performance_2021-12_v2 \-H 'Content-Type: application/json' \-d '{
-curl -X PUT \ +
-http://localhost:9200/rdf_performance_2021-12_v2 \ +
--H 'Content-Type: application/json' \ +
--d '{+
     "mappings": {     "mappings": {
         "properties": {         "properties": {
-                "parameters.winlink1000OduAirSesState.currentValue": {+               "parameters.winlink1000OduAirSesState.currentValue": {
                     "type": "text"                     "type": "text"
                 },                 },
Line 82: Line 85:
                 }                 }
         }         }
- 
     }     }
 }' }'
 +</WRAP>
  
-GET: 
  
 +**GET:
 +**
 +<WRAP center round box 90%>
 {"acknowledged":true,"shards_acknowledged":true,"index":"rdf_performance_2021-12_v3"} {"acknowledged":true,"shards_acknowledged":true,"index":"rdf_performance_2021-12_v3"}
  
 +</WRAP>
  
-3.  Transfer data into new index? 
- 
-RUN: 
  
 +**3.  Transfer data into new index, very important.  Do not proceed if this step fails.  If it does for any reason, start over again with a new index name to transfer to.
 +**
 +**RUN (IF YOU CUT AND PASTE, REDUCE LINE SPACES, IMPORTANT):
 +**
 +<WRAP center round box 90%>
 curl -X POST \ curl -X POST \
     http://localhost:9200/_reindex \     http://localhost:9200/_reindex \
Line 106: Line 114:
     }     }
 }' }'
 +</WRAP>
  
-GET: 
  
 +**GET:
 +**
 +<WRAP center round box 90%>
 {"took":927,"timed_out":false,"total":638,"updated":0,"created":638,"deleted":0,"batches":1,"version_conflicts":0,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1.0,"throttled_until_millis":0,"failures":[]} {"took":927,"timed_out":false,"total":638,"updated":0,"created":638,"deleted":0,"batches":1,"version_conflicts":0,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1.0,"throttled_until_millis":0,"failures":[]}
 +</WRAP>
  
  
-4.  Verify  (This step may be bad...?) +**4.  Verify  (This step does not always work for some reason, though it is ok if it fails) 
- +** 
-RUN: +**RUN: 
 +** 
 +<WRAP center round box 90%>
 curl -X GET \ curl -X GET \
  
Line 125: Line 138:
     }     }
 }' }'
 +</WRAP>
  
-GET: +**5.  Delete old index since the transfer in step 3 went well.  Again, if the transfer failed, do not proceed, start over. 
- +** 
-5.  Delete old index? +**RUN: 
- +** 
-RUN: +<WRAP center round box 90%>
 curl -X DELETE \ curl -X DELETE \
 http://localhost:9200/rdf_performance_2021-12 http://localhost:9200/rdf_performance_2021-12
 +</WRAP>
  
- +**GET: 
-GET: +** 
 +<WRAP center round box 80%>
 {"acknowledged":true} {"acknowledged":true}
 +</WRAP>
  
- +**6. Add new index finalized with an alias to the previous in case anything is specifically looking for the old key... which should not happen... but just to be safe. 
-6.1  Alias +** 
- +**RUN:   
-RUN: +** 
- +<WRAP center round box 90%>
-curl -X POST \ +
-http://localhost:9200/_aliases \ +
--d '{ +
-    "actions": [ +
-        { +
-            "remove": { +
-                "index": "rdf_performance_2021-12_v2", +
-                "alias": "rdf_performance_2021-12" +
-            } +
-        } +
-    ] +
-}' +
- +
- +
-GET:  NOTE:  DID NOT WORK!!!  Maybe nothing to remove? +
- +
-6.2 +
- +
-RUN:   +
 curl -X POST \ curl -X POST \
 http://localhost:9200/_aliases \ http://localhost:9200/_aliases \
Line 178: Line 173:
     ]     ]
 }' }'
 +</WRAP>
  
-GET:  
  
 +**GET: 
 +**
 +<WRAP center round box 90%>
 {"acknowledged":true} {"acknowledged":true}
 +</WRAP>
 +
  
-6.3 ALL DONE!+**7. ALL DONE!**
development/elasticsearch/elasticsearch.1639424960.txt.gz · Last modified: 2021/12/13 19:49 by slawrence

Page Tools

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki