Timestamp field in Elasticsearch

Consider the following scenario, that you want to know when a specific document is added to Elasticsearch. In initial versions of Elasticsearch, each document contained a meta field named @timestamp that contained information about the date the document was indexed.

However, it is currently impossible to tell when a document was added to Elasticsearch. There are two methods for achieving this.

From the Application level, you can add time to each document.
To add time to each document, use the ingest pipeline. To do that, follow the instructions below:

Steps to add timestamp field in each document using Ingest pipeline
Ingest pipeline is created as depicted below. There are 3 processors in the ingest pipeline :

The timestamp field's value for "ingest.timestamp" is set using the set processor command.
NOTE: ingest.timestamp stores the time at which the document is added in Elasticsearch. The time is in this format 2022-11-12T07:48:45.760165787Z
The timestamp can be changed to Epoch time by a script processor.
The timestamp field can be removed by the delete processor. Since, according to the script, now @timestamp stores the epoch time.

PUT /_ingest/pipeline/ts-pipeline
{
   "description": "Add timestamp field to all documents",
   "processors": [
       {
           "set": {
               "field": "timestamp",
               "value": "{{_ingest.timestamp}}"
           }
       },
       {
           "script": {
               "id":"calculate-timestamp"
           }
       },
       {
           "remove": {
               "field": "timestamp"
           }
       }
   ]
}

The following script is used by the script processor to change the timestamp from date format to epoch format.

  DateTimeFormatter dtf = DateTimeFormatter.ofPattern(\"yyyy-MM-dd'T'HH:mm:ss.SSSSSSSSS'Z'\"); LocalDateTime ldt = LocalDateTime.parse(ctx['timestamp'], dtf); ZonedDateTime zdt = ldt.atZone(ZoneId.of('Z')); ctx['@timestamp']=zdt.toInstant().toEpochMilli();
  When documents are indexed, @timestamp the field is added to each document (in epoch format)

Working Example

Using the previously created ingest pipeline, add a document to the index.

  POST timestamp-test/_doc/1?pipeline=ts-pipeline
  {
    "city":"indore"
  }

Retrieve the added document using the Get API

  GET /timestamp-test/_doc/1
  {
      "_index": "timestamp-test",
      "_id": "1",
      "_version": 1,
      "_seq_no": 0,
      "_primary_term": 1,
      "found": true,
      "_source": {
          "city": "indore",
          "timestamp": "2022-11-12T07:48:45.760165787Z"
      }
  }

You'll notice that Elasticsearch has added a new field called @timestamp that contains the time the document was added.

Timestamp field in Elasticsearch

Steps to add timestamp field in each document using Ingest pipeline