Database mappings #

With Elasticsearch, it is possible to define mappings for collections. These mappings allow you to configure the way Elasticsearch will handle these collections.

There are 3 root fields for mapping configuration:

• properties: collection types definition
• dynamic: dynamic mapping policy against new fields
• _meta: collection metadata

The following API methods can be used to modify these mappings:

• collection:create
• collection:updateMapping

Properties types definition #

The field type definitions that will be inserted in a collection allow Elasticsearch to index your data for future searches.

Especially when searching on fields with special types such as date or geo_shape.

Refer to the Elasticsearch documentation for an exhaustive list of available types: Elasticsearch mapping types

Example #

If I want the following document to be correctly indexed:

{
  "category": "limousine",
  "distance": 120990,
  "position": {
    "lat": 27.730400,
    "lon": 85.328467
  },
  "driver": {
    "name": "liia mery"
  }
}

The following mapping must first be defined:

{
  "properties": {
    "category": { "type": "keyword" },
    "distance": { "type": "integer" },
    "position": { "type": "geo_point" },
    "driver": {
      "properties": {
        "name": { "type": "keyword" }
      }
    }
  }
}

This mapping is then passed in the body to the methods collection:create or collection:updateMapping.

# First create a collection yellow-taxi in the nyc-open-index
curl -X PUT -d '{"properties":{"category":{"type":"keyword"},"distance":{"type":"integer"},"position":{"type":"geo_point"},"driver":{"properties":{"name":{"type":"keyword"}}}}}' -H "Content-Type: application/json" "http://localhost:7512/nyc-open-data/yellow-taxi?pretty"

# Then create the desired document
curl -X POST -d '{"category":"limousine","distance":120990,"position":{"lat":27.7304,"lon":85.328467},"driver":{"name":"liia meh ry"}}' -H "Content-Type: application/json" "http://localhost:7512/nyc-open-data/yellow-taxi/_create?pretty"

Dynamic mapping policy #

For each collection, you can set the policy against new fields that are not referenced in the collection mapping by modifying the dynamic root field.

The value of this configuration will change the way Elasticsearch manages the creation of new fields that are not declared in the collection mapping.

• "true": Stores the document and updates the collection mapping with the inferred type
• "false": Stores the document and does not update the collection mapping (fields are not indexed)
• "strict": Rejects the document

Refer to Elasticsearch documentation for more informations: Elasticsearch dynamic mapping

The default policy for new collections is "true" and is configurable in the kuzzlerc file under the key services.db.dynamic.

It is also possible to specify a different dynamic mapping policy for nested fields. This can be useful in imposing a strict policy on the collection while allowing the introduction of new fields in a specific location.

Example #

If you want a strict dynamic policy for your entire collection, you have to define it in root level but you can have a different policy for nested types:

{
  "dynamic": "strict"
  "properties": {
    "driver": {
      "dynamic": "false"
      "properties": // allow insertion of new fields in the driver nested field
    }
  }
}

# Define a strict dynamic policy for the yellow-taxi collection
curl -X PUT -d '{ "dynamic": "strict" }' -H "Content-Type: application/json"  "http://localhost:7512/nyc-open-data/yellow-taxi?pretty"

# Try to create a document with a field that is not referenced in the mapping
curl -X POST -d '{"language":"nepali"}' -H "Content-Type: application/json" "http://localhost:7512/nyc-open-data/yellow-taxi/_create?pretty"

# You should see an error with the following message:
# "mapping set to strict, dynamic introduction of [language] within [yellow-taxi] is not allowed"

Collection metadata #

Elasticsearch allows the definition of metadata that is stored next to the collections in the root field _meta. These metadata are ignored by Elasticsearch, they can contain any type of information specific to your application.

Refer to Elasticsearch documentation for more informations: Elasticsearch mapping meta field

These metadata can be retrieved with the collection:getMapping API method.

Example #

{
  "_meta": {
    "area": "Panipokhari"
  }
}

# Add collection metadata
curl -X PUT -d '{ "_meta": { "area": "Panipokhari" } }' -H "Content-Type: application/json"  "http://localhost:7512/nyc-open-data/yellow-taxi/_mapping?pretty"

# Retrieve it
curl -X GET -H "Content-Type: application/json"  "http://localhost:7512/nyc-open-data/yellow-taxi/_mapping?pretty"

What Now? #

• Learn to work with Persistent Data
• Read our Elasticsearch Cookbook to learn more about how querying works in Kuzzle
• Use document metadata to find or recover documents
• Keep track of data changes using Real-time Notifications