Painless Enrichment
Search requirements can change faster than your index schema. In a retail product catalog, you may need new derived fields for filtering or reporting after the products index is already live.
Painless is the built-in scripting language for Elasticsearch. If the base fields already exist (price, list_price, cost, inventory_count), /_update_by_query can enrich documents in place without needing a full reindex.
Map the new fields
Before touching any documents, add the derived fields to the products mapping. If you skip this, dynamic mapping picks the types on first write, and price_band ends up as text with a keyword subfield.
PUT /products/_mapping
{
"properties": {
"discount_pct": { "type": "integer" },
"margin_pct": { "type": "integer" },
"price_band": { "type": "keyword" },
"stock_state": { "type": "keyword" }
}
}
Dry run
It’s worth testing the math with /_scripts/painless/_execute and some sample values first, so a bad formula shows up on one fake product instead of the whole catalog.
POST /_scripts/painless/_execute
{
"script": {
"source": """
double discount = (params.list_price - params.price) * 100.0 / params.list_price;
double margin = (params.price - params.cost) * 100.0 / params.price;
return [
"discount_pct": (int) Math.round(discount),
"margin_pct": (int) Math.round(margin)
];
""",
"params": {
"price": 79.99,
"list_price": 99.99,
"cost": 42.00
}
}
}
Store the script
Once the formula looks right, store a reusable script in cluster state so update jobs can call it by ID. This version skips any field whose inputs are missing and adds two simple buckets for price and stock.
PUT /_scripts/products-enrichment-v1
{
"script": {
"lang": "painless",
"source": """
if (ctx._source.list_price != null &&
ctx._source.list_price > 0 &&
ctx._source.price != null) {
double discount =
(ctx._source.list_price - ctx._source.price) * 100.0 / ctx._source.list_price;
ctx._source.discount_pct = (int) Math.round(discount);
}
if (ctx._source.price != null &&
ctx._source.price > 0 &&
ctx._source.cost != null) {
double margin =
(ctx._source.price - ctx._source.cost) * 100.0 / ctx._source.price;
ctx._source.margin_pct = (int) Math.round(margin);
}
if (ctx._source.price != null) {
if (ctx._source.price < 25) {
ctx._source.price_band = "budget";
} else if (ctx._source.price < 100) {
ctx._source.price_band = "mid";
} else {
ctx._source.price_band = "premium";
}
}
if (ctx._source.inventory_count != null) {
if (ctx._source.inventory_count > 0) {
ctx._source.stock_state = "in_stock";
} else {
ctx._source.stock_state = "out_of_stock";
}
}
"""
}
}
Note that the math multiplies by 100.0 before dividing. _source returns values exactly as they were indexed, so a product stored with "price": 80, "list_price": 100 hands the script two integers. Integer division would round the discount down to zero. The dry run can’t catch this because its sample values are decimals.
Backfill
Run /_update_by_query asynchronously so the request returns right away instead of holding the connection open. The query only matches documents missing at least one derived field, so a rerun skips anything that’s already enriched.
POST /products/_update_by_query?conflicts=proceed&wait_for_completion=false&slices=auto
{
"query": {
"bool": {
"should": [
{ "bool": { "must_not": { "exists": { "field": "discount_pct" } } } },
{ "bool": { "must_not": { "exists": { "field": "margin_pct" } } } },
{ "bool": { "must_not": { "exists": { "field": "price_band" } } } },
{ "bool": { "must_not": { "exists": { "field": "stock_state" } } } }
],
"minimum_should_match": 1
}
},
"script": {
"id": "products-enrichment-v1"
}
}
The response is just a task id:
{
"task": "r1A2WoRbTwKZ516z6NEs5A:36619"
}
Monitor progress every 5 seconds until the task reports "completed": true.
watch -n 5 'curl -s "http://your-cluster:9200/_tasks/r1A2WoRbTwKZ516z6NEs5A:36619" | jq "{completed: .completed, updated: .task.status.updated, version_conflicts: .task.status.version_conflicts, failures: .response.failures}"'
Verify
Once the task is done, pull a few documents and check that the new fields are there.
GET /products/_search
{
"size": 3,
"_source": [
"name",
"price",
"discount_pct",
"margin_pct",
"price_band",
"stock_state"
],
"query": {
"match_all": {}
}
}
This only works because the change is additive. If you need to change the type or analyzer of a field that already exists, you’re back to reindexing.