I am pretty new to ES. I have been trying to search for a db migration tool for long and I could not find one. I am wondering if anyone could help to point me to the right direction.
I would be using Elasticsearch as a primary datastore in my project. I would like to version all mapping and configuration changes / data import / data upgrades scripts which I run as I develop new modules in my project.
In the past I used database versioning tools like Flyway or Liquibase.
Are there any frameworks / scripts or methods I could use with ES to achieve something similar ?
Does anyone have any experience doing this by hand using scripts and run migration scripts at least upgrade scripts.
Thanks in advance!
From this point of view/need, ES have a huge limitations:
What does that mean in context of your question? You, basically, can’t have classic migration tools for ES. And here’s what can make your work with ES easier:
use strict mapping (
"dynamic": "strict" and/or
index.mapper.dynamic: false, take a look at mapping docs). This will protect your indexes/types from
being accidentally dynamically mapped with wrong type
get explicit error in case when you miss some error in data-mapping relation
you can fetch actual ES mapping and compare it with your data models. If your PL have high enough level library for ES, this should be pretty easy
you can leverage index aliases for migrations
So, a little bit of experience. For me, currently reasonable flow is this:
news) which points to actual index (i.e.
Every time code being deployed, you
All of this actually means that you’re good to go with mappping/data you have, just work with data as always.
bulkrequests for fast reindexing
This is production-tested solution. Caveats around such approach:
To sum up this: