I’m excited to announce that our ObjectRocket Elasticsearch product now includes automated index curation, as well as the general availability of Elasticsearch version 2.3!
At ObjectRocket, our mission is to make it easier for developers to focus on building great apps instead of worrying about the complexities of data management. Adding the automated index curation is one more feature to empower your team to focus on the product and not the infrastructure.
What does it do for me?
The ObjectRocket Elasticsearch curator feature allows you to delete or close an index based on the age of the index, the index naming convention, or a combination of both. All without the need for an additional server running an easily forgotten and difficult to monitor cron tab.
How easy is it to use the automated curator?
Very easy, especially if you’re using the default index naming pattern from Logstash which appends the date on each index. This naming convention makes it incredibly easy to keep track of your indexes and manage them over time. In the Logstash config it would look like the following example:
index => "interesting-index-name-%{+YYYY.MM.dd}"
Set it and forget it
Let’s get started with it! To use the curator go to your ObjectRocket Elasticsearch details page and scroll down to see the Curator section and click on the ‘Add Task’ button.
After clicking the ‘Add Task’ button you will be presented with the following form. This form is all you need to complete in order to create your automated index curation task.
Here you can see where I named the task, selected the Task Type to Delete, specified the date range, and then selected the Prefix filter and entered the non-date specific part of the index name.
As noted on the form, the curation tasks will ignore all indexes related to your Kibana dashboards and Marvel. So rest easy, your dashboards are safe.
Click the ‘Add Task’ button to finish setting up your automated curation task. This will return you to the details page where you can see the task and delete it when you wish.
How often do the automated curation tasks run?
The curator runs at least once every five minutes, with the average tempo being every two minutes. This allows for a level granularity that should work for even the most aggressive of data flows.
That’s great, but what about the availability of Elasticsearch 2.3?
Elasticsearch 2.3 has matured with the recent updates, knocking out a number of bugs and enhancing performance to the level where we are ready to offer it in our IAD, DFW, ORD and LON data centers.
The hottest new features include:
- Reindex API – Allows you to hydrate a new index with the data from an existing index. No more external scripts!
- Task Manager API – Get better clarity on the background tasks running across the cluster.
- Update-by-Query API – Easily update the documents that match your query!
New accounts on ObjectRocket get the first 30 days FREE! How awesome is that!?
Pretty cool!
And if you are as excited as we are, give us a tweet and let your colleagues know that ObjectRocket Elasticsearch now includes automated index curation!