Introducing AutoKey; automated shard key definition

Scaling on ObjectRocket gets easier now that MongoDB shard key creation can be automated with AutoKey.

Part of our philosophy at ObjectRocket is to ensure customers have a seamless and fantastic MongoDB experience. Customers focus on the application, we take care of the database. The new AutoKey feature furthers our goal of a massively automated database as a service offering.Since our inception, ObjectRocket has had Rocketscale – an automated process that adds shards to customer instances as they grow. When a customer starts to run out of space, Rocketscale adds a shard, and the balancer starts moving chunks to the new shard. Business continues, performance stays fast, all is good. That is wonderful, however, each collection still needs to have a shard key defined. A shard key is the key in which data is split among the shards participating in a MongoDB cluster. Defining a shard key requires an understanding of the application access patterns. Sometimes it makes sense to really engineer a great key, but other times, a simple and generic key can be used. Not every collection is going to be huge, used frequently, or have specific access patterns requiring a highly engineered key.With the release of MongoDB 2.4 hashed based shard keys became available. While hashed based shard keys are not perfect for every scenario, they are a great general-purpose shard key for a large set of use cases.

Enter AutoKey

AutoKey automates the process of adding a hashed based index and shard key on collections in an ObjectRocket instance. Once you turn on AutoKey, shard keys will automatically be defined where they don’t already exist (for collections > 256MB). The AutoKey daemon fires up periodically and checks for keys to create. Once it finds missing keys on collections it goes ahead and creates the key, the indexes, and notifies the customer through our notification interface. AutoKey operates on an entire DB, so the user can set manually define shard keys, and let AutoKey pick up the rest.The index AutoKey creates has the following simple definition:

{
  "v" : 1,
  "key" : {
    "_id" : "hashed"
  },
  "ns" : "test_hashkey.testhash",
  "background" : true,
  "name" : "_id_hashed"
}

The shard key definition is:

{
  "_id" : "my collection",
  "lastmod" : ISODate("1970-01-17T03:12:59.552Z"),
  "dropped" : false,
  "key" : {
    "_id" : "hashed"
  },
  "unique" : false,
  "lastmodEpoch" : ObjectId("5316709f5b769f128aadc1fc")
}

Getting started with AutoKey

Using AutoKey is easy. Just navigate to the instance settings pane, select an instance and click on the settings panel. Click on AutoKey to toggle it on or off. It’s that simple. If you would like to specifically define some shard keys in various collections and leave others to AutoKey that’s fine too. Simply define your shard keys as you normally would and AutoKey skips them when defining new keys.There is further reading in our docs, and as always, if you have any questions or concerns simply hit support for help.