ObjectRocket was built on the core premise of enabling simple and reliable scalability for MongoDB.
At its core, ObjectRocket utilizes MongoDB’s native scaling architecture called “sharding”. But it’s a cloud world, and in the cloud there are interesting things we can do when we have lots of compute at our disposal. Customers can start off small, and grow automatically. When users sign up, they instantly get a fully provisioned sharded cluster in whatever plan size they choose. All of the complicated sharding components are automatically setup, and there is zero configuration for a customer to complete. So scaling on ObjectRocket is seamless.
Here is how we do it.
At the core of the system is our automation agent called RocketScale™. This component is a back end job written in python/APScheduler. Its job in life is to detect when a customer needs to grow, and then handles the process of adding a shard to the cluster. Each customer and instance pair gets an isolated APScheduler job queue for RocketScale to poll and analyze.
RocketScale works using some basic metadata instructions as it polls, one configurable by the customer. The setting named autoadd_shard_threshold is customer configurable, and instructs RocketScale when to add a new shard. Rocketscale polls our metadata for this threshold, then compares it to the actual disk usage of each instance. If the disk space used by the customer exceeds the threshold, a new shard is added. It’s just a simple bit of metadata as:
{autoadd_shard_threshold: 80}This setting is a configurable in the ObjectRocket UI under instances/{instance_name}. By default new instances are set to a default 80% threshold. We recommend sharding early and often to avoid any possible issues with the MongoDB balancer, and 80% is a nice compromise of space usage and ease of scaling. Setting RocketScale to 0 or leaving it empty disables this process.
It should be noted that MongoDB won’t be able to use multiple shards unless there are shard keys defined on the major collections. ObjectRocket customers must create these keys using our UI in order to instruct MongoDB how to logically split the data. Selecting shard keys can be time consuming and tricky, and we have some tools that are coming soon to help in this process. For today, users simply specify the shard key(s) they would like using the UI at https://app.objectrocket.com.
One interesting aspect of this process is that RocketScale always has a shard at hand to instantly add because of the way we provision shards. We pre-provision shards ahead of time in various plan sizes. Adding a shard is simply grabbing the next shard in line and assigning it to a customer. This is all done in the background and is very quick to complete. ObjectRocket manages the number of candidate shards in the queue and ensuring that we have the proper capacity to add for customers as needed.
RocketScale has a couple sanity checks built into it to ensure it’s doing the correct thing. For instance:
- Checks the version of MongoDB to ensure it matches
- Checks to ensure we haven’t added many shards in the last few minutes (runaway sharding!)
- Sanity checks the instance is OK and usable
When a shard is added, the ObjectRocket notification system sends a message regarding the change. The message generally has the format of:
Hi ObjectRocket User,
Your ObjectRocket instance named ‘foo’ is getting low on storage space. RocketScale (our automatic scaling engine) will be adding a shard soon to horizontally scale your instance, and your monthly bill will increase accordingly.
The following collections don’t have any shard keys defined, and won’t make use of this additional shard:
- foo.mybigcollectionnokeys
- foo.mysmallcollectionnokeys
Even if additional shards are added, if a collection without shard keys uses all of the remaining storage space on your instance, you could experience downtime. ObjectRocket strongly recommends you define shard keys for all of your collections.
You can add shard keys to your collections by following these steps:
- Browse to https://app.objectrocket.com/instances/xxxxxxxx.
- Click on the database name containing the collection without shard keys.
- Click on the collection name that does not have a shard key.
- Click on the ‘Shard Key’ tab.
- Click on ‘Add Shard Key’.
An excellent guide to selecting shard keys can be found at http://docs.mongodb.org/manual/tutorial/choose-a-shard-key/.
To silence future notifications for this instance, click the link below:
https://app.objectrocket.com/73h33bbbdd337754
If you have questions or issues, you can always ping us at support@objectrocket.com
-The ObjectRocket Team
In addition, ObjectRocket support also sees a message that the customer grew so we are in the loop in case the customer has questions.
Once a new shard is added, the MongoDB balancer is kicked off to balance the chunk data onto the new shard. In some cases the customer may have selected a balancer window and thus the balancer will not fire up until the specified window starts. Managing these settings is easy with our recent Balancer Ease of Use Enhancements.
In addition to RocketScale there are user controllable mechanisms like the UI and the API where a shard can be added, but we will save those items for next time.