Experimental pluggable storage engines in MongoDB

At MongoDB World last month MongoDB founder and CTO Eliot Horowitz announced support for pluggable storage engines scheduled for the 2.8 release. This is exciting stuff as it means mongo users will now be able to choose a storage engine that best suits their workload and with the API planned to have full support of All MongoDB features, while not having to give up any of the current functionality that they enjoy. Not only that, but nodes in the same replica set will be able to use different storage engines, enabling all sorts of interesting configurations for varying needs.

The great thing about MongoDB being fully open source is that we don’t have to wait until 2.8 is actually released to play around with these very experimental features. The entirety of the MongoDB source code can be cloned from GitHub and compiled to include any experimental features currently being worked on.

In the example below I’ll show you how to build mongo with the rocksdb example storage engine presented at MongoDB world.

Starting from a freshly installed CentOS 6.5 cloud instance, we’ll grab the basic dependancies:

$ yum groupinstall 'Development Tools'
$ yum install git glibc-devel scons`

Next will get the MongoDB source code from GitHub:

$ git clone https://github.com/mongodb/mongo.git

Now all that’s left is to compile the source with RocksDB support enabled:

$ scons --rocksdb=ROCKSDB mongo mongod

Or speed it up by using the -j option to specify the number of parallel jobs to use, if you plan to dedicate the system your compiling on for the time being a good indicator is the number of cores in your machine +1, mine looked like:

$ scons -j 17 --rocksdb=ROCKSDB mongo mongod

It’s worth noting that pluggable storage engine support and the RocksDB engine are completely experimental at this point so there’s a good chance you’ll encounter errors and be unable to compile from master, that’s to be expected at this stage. If you’d like to keep an eye on how things are progressing the MongoDB dev mailing list is a good place to start.

Once the compile has finished you’ll want to start up a mongod process using the new –storageEngine parameter:

$ ./mongod --storageEngine rocksExperiment

And finally you can test everything by connecting and inserting a simple document, then using db.stats(). You should see RocksDB statistics piped back to you if everything has gone as planned.

As you can see it’s fairly simple to get up and running with experimental features enabled. I’m very excited to see the pluggable storage engine code progress and see more new engines announced as we get closer to the 2.8 release.