EMC Atmos – Building Clouds in the Lab

I have been very excited lately at all the momentum in the marketplace on all things Cloud. There are so many companies offering public cloud services, private cloud software and federations between the private and public clouds. There are even “cloud brokers” which allow you to leverage them as a one stop shop to move objects between the clouds……..it’s great stuff!

One interesting piece of software is EMC’s Atmos. Atmos is EMC’s cloud platform. It can be purchased as a turn-key set of hardware and software or you can just purchase the software and run it in a virtual machine and use your existing storage to backend it. EMC use to offer Atmos as an online service, but has since pulled the plug on this. They continue to support current customers but are taking on no new customers. Other companies such as AT&T’s Synaptic service leverage EMC Atmos to offer public based cloud services based on the EMC Atmos platform.

Cloud is not your typical storage platform. It’s not NFS, CIFS, FTP. Cloud is object based storage. Atmos separates the user data and metadata. By using metadata, you can group objects logically. Then policies are applied to objects, based on the metadata to determine replications levels and replica placement. You may choose for example to have 5 copies of a particular type of data, all in separate geographical data centers. Or you may choose to only have two copies of lesser important data.

Atmos uses what’s called GeoProtect to protect the data. This is sort of like RAID in that it spreads segments of the data across multiple locations so that if a location is lost, the data can be rebuilt.

In the lab this weekend, I went to town and built four Atmos servers. Two Resource Management Segments (RMG’s) each with a single Installation Segment (IS). Version is 1.3. Special Thanks to Chad Sakac for making the software available and packaging it for easy installation.  I did have a few issues. The original Atmos packages that Chad built were designed or ESX not ESXi.  So the scripts that would normally run in the service console could not be used.  Good news is, it’s a well written easy to read bash script and you simply have to make sure you follow it and setup everything like you’re suppose to in ESXi.  This means creating two private virtual switches, one for each RMG, and a public switch which all nodes connect into.

I could not untar the very large tar files with 7-zip.  It would give me errors about skipping segment ‘S’.  I had to move them over to my CentOS 5.5 VM, untar them there, then SCP them over to my datastore.  I love Veeam FastSCP by the way :).  Here is what Atmos looks like built inside of my ESXi Host:

 

 

 

 

 

 

I also configured the free iPhone App Atmos iSafe application.  This allows you to move stuff to and from the cloud similar to Dropbox.  Dropbox is a cloud service that is back-ended up Amazon S3 which is an object based cloud service similar to Atmos.  here are some pics of the iPhone app working with my Atmos cloud in my lab:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Next up I hope to upgrade this Atmos from 1.3 to 1.4 using EMC’s great procedure generator.  Then after that I will federate my private cloud to a public Atmos cloud and run some custom policies…..great stuff!

This entry was posted in Atmos, EMC and tagged , , . Bookmark the permalink.

5 Responses to EMC Atmos – Building Clouds in the Lab

  1. Jerome Lecat says:

    Hi Brian,
    Thank you for your post. I am wondering if you testeted ATMOS for a large number of objects. I have read blogposts on ATMO stating that performance slows down dramatically, and even some errors appear in multi-datacenter situation, when used for more than 100 million objects. DO your test reflect this ?
    Best Regards,
    Jérôme

    • brian says:

      Jerome,

      I would be interested in any data you have on this as I am not aware. The platform is built to scale to billions of objects, so it should not be bogging down with a few million. I have not tested it to this level, nor do I have a platform that I can do that with. My plans to work with Atmos are more about integration and functionality. For example I am currently working to federate my internal Atmos private cloud to Atmos Online.

  2. Jerome Lecat says:

    Hi Brian,

    Take a look at Prediction 1 on this post http://www.networkcomputing.com/cloud-storage/forecast-shakeups-looming-in-the-cloud.php by Tom Trainer.

    I am really interested to know if you have heard of actual test or deployment which go in the billions of objects ?

    Best Regards, Jérôme

  3. Jerome Lecat says:

    Agreed with you it is difficult to test… yet there are more and more real use cases which require the storage of many billions of object. This is the case for every of our webmail provider client. The numbers are simple : when you have 10 million users x 10 000 mail per user, that’s 100 billions objects. The question is how can clients test it before they deploy ?

Leave a Reply