Migrating from The Rackspace Cloud to Amazon EC2

We had a fair bit of data piled up at our virtual server on The Rackspace Cloud’s Cloud Sites system (previously known as Mosso). When we decided to move off Cloud Sites and on to Amazon EC2 moving that data quickly became extremely important. MySQL offers several methods of transporting data from the database and mirroring/replication seemed to be the ideal fit since we’d get our data synchronized in real time with the old database. Unfortunately, because of the way Cloud Sites are configured we can’t use mirroring at all (it’s already being used internally by Rackspace for scaling).

Since, we were using Cloud Sites we didn’t have SSH access to our box. Rackspace recommended we use mysqldump on another server and connect to our MySQL instance to grab the data directly over the internet. Two problems with that:

  1. Moving 15GB. of data on a single connection sequentially would take forever. (We were getting only ~200Kb/sec)
  2. The data would not be compressed, so we’d have to move the full 15GB.

Our solution:

  1. Setup a virtual Cloud Server on Rackspace Cloud with enough space for the data.
  2. Use mysqldump on this new server to connect to Cloud Sites and grab the data. Since they’re in the same server farm you get much much higher transfer rates.
  3. Compress using gzip and place in a web server document directory.
  4. On the Amazon server use aget to download the newly compressed gz file. We used 10 parts and got about 1800Kb/sec.
  5. Unzip and execute the SQL.

This reduced the amount of time needed to manageable proportions and therefore caused a minimal service outage.

Tags: , , , , , , , ,

This entry was posted on Wednesday, July 15th, 2009 at 9:56 am and is filed under Code, Interwebs. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

4 Comments

  1. Atanas

    November 22nd, 2009 at 4:12 pm

    Hi Viz,

    I am currently researching various hosting options and Amazon’s EC2 got my attention. As I don’t have any experience with cloud computing, I was wandering if you could share whether you’re happy with the service in general and if you’ve stumbled upon any pitfalls.

    Thank you very much!

    Kind regards,

    Atanas.

  2. Vishal Parpia

    November 23rd, 2009 at 9:49 am

    Hi Atanas,

    I am now using Amazon’s EC2 for three separate projects and so far it has been fantastic. Their service offering is the best in the business and they are consistently innovating and releasing new products before their competitors. I would recommend them over any other cloud offering at this point in time.

    Cheers,
    Vishal.

  3. KD

    February 2nd, 2011 at 4:54 pm

    Hi Vishal,

    I’m developing SaaS application (.net), which i need to host on cloud server.

    Do i need to make some changes on coding level also to host this?

    And what if i have to migrate this from EC2 to other vendor? do i need to change again to make changes in my application?

    Thanks in advance,
    KD

  4. Vishal Parpia

    February 2nd, 2011 at 5:07 pm

    Hi KD,

    You shouldn’t need to make any changes to the application since cloud hosting will allow you to use virtually any software you run on traditional equipment. However, if you really want to leverage the cloud, you probably need to change specific parts of your app to better use cloud infrastructure for storage, queueing, messaging, processing and data.

    If you’re interested in portability I suggest reading up on the Simple Cloud API. Apps built using it should be totally portable between supported cloud environments.

    Cheers,
    Vishal.

Leave a Reply

«
»