amazon

We had a fair bit of data piled up at our virtual server on The Rackspace Cloud’s Cloud Sites system (previously known as Mosso). When we decided to move off Cloud Sites and on to Amazon EC2 moving that data quickly became extremely important. MySQL offers several methods of transporting data from the database and mirroring/replication seemed to be the ideal fit since we’d get our data synchronized in real time with the old database. Unfortunately, because of the way Cloud Sites are configured we can’t use mirroring at all (it’s already being used internally by Rackspace for scaling).

Since, we were using Cloud Sites we didn’t have SSH access to our box. Rackspace recommended we use mysqldump on another server and connect to our MySQL instance to grab the data directly over the internet. Two problems with that:

1. Moving 15GB. of data on a single connection sequentially would take forever. (We were getting only ~200Kb/sec)
2. The data would not be compressed, so we’d have to move the full 15GB.

Our solution:

1. Setup a virtual Cloud Server on Rackspace Cloud with enough space for the data.
2. Use mysqldump on this new server to connect to Cloud Sites and grab the data. Since they’re in the same server farm you get much much higher transfer rates.
3. Compress using gzip and place in a web server document directory.
4. On the Amazon server use aget to download the newly compressed gz file. We used 10 parts and got about 1800Kb/sec.
5. Unzip and execute the SQL.

This reduced the amount of time needed to manageable proportions and therefore caused a minimal service outage.

This happened about 2 weeks ago. I was on a Skype conference call with Rackspace and Mosso trying to get a really ridiculous level of redundancy setup for a client. That is not the point of the story.

The point is Rackspace put us on hold. When Rackspace puts you on hold you hear music piped from a local radio station in Texas. They were playing a song that sounded good to me, so I switched Skype’s output to speaker and used Shazam on my iPhone to figure out what song it was. Shazam, for the uninitiated, records the audio it hears on the iPhone’s microphone and uploads it to a server that figures out what song it is. Shazam has not failed me yet. The result showed up in about a second and I searched for the song on Amazon MP3, bought and downloaded it. All this happened before the support guy took us off hold.

If you had told me this was possible when I was 15 I would have thought you were on drugs.

The song was Leon Russell’s – Roll Away the Stone.