Live Cross-Cloud Migrations With Aeolus and Snap

[Mo Morsi]

On 11/04/2011 08:36 AM, Richard W.M. Jones wrote:
> On Fri, Nov 04, 2011 at 09:59:39AM +0000, Richard W.M. Jones wrote:
>> On Thu, Nov 03, 2011 at 09:17:55PM -0400, Mo Morsi wrote:
>>> See the following step-by-step guide (complete w/ screenshots ) [1]
>>> that I threw together on how to use Aeolus and Snap [2] to migrate a
>>> running instance between two separate cloud providers with no
>>> downtime (demonstrated is EC2 / rackspace but this will work to/from
>>> RHEV-M or any other cloud provider supported by aeolus/deltacloud
>>> [3])
>>>
>>>    -Mo
>>>
>>> [1] http://mo.morsi.org/blog/node/347
>>>
>>> [2] https://github.com/movitto/snap
>>>
>>> [3] http://incubator.apache.org/deltacloud/drivers.html#h2
>> You mention in the blog posting that you can restore to another Linux
>> distro.  But would this really work?  How about if there were two
>> different Apache versions, each with a slightly different
>> configuration syntax?  Or more plausibly, two different PostgreSQL
>> versions (PG's on-disk format is not compatible across some version
>> changes).
> Some follow-up questions ..
>
> How large is the snap metadata, ie. the stuff that you copy between
> the machines?  How large would it be given, say, a typical
> database-backed webserver installation where you might have lots of
> static contents and some database tables?

One of the nice things that I added to Snap was the ability to ignore 
static content managed by the package management system. For example 
when taking a snapshot of the filesystem, only the files modified post 
installation and the files not tracked by the package system will be 
backed up and restored.

It should be simple enough to expand upon this concept, adding 
additional hooks to call out to to determine what exactly should be 
backed up and restore (hooks to be invoked during the backup / 
restoration process is already a feature on the project todo list / 
backlog).


>
> Is the metadata in an ad-hoc format and how hard would it be to turn
> it into a standard format (probably one that we would standardize
> ourselves)?  Can it be useful in other contexts -- eg.  could a system
> administrator look at the output in order to get a definitive list of
> the changes made to the machine?  Could it be useful for auditing?
> Could the format be diffed?
>

Right now the snapshot is a simple tarball containing the actual 
contents of the snapshot and the metadata in XML files. So for example 
there is a packages.xml file which contains the packages which have been 
recorded, services.xml containing the services and associated metadata, 
etc. We can use this as the basis of the standard, easily encapsulating 
any required information there.

   -Mo