On 06/03/2014 09:25 AM, Dan Callaghan wrote:
Excerpts from Jaroslav Kortus's message of 2014-06-03 05:29:59
+1000:
> Nick, would it be possible to support some more general API to these
> "cloudy" things? So we could connect beaker to openstack, or ec2,
> or... anything else :). Deltacloud for example, if we don't have
> anything better (I'm not tracking it that much).
The initial support is targetting OpenStack because there are already
supported OpenStack deployments running within the company, which we can
use for development without worrying about extra cloud abstraction
layers.
OpenStack's APIs are generally EC2-compatible so it may make sense in
future to swap them out for an EC2-compatible abstraction layer.
There's also the fact that getting dynamic provisioning working on *one*
kind of cloud system is already a hard problem - and with OpenStack we
get a solution that can work with both public and private clouds
immediately. Once we have all those details thrashed out based on
OpenStack, adding EC2 support as well may be feasible, but testing
becomes significantly more difficult (since it can't be done for free,
unless you rely on a "private EC2 cloud" solution like Eucalyptus for
testing, which then brings its own compatibility and setup challenges).
We also have some significant "keep the compute near the data" concerns
in Beaker's provisioning model. Since we're not currently in a position
to move the relevant data to the cloud, that means placing the compute
resources near the data instead. We already have suitable OpenStack
resources available internally, so that's just a matter of configuring
things appropriately, whereas for EC2 integration it would represent an
entirely unsolved problem.
Personally I expect that longer-term Beaker will start to rely more
on
OpenStack-specific APIs. Unlike EC2, OpenStack is both open source *and*
Red Hat has a huge investment in it already, so if there is a feature we
want for Beaker (for example, streaming serial console logs and/or
read-write character device access for serial consoles) there is
actually a decent chance of it landing in OpenStack.
Right - when we eventually do image based provisioning support, it will
almost certainly be based on OpenStack Glance, rather than inventing our
own image library system. Similarly, we're currently in the very early
days of looking at our options for cleaning up the whole distro
library/sync system (preferably being able to get Beaker out of the data
replication business entirely), and OpenStack Swift is one of the
candidates for a distributed object store that we could use to solve
that problem rather than continuing to maintain our own relatively ad
hoc solution. Longer term, we may even end up outsourcing the whole
identity management problem to Keystone rather than continuing to add
identity management features to Beaker (that would be a big project
though, and one that is so far down our priority list I can barely even
see it...).
OpenStack Ironic also holds the promise of eventually being able to
offload even some bare metal provisioning requests, rather than only
being able to offload VM requests (I actually wouldn't be surprised if
we end up borrowing some of Ironic's provisioning code for our own bare
metal image based provisioning support once we get that far).
OpenStack mostly gets pitched as an all-in-one
Infrastructure-as-a-Service system (since that's the easiest aspect to
sell to executives), but it's this ability to mix and match the various
backend components for local and distributed use cases that really makes
it shine from a developer's perspective. In Beaker's case, that means
that there are things we're currently doing because we had to (there
were no open source alternatives at the time), which we can now reassess
and say "Do we need to keep working on this? Or can we start deleting
some of our custom code by relying on OpenStack components to handle it
instead?".
Cheers,
Nick.
--
Nick Coghlan
Red Hat Hosted & Shared Services
Software Engineering & Development, Brisbane
Testing Solutions Team Lead