On 06/29/2011 05:11 AM, Chris Lalancette wrote:
On 06/28/11 - 03:12:02PM, Steven Dake wrote:
I had originally thought deltacloud might be involved here in some way to consistently load instance data into the instance (and then provide a consistent way of accessing it inside the vm that doesn't involve writing 10 different access mechanisms).
I don't know enough about deltacloud, but if the insertion functionality is currently a gap in any virtual machine manager (such as paravirt Xen) or RHEVH, the entire Aeolus solution as well as cloud-ha becomes non-functional on those VMMs.
The question for deltacloud evaluation of this functionality then boils down to:
- is this functionality available on every deltacloud cloud provider
implementation
Not readily or consistently. For instance, rhev-m has "hook" functionality, but we haven't yet figured out what it means to use the hooks to provide this functionality for rhev-m.
VMWare has some form of data injection available, as long as you install their guest agent, enabling certain api functionality that deltacloud could talk to.
It's significantly different for each provider, that it takes research each time a new provider is taken on.
- is the functionality inside the vm consistent (does a user only call
a "read_my_id" function or read a file inside the vm) - if library access what is the new dependency inside the vm?
No. In fact, for the case of EC2, this particularly breaks down. Because EC2 stores the "user data" in a place only accessible over the network, the VM needs to ask a known ec2 location for the user data via an HTTP GET.
Since dbus is probably already firmly set in stone by the time networking comes up, it's likely too late to rely on ec2's user data to store to this information and have it impact dbus after retrieval.
Just to be clear here, I don't know that this is the case. I haven't personally looked into dbus, and how to configure the UUID on-the-fly. This definitely bears investigation, as I feel that getting dbus to properly expose this information is our best path forward.
looks like upstream is taking a crack at moving this into systemd rather then dbus........
http://lists.freedesktop.org/archives/dbus/2011-March/014187.html
dbus does not have a set system id API.
if we add a dbus set system id API and call it, say at the end of the startup sequence, some apps will be aware of the old id and others will be aware of the new id. I expect this would break things badly, but I am not a dbus expert so can't comment here.
Perhaps we should let dbus do its own thing and just create a new file. Then we have two files
/etc/machine_id (worthless in a cloud environment except for dbus operation) /etc/vm_machine_id (written by audrey or a dbus/systemd api) which can be set after the network is operational. Then the machine id logic can be put in one place (audrey) rather then a bunch of separate apps.
I still have a bit of confusion how current VMs get their unique id inside a vm in the current model to communicate with Audrey's config server.
more fuel on the fire: http://www.freedesktop.org/wiki/Software/systemd/hostnamed
Lets go back to basics, the requirements:
R1: vm needs to have some mechanism of retrieving a globally unique UUID across all VMs in a cloud enviornment R2. whatever launches the vm needs to know how to map a UUID to an image R3. To obtain the vm UUID inside a vm, the network must be operational (is this true in atleast some cloud environments? If so, then it is a requirement).
Yes, this is all about right. Requirement 3 is the difficult one.
But maybe we are looking at this the wrong way. If we start from the premise that dbus/systemd can generate it's own UUID at boot time (suitably configured), then maybe we should think about using *that* as the UUID for the machine. What this would then require is a mapping between the UUID
This would be a great option if anyone has any ideas how to generate that mapping. The issue I get stuck on when thinking through this model is how does the dbus id get out of the VM to make the mapping. The problem is easy if only one vm starts at a time, but when multiple vms start, I don't see a reasonable way to correlate the info.
This creates a different set of requirements: 1) dbus/systed must generate a fresh machine id on each boot 2) a translation service must exist to convert dbus ids to cloud ids.
1 seems challenging but solvable if we plan to run on older distros (since these distros will have older versions of dbus/systemd). The id currently isn't generated freshly on each boot.
One hacky workaround to 1 is the addition of an init script that deletes the machine id file if it exists as the first thing that runs in the system for older distros, and newer distros systemd could be configured in some way to generate a fresh uuid via changes to their code base.
2 is more challenging - I don't have many ideas here. Audrey could potentially determine the runtime vm UUID and send it to the config server (which then could act as a translation service). This model turns the audrey config service into a T component which from a design standpoint is difficult to manage..
the cloud machinery has (i.e. the one that was generated prior to launch) and the UUID that dbus generated. It's a little ugly, but it prevents us from having problems when we restart dbus/systemd on an already running system.
Again, I'm just throwing out ideas here. I still think we need to look closer at dbus/systemd and see if we can possibly send some patches to those to support changing the UUID in a cleaner manner. Then we can determine what the best path forward is.
This leads to the same legacy problem - older distros wont have those patch sets and audrey as well as pacemaker-cloud wont be operational on those legacy distros.