On 06/29/2011 08:11 AM, Chris Lalancette wrote:
On 06/28/11 - 03:12:02PM, Steven Dake wrote:
I had originally thought deltacloud might be involved here in some way to consistently load instance data into the instance (and then provide a consistent way of accessing it inside the vm that doesn't involve writing 10 different access mechanisms).
I don't know enough about deltacloud, but if the insertion functionality is currently a gap in any virtual machine manager (such as paravirt Xen) or RHEVH, the entire Aeolus solution as well as cloud-ha becomes non-functional on those VMMs.
The question for deltacloud evaluation of this functionality then boils down to:
- is this functionality available on every deltacloud cloud provider
implementation
Not readily or consistently. For instance, rhev-m has "hook" functionality, but we haven't yet figured out what it means to use the hooks to provide this functionality for rhev-m.
VMWare has some form of data injection available, as long as you install their guest agent, enabling certain api functionality that deltacloud could talk to.
It's significantly different for each provider, that it takes research each time a new provider is taken on.
- is the functionality inside the vm consistent (does a user only call
a "read_my_id" function or read a file inside the vm) - if library access what is the new dependency inside the vm?
No. In fact, for the case of EC2, this particularly breaks down. Because EC2 stores the "user data" in a place only accessible over the network, the VM needs to ask a known ec2 location for the user data via an HTTP GET.
Since dbus is probably already firmly set in stone by the time networking comes up, it's likely too late to rely on ec2's user data to store to this information and have it impact dbus after retrieval.
Just to be clear here, I don't know that this is the case. I haven't personally looked into dbus, and how to configure the UUID on-the-fly. This definitely bears investigation, as I feel that getting dbus to properly expose this information is our best path forward.
looks like upstream is taking a crack at moving this into systemd rather then dbus........
http://lists.freedesktop.org/archives/dbus/2011-March/014187.html
dbus does not have a set system id API.
if we add a dbus set system id API and call it, say at the end of the startup sequence, some apps will be aware of the old id and others will be aware of the new id. I expect this would break things badly, but I am not a dbus expert so can't comment here.
Perhaps we should let dbus do its own thing and just create a new file. Then we have two files
/etc/machine_id (worthless in a cloud environment except for dbus operation) /etc/vm_machine_id (written by audrey or a dbus/systemd api) which can be set after the network is operational. Then the machine id logic can be put in one place (audrey) rather then a bunch of separate apps.
I still have a bit of confusion how current VMs get their unique id inside a vm in the current model to communicate with Audrey's config server.
more fuel on the fire: http://www.freedesktop.org/wiki/Software/systemd/hostnamed
Lets go back to basics, the requirements:
R1: vm needs to have some mechanism of retrieving a globally unique UUID across all VMs in a cloud enviornment R2. whatever launches the vm needs to know how to map a UUID to an image R3. To obtain the vm UUID inside a vm, the network must be operational (is this true in atleast some cloud environments? If so, then it is a requirement).
Yes, this is all about right. Requirement 3 is the difficult one.
But maybe we are looking at this the wrong way. If we start from the premise that dbus/systemd can generate it's own UUID at boot time (suitably configured), then maybe we should think about using *that* as the UUID for the machine. What this would then require is a mapping between the UUID the cloud machinery has (i.e. the one that was generated prior to launch) and the UUID that dbus generated. It's a little ugly, but it prevents us from having problems when we restart dbus/systemd on an already running system.
Again, I'm just throwing out ideas here. I still think we need to look closer at dbus/systemd and see if we can possibly send some patches to those to support changing the UUID in a cleaner manner. Then we can determine what the best path forward is.
This makes me wonder if copy-on-write can be used here to seed the uuid we want. Knowing nothing about how to leverage copy-on-write and nothing about dbus puts a pretty big cloud over this for me. But, thought I'd throw it out there...
There are a couple guys in the office here that work with dbus, I'll see what they have to say.