On 07/22/2011 05:13 PM, Hugh Brock wrote:
On Fri, Jul 22, 2011 at 03:30:11PM -0400, Greg Blomquist wrote:
On 07/22/2011 02:34 PM, Steven Dake wrote:
On 07/22/2011 10:22 AM, Greg Blomquist wrote:
On 07/22/2011 01:16 PM, Mark McLoughlin wrote:
Hi Greg,
On Fri, 2011-07-22 at 13:05 -0400, Greg Blomquist wrote:
If the deltacloud ID is the UUID presented to deltacloud by the cloud provider for the launched instance, then the only reason not to use the deltacloud ID for the config server and audrey startup script is that it requires the guest to launch and report this ID before the configurations can be handed to the config server.
Yeah, it's not ideal.
That's not terrible, but just makes the launch sequence more serial instead of parallelizing the launch and seeding the configs to the config server. If it's necessary to reduce the number of IDs floating around, we can do this.
Well, AFAICT, the only ID that the cloud HA stuff will know about from conductor or deltacloud is the deltacloud instance ID. So, that's what we want the guest agent to use when reporting back.
Ah, good point.
If you choose to add another ID for config server purposes, that would mean the cloud HA stuff would need to find that out. And I think that's a step too far in terms of intertwining the architecture. The cloud HA stuff should work with standalone deltacloud, without conductor or config server IMHO.
Yep, I'm sold.
The first cut I'm working on will generate IDs for Audrey. But, we can circle back on this and revamp to use the deltacloud ID. The two changes this implies on the Audrey side are
- the audrey startup script should use the UUID API to acquire the
Immutable UUID (assuming this is what maps to Deltacloud ID) 2) the conductor launch sequence will be modified to wait to push configs to config server until the instance is launched and can report back the deltacloud ID
Sound about right?
This proposal sounds good but leaves out a bit more detail further down the bootstrap order.
We also need this ID loaded into matahari as well as matahari qpidd authentication data. This is where the discussion around the machine-id came from. If it is a different machine-external API coming out of matahari but available as soon as matahari is started, this WFM.
This could likely be done by audrey with following start order:
audrey client starts audrey sets matahari authentication information on the filesystem audrey sets UUID information on the filesystem audrey starts matahari
There's still a bootstrapping issue here. Well, maybe...possibly depending on who cares.
The problem is that the matahari authentication information would likely come from the instance configuration delivered by the config server. The audrey client would not get configuration data from the config server until the config server learns about the instance from conductor. Conductor wouldn't alert the config server about the instance until it learns the deltacloud ID from the deltacloud driver.
So, it looks like this (I think--assuming we're talking about a cloud engine environment):
- conductor tells deltacloud to launch guest
- guest boots in cloud provider
- audrey client starts
oops, 3a. audrey client reads user data to find out how to contact config server
- audrey client contacts config server, but finds nothing there...yet
- deltacloud driver tells conductor deltacloud ID for guest
- conductor tells config server about instance configs
- audrey client contacts config server, gets configs
- audrey client sets matahari auth information on filesystem
- audrey sets UUID information on the filesystem
- audrey starts matahari (finally)
- matahari can auth against broker
Hmm...
Given that we now have a working userdata mechanism for EC2 and VMWare, and a soon-to-be-working one for RHEV-M... could we not leverage that to compress steps 3-7? Meaning, if we dump the instance configs into userdata, the audrey client could read them from there, then go straight to contacting config server to get its matahari auth information?
Tell me if i'm out of my mind here.
Well, I'm no professional... :)
The trick with user data mechanisms (especially in the public cloud) is that it's limited in size per guest. Of course, this could be part of the selection criteria for deciding on a provider account before launching (does your user data exceed the allowed size of user data in the provider?). But, we'd have to have a way to reliably capture that size (data entry from the user seems a little too brittle here).
Maybe there's a way to split the difference (some config data is required to be passed via "user data" b/c it's needed early, while other data can be acquired after networking, etc. is up). We'd still have to have a way to reliably capture the size of the provider's user data. And, this also introduces the need for a way to determine what user data requires early binding, and what can go the default deferred binding route.
The underlying problem remains that, for any arbitrary configurable guest, we don't know how much user data will be required until we get ready to launch. All this means is that it's possible to blow all of the user data limits out of the water, leaving no way to launch the guest, b/c we place no limits on this data.
--Hugh