On 01/06/2012 12:48 AM, Steven Lawrance wrote:
Excerpts from Yaniv Kaul's message of Tue Dec 20 20:58:55 +1000 2011:
On 12/20/2011 10:22 AM, Steven Lawrance wrote:
Knowing whether we can provision that VM immediately is important because otherwise we can move onto the search for a "real" system that matches the requirements, i.e. we wouldn't want a job held up because it *could* be run via RHEV while there are idle baremetal boxes sitting in the lab. But at the same time, we don't want to poll RHEVM continually for all jobs sitting in the queue.
The only real way to know if a guest can run is actually execute it.
Could RHEVM notify Beaker of changes in available resources (such as when a guest is destroyed or created)? Maybe Beaker itself could track RHEV's available resources and just poll for changes at the start of each loop over the queued recipes, rather than repeatedly asking RHEVM whether it could provision each one?
But I think we are optimizing for the worst case scenario, where we ran out of resources. I expect us to have enough resources to handle peak loads. And btw, we are talking about tens to hundreds of concurrent instances, right?
I don't think it's safe to make that assumption (particularly not for all Beaker instances). In fact, we can probably assume the opposite...
I highly doubt that, but we can approach the problem from a different angle: say we have a RHEV instance which can run 100 VMs, total of 100GB RAM and 100vCPUs. Just don't let Beaker provision more than X percent of that total number.
(BTW, next version should have a quota feature - http://www.ovirt.org/wiki/Features/DetailedQuota ).
So if we can't distinguish not being able to provision a particular recipe via RHEV "right now" as opposed to "never (with current resources)", then we at least need a cheap way to determine the former in order to avoid a bottleneck in the scheduler.
'Never' is easy, in the sense that you cannot run a VM with more vCPUs than available pCPUs on the hosts, nor one with more memory than any host has. I'm not aware of other 'never' scenarios.
That is, when a job is submitted, we want to determine whether it _can_ be satisfied by RHEV without then restricting it to RHEV and preventing it from going to baremetal hardware in the lab.
Perhaps it is enough for Beaker to retry executing virtualisable recipes when it knows the situation has changed (e.g. another recipe completed thereby freeing resources, or if it can be informed of events like new virt hosts being added). But if tens or hundreds of virtualisable recipes are sitting in the queue, that could still take quite a long time.
All events can be registered to and sent via an email notification.
Is any other notification method available?
Not right now. Anything in specific? Y.
Steven. _______________________________________________ Beaker-devel mailing list Beaker-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/beaker-devel