Hello guys!
I'm in direct contact with the guys @ CloudSigma.
I've asked them to support CPU, RAM and HDD hot swapping and they say their system is capable of this but that linux (particularly, Fedora) isn't.
They agree to do some testing on the subject but, honestly, I don't consider myself to be sufficiently knowledgeable to lead this.
Is there anybody here interested on this?
Anyway, their lead dev told them this:
"Libvirt doesn't do any of this stuff itself (hot swapping). What it does include is an ability to use the virtio balloon driver. This essentially allows you to give a guest a large amount of ram, with it then voluntarily lending some of that memory back to the system. It requires a degree of cooperation and trust between host and guest that isn't appropriate (or easy to bill) in a public cloud, but it's a useful hack on private VM deployments in the absence of proper memory hotswap in qemu-kvm."
I've read some stuff here: http://www.linux-kvm.org/page/CPUHotPlug
It's hard to be free... but I love to struggle. Love isn't asked for; it's just given. Respect isn't asked for; it's earned! Renich Bon Ciric
Just fyi...
I use Fedora 14 with some VMs on VMWare here and CPU hotplug (add and remove) works. I have not tested it with memory yet, but I'm quite sure it will work - at least adding memory...
-of
On 01/20/2011 08:53 PM, Renich Bon Ciric wrote:
Hello guys!
I'm in direct contact with the guys @ CloudSigma.
I've asked them to support CPU, RAM and HDD hot swapping and they say their system is capable of this but that linux (particularly, Fedora) isn't.
They agree to do some testing on the subject but, honestly, I don't consider myself to be sufficiently knowledgeable to lead this.
Is there anybody here interested on this?
Anyway, their lead dev told them this:
"Libvirt doesn't do any of this stuff itself (hot swapping). What it does include is an ability to use the virtio balloon driver. This essentially allows you to give a guest a large amount of ram, with it then voluntarily lending some of that memory back to the system. It requires a degree of cooperation and trust between host and guest that isn't appropriate (or easy to bill) in a public cloud, but it's a useful hack on private VM deployments in the absence of proper memory hotswap in qemu-kvm."
I've read some stuff here: http://www.linux-kvm.org/page/CPUHotPlug
It's hard to be free... but I love to struggle. Love isn't asked for; it's just given. Respect isn't asked for; it's earned! Renich Bon Ciric
http://www.woralelandia.com/ http://www.introbella.com/ _______________________________________________ cloud mailing list cloud@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/cloud
On Thu, Jan 20, 2011 at 1:57 PM, Oliver Falk oliver@linux-kernel.at wrote:
Just fyi...
I use Fedora 14 with some VMs on VMWare here and CPU hotplug (add and remove) works. I have not tested it with memory yet, but I'm quite sure it will work - at least adding memory...
Nice!
I'm setting up a test with these guys. Wanna participate in any way?
Hi Renich!
On 01/20/2011 09:19 PM, Renich Bon Ciric wrote:
On Thu, Jan 20, 2011 at 1:57 PM, Oliver Falkoliver@linux-kernel.at wrote:
Just fyi...
I use Fedora 14 with some VMs on VMWare here and CPU hotplug (add and remove) works. I have not tested it with memory yet, but I'm quite sure it will work - at least adding memory...
Nice!
I'm setting up a test with these guys. Wanna participate in any way?
If there's an agenda beforehand and I can be of use in any way... Sure. :-) Also consider the TZ...
Best, Oliver
On Thu, Jan 20, 2011 at 2:31 PM, Oliver Falk oliver@linux-kernel.at wrote:
Hi Renich!
If there's an agenda beforehand and I can be of use in any way... Sure. :-) Also consider the TZ...
Well, they're based at Zurich and I'm in Guadalajara, México so, TZ will be... Which is yours?
On 01/20/2011 11:33 PM, Renich Bon Ciric wrote:
On Thu, Jan 20, 2011 at 2:31 PM, Oliver Falkoliver@linux-kernel.at wrote:
Hi Renich!
If there's an agenda beforehand and I can be of use in any way... Sure. :-) Also consider the TZ...
Well, they're based at Zurich and I'm in Guadalajara, México so, TZ will be... Which is yours?
Zürich is fine :-) I'm in AT -> GMT+1. So, you're the one who is the show stopper :-)
I could even consider to drive there ~800 km => ~ 500 miles, while you can't. At least not by car... :-)
-of
On Thu, Jan 20, 2011 at 4:36 PM, Oliver Falk oliver@linux-kernel.at wrote:
Zürich is fine :-) I'm in AT -> GMT+1. So, you're the one who is the show stopper :-)
Man, I'm a vampire; I love late-night stuff so, no problems here.
I could even consider to drive there ~800 km => ~ 500 miles, while you can't. At least not by car... :-)
Hahaa.. still, it's 27 ºC here @ Guadalajara in winter.... ;)
Feel free to stop by and I'll show you around, if you'd like ;)
On 01/20/2011 11:39 PM, Renich Bon Ciric wrote:
On Thu, Jan 20, 2011 at 4:36 PM, Oliver Falkoliver@linux-kernel.at wrote:
Zürich is fine :-) I'm in AT -> GMT+1. So, you're the one who is the show stopper :-)
Man, I'm a vampire; I love late-night stuff so, no problems here.
OK. It's good to know you're several thousands kms away and can not bite me :-) I do the late-night stuff on Thursday - usually, so I can stay @home on Friday :-) As you might have guessed by now...
I could even consider to drive there ~800 km => ~ 500 miles, while you can't. At least not by car... :-)
Hahaa.. still, it's 27 ºC here @ Guadalajara in winter.... ;)
You have to divide this by 10 to get Vienna temperature... *brrr*.... Man, how much I miss the weather from Phoenix, where I used to be in March 2 years ago...
Feel free to stop by and I'll show you around, if you'd like ;)
I wish, I would go there any time soon. I have open invitations from Chile, Boston, New York and New Orleans as well. I should definitely plan a 4 weeks trip for north and south America :-)
-of
On Thu, Jan 20, 2011 at 01:53:25PM -0600, Renich Bon Ciric wrote:
Hello guys!
I'm in direct contact with the guys @ CloudSigma.
I've asked them to support CPU, RAM and HDD hot swapping and they say their system is capable of this but that linux (particularly, Fedora) isn't.
They agree to do some testing on the subject but, honestly, I don't consider myself to be sufficiently knowledgeable to lead this.
Is there anybody here interested on this?
Anyway, their lead dev told them this:
"Libvirt doesn't do any of this stuff itself (hot swapping). What it does include is an ability to use the virtio balloon driver. This essentially allows you to give a guest a large amount of ram, with it then voluntarily lending some of that memory back to the system. It requires a degree of cooperation and trust between host and guest that isn't appropriate (or easy to bill) in a public cloud, but it's a useful hack on private VM deployments in the absence of proper memory hotswap in qemu-kvm."
That is correct. The live memory adjustment for KVM/QEMU via libvirt is ballooning, not memory hotplug. Likewise for Xen and VMWare. Real memory hotplug is a future RFE.
I've read some stuff here: http://www.linux-kvm.org/page/CPUHotPlug
Yep, CPU hotplug is available for KVM/libvirt, but support among common guest OS is flakey. It doesn't work at all for Windows, and for Linux only hotplug works, but not unplug. Xen however did allow hot-unplug because it was using a different mechanism IIUC.
For KVM, HDD hotplug already works for virtio disks, or SCSI disks, or USB disks. It doesn't work for IDE disks. For Xen hotplug works for xen paravirt disks.
For KVM, NIC hotplug also works, and in general any kind of emulated PCI device can be hotplugged. For Xen, again their xen paravirt NICs could be hotplugged.
Daniel
Am 21.01.2011 12:04, schrieb Daniel P. Berrange:
On Thu, Jan 20, 2011 at 01:53:25PM -0600, Renich Bon Ciric wrote:
Hello guys!
I'm in direct contact with the guys @ CloudSigma.
I've asked them to support CPU, RAM and HDD hot swapping and they say their system is capable of this but that linux (particularly, Fedora) isn't.
They agree to do some testing on the subject but, honestly, I don't consider myself to be sufficiently knowledgeable to lead this.
Is there anybody here interested on this?
Anyway, their lead dev told them this:
"Libvirt doesn't do any of this stuff itself (hot swapping). What it does include is an ability to use the virtio balloon driver. This essentially allows you to give a guest a large amount of ram, with it then voluntarily lending some of that memory back to the system. It requires a degree of cooperation and trust between host and guest that isn't appropriate (or easy to bill) in a public cloud, but it's a useful hack on private VM deployments in the absence of proper memory hotswap in qemu-kvm."
That is correct. The live memory adjustment for KVM/QEMU via libvirt is ballooning, not memory hotplug. Likewise for Xen and VMWare. Real memory hotplug is a future RFE.
I've read some stuff here: http://www.linux-kvm.org/page/CPUHotPlug
Yep, CPU hotplug is available for KVM/libvirt, but support among common guest OS is flakey. It doesn't work at all for Windows, and for Linux only hotplug works, but not unplug. Xen however did allow hot-unplug because it was using a different mechanism IIUC.
For KVM, HDD hotplug already works for virtio disks, or SCSI disks, or USB disks. It doesn't work for IDE disks. For Xen hotplug works for xen paravirt disks.
For KVM, NIC hotplug also works, and in general any kind of emulated PCI device can be hotplugged. For Xen, again their xen paravirt NICs could be hotplugged.
Erm. I'm really quite sure, I have hotplugged CPUs within my VMWare env... I will do a test again for both CPU and memory and tell you the results.
-of
On 01/21/2011 01:26 PM, Oliver Falk wrote: [ ... ]
Erm. I'm really quite sure, I have hotplugged CPUs within my VMWare env... I will do a test again for both CPU and memory and tell you the results.
Update: Using Fedora 14 and VMWare vSphere. You can add CPUs and Memory, but the OS doesn't recognise it. You can *not* remove CPUs or Memory - at lest not if you select Linux as the OS.
I must have dreamed about this being possible :-/
-of
On Fri, Jan 21, 2011 at 5:04 AM, Daniel P. Berrange berrange@redhat.com wrote:
That is correct. The live memory adjustment for KVM/QEMU via libvirt is ballooning, not memory hotplug. Likewise for Xen and VMWare. Real memory hotplug is a future RFE.
I've read some stuff here: http://www.linux-kvm.org/page/CPUHotPlug
Yep, CPU hotplug is available for KVM/libvirt, but support among common guest OS is flakey. It doesn't work at all for Windows, and for Linux only hotplug works, but not unplug. Xen however did allow hot-unplug because it was using a different mechanism IIUC.
For KVM, HDD hotplug already works for virtio disks, or SCSI disks, or USB disks. It doesn't work for IDE disks. For Xen hotplug works for xen paravirt disks.
For KVM, NIC hotplug also works, and in general any kind of emulated PCI device can be hotplugged. For Xen, again their xen paravirt NICs could be hotplugged.
So, in practical terms,
RAM "extension" is done by ballooning CPU hotplugging is possible but not unplugging. And HDD hotplugging is possible...
I read a thread (2008) that offered a patch on this ACPI technique for RAM HotPlugging. Check it out: http://www.spinics.net/lists/kvm/msg27395.html
CPU UnPlugging is not considered as a needed feature is it? http://www.linux-kvm.org/page/CPUHotPlug "Hot-unplug does not work, and actually, does not even make that much sense."
On Fri, Jan 21, 2011 at 12:34:29PM -0600, Renich Bon Ciric wrote:
On Fri, Jan 21, 2011 at 5:04 AM, Daniel P. Berrange berrange@redhat.com wrote:
That is correct. The live memory adjustment for KVM/QEMU via libvirt is ballooning, not memory hotplug. Likewise for Xen and VMWare. Real memory hotplug is a future RFE.
I've read some stuff here: http://www.linux-kvm.org/page/CPUHotPlug
Yep, CPU hotplug is available for KVM/libvirt, but support among common guest OS is flakey. It doesn't work at all for Windows, and for Linux only hotplug works, but not unplug. Xen however did allow hot-unplug because it was using a different mechanism IIUC.
For KVM, HDD hotplug already works for virtio disks, or SCSI disks, or USB disks. It doesn't work for IDE disks. For Xen hotplug works for xen paravirt disks.
For KVM, NIC hotplug also works, and in general any kind of emulated PCI device can be hotplugged. For Xen, again their xen paravirt NICs could be hotplugged.
So, in practical terms,
RAM "extension" is done by ballooning
NB, you can't ever extend RAM beyond initial boot amount with ballooning. It merely lets you adjust it to any value below the initial boot amount, and back up.
CPU hotplugging is possible but not unplugging. And HDD hotplugging is possible...
I read a thread (2008) that offered a patch on this ACPI technique for RAM HotPlugging. Check it out: http://www.spinics.net/lists/kvm/msg27395.html
Yep, not merged though. It will likely come eventually but for now the focus has been in other areas of the RAM subsystem like transparent huge pages and KSM page merging which have a far greater benefit.
CPU UnPlugging is not considered as a needed feature is it? http://www.linux-kvm.org/page/CPUHotPlug "Hot-unplug does not work, and actually, does not even make that much sense."
I wouldn't neccessarily agree with the statement on that page. I think it is perfectly valid to be able to want to unplug CPUs. If you have a 4 CPU guest on a 4 CPU host, and need to migrate the guest to a 2 CPU host, then it could well be a better idea to unplug 2 of the guest CPUs than leave it overcommitted. Thus I expect someone will make hot-unplug work eventually
Daniel
On (Fri) Jan 21 2011 [22:01:39], Daniel P. Berrange wrote:
CPU UnPlugging is not considered as a needed feature is it? http://www.linux-kvm.org/page/CPUHotPlug "Hot-unplug does not work, and actually, does not even make that much sense."
I wouldn't neccessarily agree with the statement on that page. I think it is perfectly valid to be able to want to unplug CPUs. If you have a 4 CPU guest on a 4 CPU host, and need to migrate the guest to a 2 CPU host, then it could well be a better idea to unplug 2 of the guest CPUs than leave it overcommitted. Thus I expect someone will make hot-unplug work eventually
One can always offline the CPUs from within the guest. In any case, CPU hot-unplug requires cooperation from the guest so offlining isn't any different.
Amit
-
Amit Shah -
Daniel P. Berrange -
Oliver Falk -
Renich Bon Ciric