June 2014 - virt-who-devel - Fedora Mailing-Lists

by Radek Novacek

The signed tag '0.10' was created. Tagger: Radek Novacek <rnovacek(a)redhat.com> Date: Tue Jun 3 18:09:26 2014 +0200 0.10 Changes since the last tag '0.9': Radek Novacek (11): hyperv: support Windows Server 2012 R2 spec: set default permissions for config file to 600 Make sure that virt-who ends cleanly on sigint/sigterm signal Support for sending additional information together with guest uuids Fix crash on SIGHUP signal when system is being unregistered Fix incompatibility with python 2.4 libvirt domain instance might not have state method Restructure the code Add tests for common functionality Add ability to encrypt passwords Version 0.10

9 years, 10 months

1
0
0 / 0

[virt-who] Use libvirtd as fallback when no configrations are found

by Radek Novacek

commit 1e1e0856426d01a0c2611700300f6af11a9097dd Author: Radek Novacek <rnovacek(a)redhat.com> Date: Mon Jun 16 10:52:59 2014 +0200 Use libvirtd as fallback when no configrations are found This is needed in order to maintain compatibility with older releases of virt-who. virtwho.py | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) --- diff --git a/virtwho.py b/virtwho.py index 6baac51..e313fb9 100644 --- a/virtwho.py +++ b/virtwho.py @@ -433,7 +433,10 @@ def _main(logger, options): options.username, options.password, options.owner, options.env) virtWho.configManager.addConfig(config) if len(virtWho.configManager.configs) == 0: - logger.error("No configurations found") + # In order to keep compatibility with older releases of virt-who, + # fallback to using libvirt as default virt backend + logger.info("No configurations found, using libvirt as backend") + virtWho.configManager.addConfig(Config("virt-who", "libvirt")) else: for config in virtWho.configManager.configs: logger.info("Using virt-who configuration: %s" % config.name)

9 years, 10 months

1
0
0 / 0

[virt-who] Add missing license header

by Radek Novacek

commit c815685e18a909430d657e5bd58f74f284e18670 Author: Radek Novacek <rnovacek(a)redhat.com> Date: Mon Jun 16 10:47:09 2014 +0200 Add missing license header manager/manager.py | 19 +++++++++++++++++++ 1 files changed, 19 insertions(+), 0 deletions(-) --- diff --git a/manager/manager.py b/manager/manager.py index ed104c2..f8ed382 100644 --- a/manager/manager.py +++ b/manager/manager.py @@ -1,3 +1,22 @@ +""" +Abstraction for accessing different subscription managers. + +Copyright (C) 2014 Radek Novacek <rnovacek(a)redhat.com> + +This program is free software; you can redistribute it and/or +modify it under the terms of the GNU General Public License +as published by the Free Software Foundation; either version 2 +of the License, or (at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +""" class ManagerError(Exception):

9 years, 10 months

1
0
0 / 0

[virt-who] conf: add remark about quoting special characters

by Radek Novacek

commit 496093d3d8ad8b2bee683690a4b12172ef5a7758 Author: Radek Novacek <rnovacek(a)redhat.com> Date: Mon Jun 16 10:46:46 2014 +0200 conf: add remark about quoting special characters virt-who.conf | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) --- diff --git a/virt-who.conf b/virt-who.conf index db6ac0e..fa68c7a 100644 --- a/virt-who.conf +++ b/virt-who.conf @@ -2,6 +2,9 @@ # # These enviromental variables are only used when starting virt-who as service, # otherwise you must specify them manually. +# +# Note that if some value contains special character, it must be escapted +# or the value must be quoted - for example ampersand in the password. # Start virt-who on background, perform doublefork and monitor for virtual guest # events (if possible). It is NOT recommended to turn off this option for

9 years, 10 months

1
0
0 / 0

[virt-who] libvirtd: remove import to nonexistent file

by Radek Novacek

commit 7d1f700f879f104e9c67a6125e886571994feebe Author: Radek Novacek <rnovacek(a)redhat.com> Date: Mon Jun 16 10:46:17 2014 +0200 libvirtd: remove import to nonexistent file virt/libvirtd/libvirtd.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) --- diff --git a/virt/libvirtd/libvirtd.py b/virt/libvirtd/libvirtd.py index b0b8999..1e37759 100644 --- a/virt/libvirtd/libvirtd.py +++ b/virt/libvirtd/libvirtd.py @@ -22,7 +22,6 @@ import time import logging import libvirt import threading -from event import virEventLoopPureStart import virt

9 years, 10 months

1
0
0 / 0

[virt-who] virt: add missing import to hyperv

by Radek Novacek

commit 19c5fea141a40f0fa24b899d712232fe5c3c7e05 Author: Radek Novacek <rnovacek(a)redhat.com> Date: Mon Jun 16 10:45:45 2014 +0200 virt: add missing import to hyperv virt/virt.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) --- diff --git a/virt/virt.py b/virt/virt.py index 72f924a..3db2a85 100644 --- a/virt/virt.py +++ b/virt/virt.py @@ -55,6 +55,7 @@ class Virt(object): import esx import rhevm import vdsm + import hyperv for subcls in cls.__subclasses__(): for subsubcls in subcls.__subclasses__():

9 years, 10 months

1
0
0 / 0

[virt-who] Version 0.10

by Radek Novacek

commit ef6d443bd1395bd525f02826253293a41c440e7c Author: Radek Novacek <rnovacek(a)redhat.com> Date: Tue Jun 3 16:46:40 2014 +0200 Version 0.10 Makefile | 2 +- virt-who.spec | 6 +++++- 2 files changed, 6 insertions(+), 2 deletions(-) --- diff --git a/Makefile b/Makefile index bc6d07e..8acb13d 100644 --- a/Makefile +++ b/Makefile @@ -1,6 +1,6 @@ name = virt-who -version = 0.9 +version = 0.10 .PHONY: pack check install srpm rpm rpmlint upload diff --git a/virt-who.spec b/virt-who.spec index 81619fb..d3f51b9 100644 --- a/virt-who.spec +++ b/virt-who.spec @@ -1,5 +1,5 @@ Name: virt-who -Version: 0.9 +Version: 0.10 Release: 1%{?dist} Summary: Agent for reporting virtual guest IDs to subscription-manager @@ -80,6 +80,10 @@ fi %changelog +* Tue May 20 2014 Radek Novacek <rnovacek(a)redhat.com> 0.10-1 +- Add directory with configuration files +- Version 0.10 + * Thu Mar 13 2014 Radek Novacek <rnovacek(a)redhat.com> 0.9-1 - Remove libvirt dependency - Add dependency on m2crypto

9 years, 10 months

1
0
0 / 0

[virt-who] Add ability to encrypt passwords

by Radek Novacek

commit 019a2cced0855da7775b48483e731130e2a8cc50 Author: Radek Novacek <rnovacek(a)redhat.com> Date: Tue Jun 3 17:37:25 2014 +0200 Add ability to encrypt passwords password/__init__.py | 107 ++++++++++++++++++++++++++++++++++++++++++++++++ tests/test_password.py | 62 ++++++++++++++++++++++++++++ virt-who-config.5.gz | Bin 0 -> 643 bytes virt-who-password | 10 ++++ virt-who-password.8 | 18 ++++++++ virt-who-password.8.gz | Bin 0 -> 376 bytes virt-who.8.gz | Bin 0 -> 2237 bytes virtwhopassword.py | 60 +++++++++++++++++++++++++++ 8 files changed, 257 insertions(+), 0 deletions(-) --- diff --git a/password/__init__.py b/password/__init__.py new file mode 100644 index 0000000..1c5e867 --- /dev/null +++ b/password/__init__.py @@ -0,0 +1,107 @@ +""" +Module for encrypting and decrypting passwords. + +Copyright (C) 2014 Radek Novacek <rnovacek(a)redhat.com> + +This program is free software; you can redistribute it and/or +modify it under the terms of the GNU General Public License +as published by the Free Software Foundation; either version 2 +of the License, or (at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +""" + +import os +import stat +from M2Crypto import EVP +from binascii import hexlify, unhexlify +from cStringIO import StringIO + + +class InvalidKeyFile(Exception): + pass + + +class UnwrittableKeyFile(Exception): + pass + + +class Password(object): + KEYFILE = '/var/lib/virt-who/key' + ENCRYPT = 1 + DECRYPT = 0 + + BLOCKSIZE = 16 + + @classmethod + def _pad(cls, s): + return s + (cls.BLOCKSIZE - len(s) % cls.BLOCKSIZE) * chr(cls.BLOCKSIZE - len(s) % cls.BLOCKSIZE) + + @classmethod + def _unpad(cls, s): + return s[0:-ord(s[-1])] + + + @classmethod + def _crypt(cls, op, key, iv, data): + cipher = EVP.Cipher(alg='aes_128_cbc', key=unhexlify(key), iv=unhexlify(iv), op=op, padding=False) + inf = StringIO(data) + outf = StringIO() + while 1: + buf = inf.read() + if not buf: + break + outf.write(cipher.update(buf)) + outf.write(cipher.final()) + return outf.getvalue() + + @classmethod + def encrypt(cls, password): + key, iv = cls._read_or_generate_key_iv() + return cls._crypt(cls.ENCRYPT, key, iv, cls._pad(password)) + + @classmethod + def decrypt(cls, enc): + key, iv = cls._read_key_iv() + return cls._unpad(cls._crypt(cls.DECRYPT, key, iv, enc)) + + @classmethod + def _read_key_iv(cls): + try: + with open(cls.KEYFILE, 'r') as f: + key = f.readline().strip() + iv = f.readline().strip() + if not iv or not key: + raise InvalidKeyFile("Invalid format") + return key, iv + except IOError, e: + raise InvalidKeyFile(str(e)) + + @classmethod + def _read_or_generate_key_iv(cls): + try: + return cls._read_key_iv() + except InvalidKeyFile: + pass + if os.getuid() != 0: + raise UnwrittableKeyFile("Only root can write keyfile") + key = hexlify(cls._generate_key()) + iv = hexlify(cls._generate_key()) + try: + with open(cls.KEYFILE, 'w') as f: + f.write("%s\n%s\n" % (key, iv)) + except IOError, e: + raise UnwrittableKeyFile(str(e)) + os.chmod(cls.KEYFILE, stat.S_IRUSR | stat.S_IWUSR) + return key, iv + + @classmethod + def _generate_key(cls): + return os.urandom(32) diff --git a/tests/test_password.py b/tests/test_password.py new file mode 100644 index 0000000..177b18e --- /dev/null +++ b/tests/test_password.py @@ -0,0 +1,62 @@ +""" +Test for password encryption/decryption. + +Copyright (C) 2014 Radek Novacek <rnovacek(a)redhat.com> + +This program is free software; you can redistribute it and/or +modify it under the terms of the GNU General Public License +as published by the Free Software Foundation; either version 2 +of the License, or (at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +""" + +import os +import tempfile +from binascii import hexlify, unhexlify + +from base import unittest + +from password import Password + + +class TestPassword(unittest.TestCase): + def testEncrypt(self): + self.assertEqual(hexlify( + Password._crypt( + Password.ENCRYPT, + '06a9214036b8a15b512e03d534120006', + '3dafba429d9eb430b422da802c9fac41', + 'Single block msg')), + 'e353779c1079aeb82708942dbe77181a') + + def testDecrypt(self): + self.assertEqual( + Password._crypt( + Password.DECRYPT, + '06a9214036b8a15b512e03d534120006', + '3dafba429d9eb430b422da802c9fac41', + unhexlify('e353779c1079aeb82708942dbe77181a')), + 'Single block msg') + + def testBoth(self): + f, filename = tempfile.mkstemp() + self.addCleanup(os.unlink, filename) + Password.KEYFILE = filename + pwd = "Test password" + encrypted = Password.encrypt(pwd) + self.assertEqual(pwd, Password.decrypt(encrypted)) + self.assertEqual(os) + + def testPad(self): + self.assertEqual(hexlify(Password._pad(unhexlify("00010203040506070809"))), "00010203040506070809060606060606") + + def testUnpad(self): + self.assertEqual(hexlify(Password._unpad(unhexlify("00010203040506070809060606060606"))), "00010203040506070809") diff --git a/virt-who-config.5.gz b/virt-who-config.5.gz new file mode 100644 index 0000000..7a6ab3c Binary files /dev/null and b/virt-who-config.5.gz differ diff --git a/virt-who-password b/virt-who-password new file mode 100644 index 0000000..33fd430 --- /dev/null +++ b/virt-who-password @@ -0,0 +1,10 @@ +#!/bin/sh + +if [ -f ./virtwhopassword.py ]; +then + # Run it from local directory when available + exec /usr/bin/python ./virtwhopassword.py "$@" +else + # Run it from /usr/share/virt-who + exec /usr/bin/python /usr/share/virt-who/virtwhopassword.py "$@" +fi diff --git a/virt-who-password.8 b/virt-who-password.8 new file mode 100644 index 0000000..7668fd3 --- /dev/null +++ b/virt-who-password.8 @@ -0,0 +1,18 @@ +.TH VIRT-WHO "8" "June 2014" "virt-who" +.SH NAME +virt-who-password - Utility that encrypts passwords for virt-who. +.SH SYNOPSIS +virt-who-password [-h|--help] +.SH USAGE +virt-who-password prompts for password and writes encrypted entered password +to the standard output. + +This utility must be executed as root, because the encryption key is written +into file that is only readable by root. Note that root can decrypt the +password. + +Encryption key is written into file \fB/var/lib/virt-who/key\fR, make sure +that this file is only readable and writtable by root. +.SH AUTHOR +Radek Novacek <rnovacek at redhat dot com> + diff --git a/virt-who-password.8.gz b/virt-who-password.8.gz new file mode 100644 index 0000000..7cfabbd Binary files /dev/null and b/virt-who-password.8.gz differ diff --git a/virt-who.8.gz b/virt-who.8.gz new file mode 100644 index 0000000..c032311 Binary files /dev/null and b/virt-who.8.gz differ diff --git a/virtwhopassword.py b/virtwhopassword.py new file mode 100644 index 0000000..6c1b4bb --- /dev/null +++ b/virtwhopassword.py @@ -0,0 +1,60 @@ +#!/usr/bin/python2 +""" +Command line script for password encryption. + +Copyright (C) 2012 Radek Novacek <rnovacek(a)redhat.com> + +This program is free software; you can redistribute it and/or +modify it under the terms of the GNU General Public License +as published by the Free Software Foundation; either version 2 +of the License, or (at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +""" + + +import os +import sys +from getpass import getpass +from password import Password, UnwrittableKeyFile, InvalidKeyFile +from binascii import hexlify + +if __name__ == '__main__': + if len(sys.argv) == 2 and sys.argv[1] in ('-h', '--help'): + print """Utility that encrypts passwords for virt-who. + +Enter password that should be encrypted. This encrypted password then can be +supplied to virt-who configuration. + +This command must be executed as root! + +WARNING: root user can still decrypt encrypted passwords! +""" + sys.exit(0) + + if os.getuid() != 0: + print >>sys.stderr, "Only root can encrypt passwords" + sys.exit(1) + + try: + pwd = getpass("Password: ") + except (KeyboardInterrupt, EOFError): + print + sys.exit(1) + try: + enc = Password.encrypt(pwd) + except UnwrittableKeyFile: + print >>sys.stderr, "Keyfile %s doesn't exist and can't be created, rerun as root" % Password.KEYFILE + sys.exit(1) + except InvalidKeyFile: + print >>sys.stderr, "Can't access keyfile %s, rerun as root" % Password.KEYFILE + sys.exit(1) + print >>sys.stderr, "Use following as value for encrypted_password key in the configuration file:" + print hexlify(enc)

9 years, 10 months

1
0
0 / 0

[virt-who] Add tests for common functionality

by Radek Novacek

commit 72539d5455d942b0b68ecd54f693e2a3bce29a54 Author: Radek Novacek <rnovacek(a)redhat.com> Date: Tue Jun 3 16:42:35 2014 +0200 Add tests for common functionality tests/base.py | 28 ++++++ tests/test_config.py | 186 +++++++++++++++++++++++++++++++++++++ tests/test_esx.py | 56 +++++++++++ tests/test_libvirtd.py | 77 +++++++++++++++ tests/test_satellite.py | 157 +++++++++++++++++++++++++++++++ tests/test_subscriptionmanager.py | 7 ++ tests/test_virtwho.py | 131 ++++++++++++++++++++++++++ 7 files changed, 642 insertions(+), 0 deletions(-) --- diff --git a/tests/base.py b/tests/base.py new file mode 100644 index 0000000..d4edeff --- /dev/null +++ b/tests/base.py @@ -0,0 +1,28 @@ +""" +Basic module for tests, + +Copyright (C) 2014 Radek Novacek <rnovacek(a)redhat.com> + +This program is free software; you can redistribute it and/or +modify it under the terms of the GNU General Public License +as published by the Free Software Foundation; either version 2 +of the License, or (at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +""" + + +# hack to use unittest2 on python <= 2.6, unittest otherwise +# based on python version +import sys +if sys.version_info[0] > 2 or sys.version_info[1] > 6: + import unittest +else: + import unittest2 as unittest diff --git a/tests/test_config.py b/tests/test_config.py new file mode 100644 index 0000000..f549b1c --- /dev/null +++ b/tests/test_config.py @@ -0,0 +1,186 @@ +""" +Test reading and writing configuration files. + +Copyright (C) 2014 Radek Novacek <rnovacek(a)redhat.com> + +This program is free software; you can redistribute it and/or +modify it under the terms of the GNU General Public License +as published by the Free Software Foundation; either version 2 +of the License, or (at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +""" + + +import os +import shutil +from config import ConfigManager, InvalidOption +from tempfile import mkdtemp +from base import unittest + + +class TestReadingConfigs(unittest.TestCase): + def setUp(self): + self.config_dir = mkdtemp() + self.addCleanup(shutil.rmtree, self.config_dir) + + def testEmptyConfig(self): + manager = ConfigManager(self.config_dir) + self.assertEqual(len(manager.configs), 0) + + def testBasicConfig(self): + with open(os.path.join(self.config_dir, "test.conf"), "w") as f: + f.write(""" +[test] +type=esx +server=1.2.3.4 +username=admin +password=password +owner=root +env=staging +""") + + manager = ConfigManager(self.config_dir) + self.assertEqual(len(manager.configs), 1) + config = manager.configs[0] + self.assertEqual(config.name, "test") + self.assertEqual(config.type, "esx") + self.assertEqual(config.server, "1.2.3.4") + self.assertEqual(config.username, "admin") + self.assertEqual(config.password, "password") + self.assertEqual(config.owner, "root") + self.assertEqual(config.env, "staging") + + def testInvalidConfig(self): + with open(os.path.join(self.config_dir, "test.conf"), "w") as f: + f.write(""" +Malformed configuration file +""") + manager = ConfigManager(self.config_dir) + self.assertEqual(len(manager.configs), 0) + + def testInvalidType(self): + filename = os.path.join(self.config_dir, "test.conf") + with open(filename, "w") as f: + f.write(""" +[test] +type=invalid +server=1.2.3.4 +username=test +""") + self.assertRaises(InvalidOption, ConfigManager, self.config_dir) + + @unittest.skipIf(os.getuid() == 0, "Can't create unreadable file when running as root") + def testUnreadableConfig(self): + filename = os.path.join(self.config_dir, "test.conf") + with open(filename, "w") as f: + f.write(""" +[test] +type=esx +server=1.2.3.4 +username=admin +password=password +owner=root +env=staging +""") + os.chmod(filename, 0) + manager = ConfigManager(self.config_dir) + self.assertEqual(len(manager.configs), 0) + + def testNoOptionsConfig(self): + with open(os.path.join(self.config_dir, "test.conf"), "w") as f: + f.write(""" +[test] +type=esx +""") + manager = ConfigManager(self.config_dir) + self.assertEqual(len(manager.configs), 0) + + def testMultipleConfigsInFile(self): + with open(os.path.join(self.config_dir, "test.conf"), "w") as f: + f.write(""" +[test1] +type=esx +server=1.2.3.4 +username=admin +password=password +owner=root +env=staging + +[test2] +type=hyperv +server=1.2.3.5 +username=admin +password=password +owner=root +env=staging +""") + + manager = ConfigManager(self.config_dir) + self.assertEqual(len(manager.configs), 2) + config = manager.configs[0] + self.assertEqual(config.name, "test1") + self.assertEqual(config.type, "esx") + self.assertEqual(config.server, "1.2.3.4") + self.assertEqual(config.username, "admin") + self.assertEqual(config.password, "password") + self.assertEqual(config.owner, "root") + self.assertEqual(config.env, "staging") + config = manager.configs[1] + self.assertEqual(config.name, "test2") + self.assertEqual(config.type, "hyperv") + self.assertEqual(config.username, "admin") + self.assertEqual(config.server, "1.2.3.5") + self.assertEqual(config.password, "password") + self.assertEqual(config.owner, "root") + self.assertEqual(config.env, "staging") + + def testMultipleConfigFiles(self): + with open(os.path.join(self.config_dir, "test1.conf"), "w") as f: + f.write(""" +[test1] +type=esx +server=1.2.3.4 +username=admin +password=password +owner=root +env=staging +""") + with open(os.path.join(self.config_dir, "test2.conf"), "w") as f: + f.write(""" +[test2] +type=hyperv +server=1.2.3.5 +username=admin +password=password +owner=root +env=staging +""") + + manager = ConfigManager(self.config_dir) + self.assertEqual(len(manager.configs), 2) + + config2, config1 = manager.configs + + self.assertEqual(config1.name, "test1") + self.assertEqual(config1.type, "esx") + self.assertEqual(config1.server, "1.2.3.4") + self.assertEqual(config1.username, "admin") + self.assertEqual(config1.password, "password") + self.assertEqual(config1.owner, "root") + self.assertEqual(config1.env, "staging") + + self.assertEqual(config2.name, "test2") + self.assertEqual(config2.type, "hyperv") + self.assertEqual(config2.server, "1.2.3.5") + self.assertEqual(config2.username, "admin") + self.assertEqual(config2.password, "password") + self.assertEqual(config2.owner, "root") + self.assertEqual(config2.env, "staging") diff --git a/tests/test_esx.py b/tests/test_esx.py new file mode 100644 index 0000000..8c63760 --- /dev/null +++ b/tests/test_esx.py @@ -0,0 +1,56 @@ +""" +Test of ESX virtualization backend. + +Copyright (C) 2014 Radek Novacek <rnovacek(a)redhat.com> + +This program is free software; you can redistribute it and/or +modify it under the terms of the GNU General Public License +as published by the Free Software Foundation; either version 2 +of the License, or (at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +""" + +import logging +import urllib2 +import suds +from mock import patch + +from base import unittest +from config import Config +from virt.esx import Esx +from virt import VirtError + + +class TestEsx(unittest.TestCase): + def setUp(self): + logger = logging.getLogger() + config = Config('test', 'esx', 'localhost', 'username', 'password', 'owner', 'env') + self.esx = Esx(logger, config) + + @patch('suds.client.Client') + def test_connect(self, mock_client): + self.esx.getHostGuestMapping() + + self.assertTrue(mock_client.called) + mock_client.assert_called_with("https://localhost/sdk/vimService.wsdl") + mock_client.return_value.set_options.assert_called_once_with(location="https://localhost/sdk") + mock_client.service.RetrieveServiceContent.assert_called_once() + mock_client.service.Login.assert_called_once() + + @patch('suds.client.Client') + def test_connection_timeout(self, mock_client): + mock_client.side_effect = urllib2.URLError('timed out') + self.assertRaises(VirtError, self.esx.getHostGuestMapping) + + @patch('suds.client.Client') + def test_invalid_login(self, mock_client): + mock_client.return_value.service.Login.side_effect = suds.WebFault('Permission to perform this operation was denied.', '') + self.assertRaises(VirtError, self.esx.getHostGuestMapping) diff --git a/tests/test_libvirtd.py b/tests/test_libvirtd.py new file mode 100644 index 0000000..6077702 --- /dev/null +++ b/tests/test_libvirtd.py @@ -0,0 +1,77 @@ +""" +Test for libvirt virtualization backend. + +Copyright (C) 2014 Radek Novacek <rnovacek(a)redhat.com> + +This program is free software; you can redistribute it and/or +modify it under the terms of the GNU General Public License +as published by the Free Software Foundation; either version 2 +of the License, or (at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +""" + +import threading +from base import unittest +from mock import patch, Mock +import logging + +from config import Config +from virt import Virt, Domain, VirtError +from virt.libvirtd.libvirtd import LibvirtMonitor + + +def raiseLibvirtError(*args, **kwargs): + import libvirt + raise libvirt.libvirtError('') + + +class TestLibvirtd(unittest.TestCase): + def setUp(self): + pass + + @patch('libvirt.openReadOnly') + def test_read(self, libvirt): + logger = logging.getLogger() + config = Config('test', 'libvirt') + libvirtd = Virt.fromConfig(logger, config) + domains = libvirtd.listDomains() + libvirt.assert_called_with("") + + @patch('libvirt.openReadOnly') + def test_read_fail(self, virt): + logger = logging.getLogger() + config = Config('test', 'libvirt') + libvirtd = Virt.fromConfig(logger, config) + virt.side_effect = raiseLibvirtError + self.assertRaises(VirtError, libvirtd.listDomains) + + @patch('libvirt.openReadOnly') + def test_monitoring(self, virt): + event = threading.Event() + LibvirtMonitor().set_event(event) + LibvirtMonitor()._prepare() + LibvirtMonitor()._checkChange() + + virt.assert_called_with('') + virt.return_value.listDomainsID.assert_called() + virt.return_value.listDefinedDomains.assert_called() + virt.return_value.closed.assert_called() + self.assertFalse(event.is_set()) + + virt.return_value.listDomainsID.return_value = [1] + LibvirtMonitor()._checkChange() + self.assertTrue(event.is_set()) + event.clear() + + virt.return_value.listDomainsID.return_value = [] + LibvirtMonitor()._checkChange() + self.assertTrue(event.is_set()) + event.clear() diff --git a/tests/test_satellite.py b/tests/test_satellite.py new file mode 100644 index 0000000..e3cbac9 --- /dev/null +++ b/tests/test_satellite.py @@ -0,0 +1,157 @@ +""" +Test for Satellite module, part of virt-who + +Copyright (C) 2013 Radek Novacek <rnovacek(a)redhat.com> + +This program is free software; you can redistribute it and/or +modify it under the terms of the GNU General Public License +as published by the Free Software Foundation; either version 2 +of the License, or (at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +""" +import os +import sys + +from base import unittest + +import logging +import threading +import tempfile +import pickle +from SimpleXMLRPCServer import SimpleXMLRPCServer, SimpleXMLRPCRequestHandler + +from manager.satellite import Satellite, SatelliteError + +TEST_SYSTEM_ID = 'test-system-id' + + +class RequestHandler(SimpleXMLRPCRequestHandler): + rpc_paths = ('/XMLRPC',) + + +class FakeSatellite(SimpleXMLRPCServer): + def __init__(self): + SimpleXMLRPCServer.__init__(self, ("localhost", 8080), requestHandler=RequestHandler) + self.register_function(self.new_system_user_pass, "registration.new_system_user_pass") + self.register_function(self.refresh_hw_profile, "registration.refresh_hw_profile") + self.register_function(self.virt_notify, "registration.virt_notify") + + def new_system_user_pass(self, profile_name, os_release_name, version, arch, username, password, options): + if username != "username": + raise Exception("Wrong username") + if password != "password": + raise Exception("Wrong password") + print "RPC: new_system_user_pass", profile_name, os_release_name, version, arch, username, password, options + return {'system_id': TEST_SYSTEM_ID} + + def refresh_hw_profile(self, system_id, profile): + print "RPC: refresh_hw_profile", system_id, profile + if system_id != TEST_SYSTEM_ID: + raise Exception("Wrong system id") + return "" + + def virt_notify(self, system_id, plan): + print "RPC: virt_notify", system_id, plan + if system_id != TEST_SYSTEM_ID: + raise Exception("Wrong system id") + + if plan[0] != [0, 'exists', 'system', {'uuid': '0000000000000000', 'identity': 'host'}]: + raise Exception("Wrong value for virt_notify: invalid format of first entry") + if plan[1] != [0, 'crawl_began', 'system', {}]: + raise Exception("Wrong value for virt_notify: invalid format of second entry") + if plan[-1] != [0, 'crawl_ended', 'system', {}]: + raise Exception("Wrong value for virt_notify: invalid format of last entry") + for item in plan[2:-1]: + if item[0] != 0: + raise Exception("Wrong value for virt_notify: invalid format first item of an entry") + if item[1] != 'exists': + raise Exception("Wrong value for virt_notify: invalid format second item of an entry") + if item[2] != 'domain': + raise Exception("Wrong value for virt_notify: invalid format third item of an entry") + if not item[3]['uuid'].startswith("guest"): + raise Exception("Wrong value for virt_notify: invalid format uuid item") + return 0 + + +class Options(object): + def __init__(self, server, username, password): + self.server = server + self.username = username + self.password = password + + +class TestSatellite(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.logger = logging.getLogger() + cls.fake_server = FakeSatellite() + cls.thread = threading.Thread(target=cls.fake_server.serve_forever) + cls.thread.start() + + @classmethod + def tearDownClass(cls): + cls.fake_server.shutdown() + + def test_wrong_server(self): + options = Options("wrong_server", "abc", "def") + s = Satellite(self.logger, options) + #self.assertRaises(SatelliteError, s.connect, "wrong_server", "abc", "def") + s.hypervisorCheckIn("owner", "env", {}, "test") + #self.assertRaises(SatelliteError, s.connect, "localhost", "abc", "def") + + def test_new_system(self): + options = Options("http://localhost:8080", "username", "password") + options.force_register = True + s = Satellite(self.logger, options) + + # Register with wrong username + #self.assertRaises(SatelliteError, s.connect, "http://localhost:8080", "wrong", "password", force_register=True) + + # Register with wrong password + #self.assertRaises(SatelliteError, s.connect, "http://localhost:8080", "username", "wrong", force_register=True) + + def test_hypervisorCheckIn(self): + options = Options("http://localhost:8080", "username", "password") + options.force_register = True + s = Satellite(self.logger, options) + + mapping = { + 'host-1': ['guest1-1', 'guest1-2'], + 'host-2': ['guest2-1', 'guest2-2', 'guest2-3'] + } + result = s.hypervisorCheckIn("owner", "env", mapping, "type") + self.assertTrue("failedUpdate" in result) + self.assertTrue("created" in result) + self.assertTrue("updated" in result) + + def test_hypervisorCheckIn_preregistered(self): + temp, filename = tempfile.mkstemp(suffix=TEST_SYSTEM_ID) + self.addCleanup(os.unlink, filename) + f = os.fdopen(temp, "wb") + pickle.dump({'system_id': TEST_SYSTEM_ID}, f) + f.close() + + options = Options("http://localhost:8080", "username", "password") + s = Satellite(self.logger, options) + + s.HYPERVISOR_SYSTEMID_FILE = filename.replace(TEST_SYSTEM_ID, '%s') + mapping = { + 'host-1': ['guest1-1', 'guest1-2'], + 'host-2': ['guest2-1', 'guest2-2', 'guest2-3'] + } + result = s.hypervisorCheckIn("owner", "env", mapping, "type") + self.assertTrue("failedUpdate" in result) + self.assertTrue("created" in result) + self.assertTrue("updated" in result) + + +if __name__ == '__main__': + unittest.main() diff --git a/tests/test_subscriptionmanager.py b/tests/test_subscriptionmanager.py new file mode 100644 index 0000000..ea13c48 --- /dev/null +++ b/tests/test_subscriptionmanager.py @@ -0,0 +1,7 @@ + +from base import unittest + + +class TestSubscriptionManager(unittest.TestCase): + def test_(self): + pass diff --git a/tests/test_virtwho.py b/tests/test_virtwho.py new file mode 100644 index 0000000..6e717b5 --- /dev/null +++ b/tests/test_virtwho.py @@ -0,0 +1,131 @@ +""" +Test for basic virt-who operations. + +Copyright (C) 2014 Radek Novacek <rnovacek(a)redhat.com> + +This program is free software; you can redistribute it and/or +modify it under the terms of the GNU General Public License +as published by the Free Software Foundation; either version 2 +of the License, or (at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +""" + +import sys +import os +from base import unittest +import logging + +from mock import patch, Mock + +from virtwho import parseOptions, VirtWho +from config import Config +from virt import VirtError +from manager import ManagerError + + +class TestOptions(unittest.TestCase): + def setUp(self): + self.clearEnv() + + def clearEnv(self): + for key in os.environ.keys(): + if key.startswith("VIRTWHO"): + del os.environ[key] + + def test_default_cmdline_options(self): + sys.argv = ["virtwho.py"] + _, options = parseOptions() + self.assertFalse(options.debug) + self.assertFalse(options.background) + self.assertFalse(options.oneshot) + self.assertEqual(options.interval, 3600) + self.assertEqual(options.smType, 'sam') + self.assertEqual(options.virtType, None) + + def test_options_debug(self): + sys.argv = ["virtwho.py", "-d"] + _, options = parseOptions() + self.assertTrue(options.debug) + + sys.argv = ["virtwho.py"] + os.environ["VIRTWHO_DEBUG"] = "1" + _, options = parseOptions() + self.assertTrue(options.debug) + + def test_options_virt(self): + for virt in ['esx', 'hyperv', 'rhevm']: + self.clearEnv() + sys.argv = ["virtwho.py", "--%s" % virt, "--%s-owner=owner" % virt, + "--%s-env=env" % virt, "--%s-server=localhost" % virt, + "--%s-username=username" % virt, + "--%s-password=password" % virt] + _, options = parseOptions() + self.assertEqual(options.virtType, virt) + self.assertEqual(options.owner, 'owner') + self.assertEqual(options.env, 'env') + self.assertEqual(options.server, 'localhost') + self.assertEqual(options.username, 'username') + self.assertEqual(options.password, 'password') + + sys.argv = ["virtwho.py"] + virt_up = virt.upper() + os.environ["VIRTWHO_%s" % virt_up] = "1" + os.environ["VIRTWHO_%s_OWNER" % virt_up] = "xowner" + os.environ["VIRTWHO_%s_ENV" % virt_up] = "xenv" + os.environ["VIRTWHO_%s_SERVER" % virt_up] = "xlocalhost" + os.environ["VIRTWHO_%s_USERNAME" % virt_up] = "xusername" + os.environ["VIRTWHO_%s_PASSWORD" % virt_up] = "xpassword" + _, options = parseOptions() + self.assertEqual(options.virtType, virt) + self.assertEqual(options.owner, 'xowner') + self.assertEqual(options.env, 'xenv') + self.assertEqual(options.server, 'xlocalhost') + self.assertEqual(options.username, 'xusername') + self.assertEqual(options.password, 'xpassword') + + @patch('virt.Virt.fromConfig') + @patch('manager.Manager.fromOptions') + def test_sending_guests(self, fromOptions, fromConfig): + logger = logging.getLogger() + options = Mock() + options.oneshot = True + virtwho = VirtWho(logger, options) + config = Config("test", "esx", "localhost", "username", "password", "owner", "env") + virtwho.configManager.addConfig(config) + self.assertTrue(virtwho.send()) + + fromConfig.assert_called_with(logger, config) + self.assertTrue(fromConfig.return_value.getHostGuestMapping.called) + fromOptions.assert_called_with(logger, options) + + @patch('virt.Virt.fromConfig') + @patch('manager.Manager.fromOptions') + def test_sending_guests_errors(self, fromOptions, fromConfig): + logger = logging.getLogger() + options = Mock() + options.oneshot = True + virtwho = VirtWho(logger, options) + config = Config("test", "esx", "localhost", "username", "password", "owner", "env") + virtwho.configManager.addConfig(config) + fromConfig.return_value.getHostGuestMapping.side_effect = VirtError + self.assertFalse(virtwho.send()) + + fromConfig.assert_called_with(logger, config) + self.assertTrue(fromConfig.return_value.getHostGuestMapping.called) + fromOptions.assert_not_called() + + fromConfig.return_value.getHostGuestMapping.side_effect = None + fromOptions.return_value.hypervisorCheckIn.side_effect = ManagerError + self.assertFalse(virtwho.send()) + fromConfig.assert_called_with(logger, config) + self.assertTrue(fromConfig.return_value.getHostGuestMapping.called) + fromOptions.assert_called() + self.assertTrue(fromOptions.return_value.hypervisorCheckIn.called)

9 years, 10 months

1
0
0 / 0

[virt-who] Restructure the code

by Radek Novacek

commit cda53731c1807aad916f1ca07392fc13eacfb9d0 Author: Radek Novacek <rnovacek(a)redhat.com> Date: Tue Jun 3 16:37:08 2014 +0200 Restructure the code Many improvements in the code structure, some highlights: * All the virt backend are inherited from base class * All the subscription managers are inherited from base class * Modules moved to subdirectories * Ability to use configuration files * Use daemon library for daemonization * Makefile | 12 +- config.py | 150 ++++ daemon/daemon.py | 775 ++++++++++++++++++++ event.py | 383 ---------- log.py | 5 +- manager/__init__.py | 2 + manager/manager.py | 24 + manager/satellite/__init__.py | 2 + satellite.py => manager/satellite/satellite.py | 61 +- manager/subscriptionmanager/__init__.py | 2 + .../subscriptionmanager/subscriptionmanager.py | 41 +- test.py | 83 --- virt-who | 6 +- virt-who-config.5 | 50 ++ virt-who.spec | 9 +- virt.py | 148 ---- virt/__init__.py | 3 + virt/esx/__init__.py | 2 + vsphere.py => virt/esx/esx.py | 52 +- virt/hyperv/__init__.py | 2 + hyperv.py => virt/hyperv/hyperv.py | 61 ++- ntlm.py => virt/hyperv/ntlm.py | 224 +++--- virt/libvirtd/__init__.py | 2 + virt/libvirtd/libvirtd.py | 168 +++++ virt/rhevm/__init__.py | 2 + rhevm.py => virt/rhevm/rhevm.py | 9 +- virt/vdsm/__init__.py | 2 + vdsm.py => virt/vdsm/vdsm.py | 15 +- virt/virt.py | 88 +++ virt-who.py => virtwho.py | 442 ++++-------- 30 files changed, 1731 insertions(+), 1094 deletions(-) --- diff --git a/Makefile b/Makefile index dffebad..bc6d07e 100644 --- a/Makefile +++ b/Makefile @@ -13,13 +13,23 @@ check: pyflakes *.py install: - install -d $(DESTDIR)/usr/share/$(name)/ $(DESTDIR)/usr/bin $(DESTDIR)/etc/rc.d/init.d $(DESTDIR)/etc/sysconfig $(DESTDIR)/usr/share/man/man8/ + for dir in {daemon,password,manager,virt} manager/{satellite,subscriptionmanager} virt/{esx,hyperv,libvirtd,rhevm,vdsm}; do \ + echo $$dir ; \ + install -d $(DESTDIR)/usr/share/$(name)/$$dir/ ; \ + install -pm 0644 $$dir/*.py $(DESTDIR)/usr/share/$(name)/$$dir/ ; \ + done install -pm 0644 *.py $(DESTDIR)/usr/share/$(name)/ + install -d $(DESTDIR)/usr/bin $(DESTDIR)/etc/rc.d/init.d $(DESTDIR)/etc/sysconfig $(DESTDIR)/usr/share/man/man8/ $(DESTDIR)/usr/share/man/man5/ $(DESTDIR)/var/lib/virt-who/ install virt-who $(DESTDIR)/usr/bin/ + install virt-who-password $(DESTDIR)/usr/bin/ install virt-who-initscript $(DESTDIR)/etc/rc.d/init.d/virt-who install -pm 0644 virt-who.conf $(DESTDIR)/etc/sysconfig/virt-who gzip -c virt-who.8 > virt-who.8.gz install -pm 0644 virt-who.8.gz $(DESTDIR)/usr/share/man/man8/ + gzip -c virt-who-password.8 > virt-who-password.8.gz + install -pm 0644 virt-who-password.8.gz $(DESTDIR)/usr/share/man/man8/ + gzip -c virt-who-config.5 > virt-who-config.5.gz + install -pm 0644 virt-who-config.5.gz $(DESTDIR)/usr/share/man/man5/ srpm: pack rpmbuild --define "_sourcedir $(PWD)" --define "_specdir $(PWD)" --define "_srcrpmdir $(PWD)" -bs $(name).spec diff --git a/config.py b/config.py new file mode 100644 index 0000000..f366180 --- /dev/null +++ b/config.py @@ -0,0 +1,150 @@ +""" +Module for reading configuration files + +Copyright (C) 2011 Radek Novacek <rnovacek(a)redhat.com> + +This program is free software; you can redistribute it and/or +modify it under the terms of the GNU General Public License +as published by the Free Software Foundation; either version 2 +of the License, or (at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +""" + +import os +import logging +from ConfigParser import SafeConfigParser, NoOptionError, Error, MissingSectionHeaderError +from password import Password + + +VIRTWHO_CONF_DIR = "/etc/virt-who.d/" +VIRTWHO_TYPES = ("libvirt", "vdsm", "esx", "rhevm", "hyperv") + + +class InvalidOption(Error): + pass + + +class Config(object): + def __init__(self, name, type, server=None, username=None, password=None, owner=None, env=None): + self._name = name + self._type = type + if self._type not in VIRTWHO_TYPES: + raise InvalidOption('Invalid type "%s", must be one of following %s' % (self._type, ", ".join(VIRTWHO_TYPES))) + self._server = server + self._username = username + self._password = password + self._owner = owner + self._env = env + + def _read_password(self, name, parser): + try: + password = parser.get(name, "password") + except NoOptionError: + password = None + if password is None: + try: + crypted = parser.get(name, "encrypted_password") + password = Password.decrypt(password) + except NoOptionError: + return None + + @classmethod + def fromParser(self, name, parser): + type = parser.get(name, "type").lower() + server = username = password = owner = env = None + if type != 'libvirt': + server = parser.get(name, "server") + username = parser.get(name, "username") + + try: + password = parser.get(name, "password") + except NoOptionError: + password = None + if password is None: + try: + crypted = parser.get(name, "encrypted_password") + password = Password.decrypt(crypted) + except NoOptionError: + password = None + + try: + owner = parser.get(name, "owner") + except NoOptionError: + owner = None + try: + env = parser.get(name, "env") + except NoOptionError: + env = None + return Config(name, type, server, username, password, owner, env) + + @property + def name(self): + return self._name + + @property + def type(self): + return self._type + + @property + def server(self): + return self._server + + @property + def username(self): + return self._username + + @property + def password(self): + return self._password + + @property + def owner(self): + return self._owner + + @property + def env(self): + return self._env + + +class ConfigManager(object): + def __init__(self, config_dir=VIRTWHO_CONF_DIR): + self._parser = SafeConfigParser() + self._configs = [] + try: + config_dir_content = os.listdir(config_dir) + except OSError: + logging.warn("Configuration directory '%s' doesn't exist or is not accessible" % config_dir) + return + for conf in config_dir_content: + try: + filename = self._parser.read(os.path.join(config_dir, conf)) + if len(filename) == 0: + logging.error("Unable to read configuration file %s" % conf) + except MissingSectionHeaderError: + logging.error("Configuration file %s contains no section headers" % conf) + + self._readConfig() + + def _readConfig(self): + self._configs = [] + for section in self._parser.sections(): + try: + config = Config.fromParser(section, self._parser) + self._configs.append(config) + except NoOptionError, e: + logging.error(str(e)) + + @property + def configs(self): + return self._configs + + def addConfig(self, config): + self._configs.append(config) diff --git a/daemon/__init__.py b/daemon/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/daemon/daemon.py b/daemon/daemon.py new file mode 100644 index 0000000..2749322 --- /dev/null +++ b/daemon/daemon.py @@ -0,0 +1,775 @@ +# -*- coding: utf-8 -*- + +# daemon/daemon.py +# Part of python-daemon, an implementation of PEP 3143. +# +# Copyright © 2008–2010 Ben Finney <ben+python(a)benfinney.id.au> +# Copyright © 2007–2008 Robert Niederreiter, Jens Klein +# Copyright © 2004–2005 Chad J. Schroeder +# Copyright © 2003 Clark Evans +# Copyright © 2002 Noah Spurrier +# Copyright © 2001 Jürgen Hermann +# +# This is free software: you may copy, modify, and/or distribute this work +# under the terms of the Python Software Foundation License, version 2 or +# later as published by the Python Software Foundation. +# No warranty expressed or implied. See the file LICENSE.PSF-2 for details. + +""" Daemon process behaviour. + """ + +import os +import sys +import resource +import errno +import signal +import socket +import atexit + + +class DaemonError(Exception): + """ Base exception class for errors from this module. """ + + +class DaemonOSEnvironmentError(DaemonError, OSError): + """ Exception raised when daemon OS environment setup receives error. """ + + +class DaemonProcessDetachError(DaemonError, OSError): + """ Exception raised when process detach fails. """ + + +class DaemonContext(object): + """ Context for turning the current program into a daemon process. + + A `DaemonContext` instance represents the behaviour settings and + process context for the program when it becomes a daemon. The + behaviour and environment is customised by setting options on the + instance, before calling the `open` method. + + Each option can be passed as a keyword argument to the `DaemonContext` + constructor, or subsequently altered by assigning to an attribute on + the instance at any time prior to calling `open`. That is, for + options named `wibble` and `wubble`, the following invocation:: + + foo = daemon.DaemonContext(wibble=bar, wubble=baz) + foo.open() + + is equivalent to:: + + foo = daemon.DaemonContext() + foo.wibble = bar + foo.wubble = baz + foo.open() + + The following options are defined. + + `files_preserve` + :Default: ``None`` + + List of files that should *not* be closed when starting the + daemon. If ``None``, all open file descriptors will be closed. + + Elements of the list are file descriptors (as returned by a file + object's `fileno()` method) or Python `file` objects. Each + specifies a file that is not to be closed during daemon start. + + `chroot_directory` + :Default: ``None`` + + Full path to a directory to set as the effective root directory of + the process. If ``None``, specifies that the root directory is not + to be changed. + + `working_directory` + :Default: ``'/'`` + + Full path of the working directory to which the process should + change on daemon start. + + Since a filesystem cannot be unmounted if a process has its + current working directory on that filesystem, this should either + be left at default or set to a directory that is a sensible “home + directory” for the daemon while it is running. + + `umask` + :Default: ``0`` + + File access creation mask (“umask”) to set for the process on + daemon start. + + Since a process inherits its umask from its parent process, + starting the daemon will reset the umask to this value so that + files are created by the daemon with access modes as it expects. + + `pidfile` + :Default: ``None`` + + Context manager for a PID lock file. When the daemon context opens + and closes, it enters and exits the `pidfile` context manager. + + `detach_process` + :Default: ``None`` + + If ``True``, detach the process context when opening the daemon + context; if ``False``, do not detach. + + If unspecified (``None``) during initialisation of the instance, + this will be set to ``True`` by default, and ``False`` only if + detaching the process is determined to be redundant; for example, + in the case when the process was started by `init`, by `initd`, or + by `inetd`. + + `signal_map` + :Default: system-dependent + + Mapping from operating system signals to callback actions. + + The mapping is used when the daemon context opens, and determines + the action for each signal's signal handler: + + * A value of ``None`` will ignore the signal (by setting the + signal action to ``signal.SIG_IGN``). + + * A string value will be used as the name of an attribute on the + ``DaemonContext`` instance. The attribute's value will be used + as the action for the signal handler. + + * Any other value will be used as the action for the + signal handler. See the ``signal.signal`` documentation + for details of the signal handler interface. + + The default value depends on which signals are defined on the + running system. Each item from the list below whose signal is + actually defined in the ``signal`` module will appear in the + default map: + + * ``signal.SIGTTIN``: ``None`` + + * ``signal.SIGTTOU``: ``None`` + + * ``signal.SIGTSTP``: ``None`` + + * ``signal.SIGTERM``: ``'terminate'`` + + Depending on how the program will interact with its child + processes, it may need to specify a signal map that + includes the ``signal.SIGCHLD`` signal (received when a + child process exits). See the specific operating system's + documentation for more detail on how to determine what + circumstances dictate the need for signal handlers. + + `uid` + :Default: ``os.getuid()`` + + `gid` + :Default: ``os.getgid()`` + + The user ID (“UID”) value and group ID (“GID”) value to switch + the process to on daemon start. + + The default values, the real UID and GID of the process, will + relinquish any effective privilege elevation inherited by the + process. + + `prevent_core` + :Default: ``True`` + + If true, prevents the generation of core files, in order to avoid + leaking sensitive information from daemons run as `root`. + + `stdin` + :Default: ``None`` + + `stdout` + :Default: ``None`` + + `stderr` + :Default: ``None`` + + Each of `stdin`, `stdout`, and `stderr` is a file-like object + which will be used as the new file for the standard I/O stream + `sys.stdin`, `sys.stdout`, and `sys.stderr` respectively. The file + should therefore be open, with a minimum of mode 'r' in the case + of `stdin`, and mode 'w+' in the case of `stdout` and `stderr`. + + If the object has a `fileno()` method that returns a file + descriptor, the corresponding file will be excluded from being + closed during daemon start (that is, it will be treated as though + it were listed in `files_preserve`). + + If ``None``, the corresponding system stream is re-bound to the + file named by `os.devnull`. + + """ + + def __init__( + self, + chroot_directory=None, + working_directory='/', + umask=0, + uid=None, + gid=None, + prevent_core=True, + detach_process=None, + files_preserve=None, + pidfile=None, + stdin=None, + stdout=None, + stderr=None, + signal_map=None): + """ Set up a new instance. """ + self.chroot_directory = chroot_directory + self.working_directory = working_directory + self.umask = umask + self.prevent_core = prevent_core + self.files_preserve = files_preserve + self.pidfile = pidfile + self.stdin = stdin + self.stdout = stdout + self.stderr = stderr + + if uid is None: + uid = os.getuid() + self.uid = uid + if gid is None: + gid = os.getgid() + self.gid = gid + + if detach_process is None: + detach_process = is_detach_process_context_required() + self.detach_process = detach_process + + if signal_map is None: + signal_map = make_default_signal_map() + self.signal_map = signal_map + + self._is_open = False + + @property + def is_open(self): + """ ``True`` if the instance is currently open. """ + return self._is_open + + def open(self): + """ Become a daemon process. + :Return: ``None`` + + Open the daemon context, turning the current program into a daemon + process. This performs the following steps: + + * If this instance's `is_open` property is true, return + immediately. This makes it safe to call `open` multiple times on + an instance. + + * If the `prevent_core` attribute is true, set the resource limits + for the process to prevent any core dump from the process. + + * If the `chroot_directory` attribute is not ``None``, set the + effective root directory of the process to that directory (via + `os.chroot`). + + This allows running the daemon process inside a “chroot gaol” + as a means of limiting the system's exposure to rogue behaviour + by the process. Note that the specified directory needs to + already be set up for this purpose. + + * Set the process UID and GID to the `uid` and `gid` attribute + values. + + * Close all open file descriptors. This excludes those listed in + the `files_preserve` attribute, and those that correspond to the + `stdin`, `stdout`, or `stderr` attributes. + + * Change current working directory to the path specified by the + `working_directory` attribute. + + * Reset the file access creation mask to the value specified by + the `umask` attribute. + + * If the `detach_process` option is true, detach the current + process into its own process group, and disassociate from any + controlling terminal. + + * Set signal handlers as specified by the `signal_map` attribute. + + * If any of the attributes `stdin`, `stdout`, `stderr` are not + ``None``, bind the system streams `sys.stdin`, `sys.stdout`, + and/or `sys.stderr` to the files represented by the + corresponding attributes. Where the attribute has a file + descriptor, the descriptor is duplicated (instead of re-binding + the name). + + * If the `pidfile` attribute is not ``None``, enter its context + manager. + + * Mark this instance as open (for the purpose of future `open` and + `close` calls). + + * Register the `close` method to be called during Python's exit + processing. + + When the function returns, the running program is a daemon + process. + + """ + if self.is_open: + return + + if self.chroot_directory is not None: + change_root_directory(self.chroot_directory) + + if self.prevent_core: + prevent_core_dump() + + change_file_creation_mask(self.umask) + change_working_directory(self.working_directory) + change_process_owner(self.uid, self.gid) + + if self.detach_process: + detach_process_context() + + signal_handler_map = self._make_signal_handler_map() + set_signal_handlers(signal_handler_map) + + exclude_fds = self._get_exclude_file_descriptors() + close_all_open_files(exclude=exclude_fds) + + redirect_stream(sys.stdin, self.stdin) + redirect_stream(sys.stdout, self.stdout) + redirect_stream(sys.stderr, self.stderr) + + if self.pidfile is not None: + self.pidfile.__enter__() + + self._is_open = True + + register_atexit_function(self.close) + + def __enter__(self): + """ Context manager entry point. """ + self.open() + return self + + def close(self): + """ Exit the daemon process context. + :Return: ``None`` + + Close the daemon context. This performs the following steps: + + * If this instance's `is_open` property is false, return + immediately. This makes it safe to call `close` multiple times + on an instance. + + * If the `pidfile` attribute is not ``None``, exit its context + manager. + + * Mark this instance as closed (for the purpose of future `open` + and `close` calls). + + """ + if not self.is_open: + return + + if self.pidfile is not None: + # Follow the interface for telling a context manager to exit, + # <URL:http://docs.python.org/library/stdtypes.html#typecontextmanager>. + self.pidfile.__exit__(None, None, None) + + self._is_open = False + + def __exit__(self, exc_type, exc_value, traceback): + """ Context manager exit point. """ + self.close() + + def terminate(self, signal_number, stack_frame): + """ Signal handler for end-process signals. + :Return: ``None`` + + Signal handler for the ``signal.SIGTERM`` signal. Performs the + following step: + + * Raise a ``SystemExit`` exception explaining the signal. + + """ + exception = SystemExit( + "Terminating on signal %(signal_number)r" % vars()) + raise exception + + def _get_exclude_file_descriptors(self): + """ Return the set of file descriptors to exclude closing. + + Returns a set containing the file descriptors for the + items in `files_preserve`, and also each of `stdin`, + `stdout`, and `stderr`: + + * If the item is ``None``, it is omitted from the return + set. + + * If the item has a ``fileno()`` method, that method's + return value is in the return set. + + * Otherwise, the item is in the return set verbatim. + + """ + files_preserve = self.files_preserve + if files_preserve is None: + files_preserve = [] + files_preserve.extend( + item for item in [self.stdin, self.stdout, self.stderr] + if hasattr(item, 'fileno')) + exclude_descriptors = set() + for item in files_preserve: + if item is None: + continue + if hasattr(item, 'fileno'): + exclude_descriptors.add(item.fileno()) + else: + exclude_descriptors.add(item) + return exclude_descriptors + + def _make_signal_handler(self, target): + """ Make the signal handler for a specified target object. + + If `target` is ``None``, returns ``signal.SIG_IGN``. If + `target` is a string, returns the attribute of this + instance named by that string. Otherwise, returns `target` + itself. + + """ + if target is None: + result = signal.SIG_IGN + elif isinstance(target, basestring): + name = target + result = getattr(self, name) + else: + result = target + + return result + + def _make_signal_handler_map(self): + """ Make the map from signals to handlers for this instance. + + Constructs a map from signal numbers to handlers for this + context instance, suitable for passing to + `set_signal_handlers`. + + """ + signal_handler_map = dict( + (signal_number, self._make_signal_handler(target)) + for (signal_number, target) in self.signal_map.items()) + return signal_handler_map + + +def change_working_directory(directory): + """ Change the working directory of this process. + """ + try: + os.chdir(directory) + except Exception, exc: + error = DaemonOSEnvironmentError( + "Unable to change working directory (%(exc)s)" + % vars()) + raise error + + +def change_root_directory(directory): + """ Change the root directory of this process. + + Sets the current working directory, then the process root + directory, to the specified `directory`. Requires appropriate + OS privileges for this process. + + """ + try: + os.chdir(directory) + os.chroot(directory) + except Exception, exc: + error = DaemonOSEnvironmentError( + "Unable to change root directory (%(exc)s)" + % vars()) + raise error + + +def change_file_creation_mask(mask): + """ Change the file creation mask for this process. + """ + try: + os.umask(mask) + except Exception, exc: + error = DaemonOSEnvironmentError( + "Unable to change file creation mask (%(exc)s)" + % vars()) + raise error + + +def change_process_owner(uid, gid): + """ Change the owning UID and GID of this process. + + Sets the GID then the UID of the process (in that order, to + avoid permission errors) to the specified `gid` and `uid` + values. Requires appropriate OS privileges for this process. + + """ + try: + os.setgid(gid) + os.setuid(uid) + except Exception, exc: + error = DaemonOSEnvironmentError( + "Unable to change file creation mask (%(exc)s)" + % vars()) + raise error + + +def prevent_core_dump(): + """ Prevent this process from generating a core dump. + + Sets the soft and hard limits for core dump size to zero. On + Unix, this prevents the process from creating core dump + altogether. + + """ + core_resource = resource.RLIMIT_CORE + + try: + # Ensure the resource limit exists on this platform, by requesting + # its current value + core_limit_prev = resource.getrlimit(core_resource) + except ValueError, exc: + error = DaemonOSEnvironmentError( + "System does not support RLIMIT_CORE resource limit (%(exc)s)" + % vars()) + raise error + + # Set hard and soft limits to zero, i.e. no core dump at all + core_limit = (0, 0) + resource.setrlimit(core_resource, core_limit) + + +def detach_process_context(): + """ Detach the process context from parent and session. + + Detach from the parent process and session group, allowing the + parent to exit while this process continues running. + + Reference: “Advanced Programming in the Unix Environment”, + section 13.3, by W. Richard Stevens, published 1993 by + Addison-Wesley. + + """ + + def fork_then_exit_parent(error_message): + """ Fork a child process, then exit the parent process. + + If the fork fails, raise a ``DaemonProcessDetachError`` + with ``error_message``. + + """ + try: + pid = os.fork() + if pid > 0: + os._exit(0) + except OSError, exc: + exc_errno = exc.errno + exc_strerror = exc.strerror + error = DaemonProcessDetachError( + "%(error_message)s: [%(exc_errno)d] %(exc_strerror)s" % vars()) + raise error + + fork_then_exit_parent(error_message="Failed first fork") + os.setsid() + fork_then_exit_parent(error_message="Failed second fork") + + +def is_process_started_by_init(): + """ Determine if the current process is started by `init`. + + The `init` process has the process ID of 1; if that is our + parent process ID, return ``True``, otherwise ``False``. + + """ + result = False + + init_pid = 1 + if os.getppid() == init_pid: + result = True + + return result + + +def is_socket(fd): + """ Determine if the file descriptor is a socket. + + Return ``False`` if querying the socket type of `fd` raises an + error; otherwise return ``True``. + + """ + result = False + + file_socket = socket.fromfd(fd, socket.AF_INET, socket.SOCK_RAW) + + try: + socket_type = file_socket.getsockopt( + socket.SOL_SOCKET, socket.SO_TYPE) + except socket.error, exc: + exc_errno = exc.args[0] + if exc_errno == errno.ENOTSOCK: + # Socket operation on non-socket + pass + else: + # Some other socket error + result = True + else: + # No error getting socket type + result = True + + return result + + +def is_process_started_by_superserver(): + """ Determine if the current process is started by the superserver. + + The internet superserver creates a network socket, and + attaches it to the standard streams of the child process. If + that is the case for this process, return ``True``, otherwise + ``False``. + + """ + result = False + + stdin_fd = sys.__stdin__.fileno() + if is_socket(stdin_fd): + result = True + + return result + + +def is_detach_process_context_required(): + """ Determine whether detaching process context is required. + + Return ``True`` if the process environment indicates the + process is already detached: + + * Process was started by `init`; or + + * Process was started by `inetd`. + + """ + result = True + if is_process_started_by_init() or is_process_started_by_superserver(): + result = False + + return result + + +def close_file_descriptor_if_open(fd): + """ Close a file descriptor if already open. + + Close the file descriptor `fd`, suppressing an error in the + case the file was not open. + + """ + try: + os.close(fd) + except OSError, exc: + if exc.errno == errno.EBADF: + # File descriptor was not open + pass + else: + error = DaemonOSEnvironmentError( + "Failed to close file descriptor %(fd)d" + " (%(exc)s)" + % vars()) + raise error + + +MAXFD = 2048 + + +def get_maximum_file_descriptors(): + """ Return the maximum number of open file descriptors for this process. + + Return the process hard resource limit of maximum number of + open file descriptors. If the limit is “infinity”, a default + value of ``MAXFD`` is returned. + + """ + limits = resource.getrlimit(resource.RLIMIT_NOFILE) + result = limits[1] + if result == resource.RLIM_INFINITY: + result = MAXFD + return result + + +def close_all_open_files(exclude=set()): + """ Close all open file descriptors. + + Closes every file descriptor (if open) of this process. If + specified, `exclude` is a set of file descriptors to *not* + close. + + """ + maxfd = get_maximum_file_descriptors() + for fd in reversed(range(maxfd)): + if fd not in exclude: + close_file_descriptor_if_open(fd) + + +def redirect_stream(system_stream, target_stream): + """ Redirect a system stream to a specified file. + + `system_stream` is a standard system stream such as + ``sys.stdout``. `target_stream` is an open file object that + should replace the corresponding system stream object. + + If `target_stream` is ``None``, defaults to opening the + operating system's null device and using its file descriptor. + + """ + if target_stream is None: + target_fd = os.open(os.devnull, os.O_RDWR) + else: + target_fd = target_stream.fileno() + os.dup2(target_fd, system_stream.fileno()) + + +def make_default_signal_map(): + """ Make the default signal map for this system. + + The signals available differ by system. The map will not + contain any signals not defined on the running system. + + """ + name_map = { + 'SIGTSTP': None, + 'SIGTTIN': None, + 'SIGTTOU': None, + 'SIGTERM': 'terminate', + } + signal_map = dict( + (getattr(signal, name), target) + for (name, target) in name_map.items() + if hasattr(signal, name)) + + return signal_map + + +def set_signal_handlers(signal_handler_map): + """ Set the signal handlers as specified. + + The `signal_handler_map` argument is a map from signal number + to signal handler. See the `signal` module for details. + + """ + for (signal_number, handler) in signal_handler_map.items(): + signal.signal(signal_number, handler) + + +def register_atexit_function(func): + """ Register a function for processing at program exit. + + The function `func` is registered for a call with no arguments + at program exit. + + """ + atexit.register(func) diff --git a/log.py b/log.py index e415001..76052e2 100644 --- a/log.py +++ b/log.py @@ -24,6 +24,7 @@ import logging.handlers import os import sys + def getLogger(debug, background): logger = logging.getLogger("rhsm-app") logger.setLevel(logging.DEBUG) @@ -42,7 +43,7 @@ def getLogger(debug, background): if debug: fileHandler.setLevel(logging.DEBUG) else: - fileHandler.setLevel(logging.WARNING) + fileHandler.setLevel(logging.INFO) logger.addHandler(fileHandler) except Exception, e: sys.stderr.write("Unable to log to %s: %s\n" % (path, e)) @@ -53,7 +54,7 @@ def getLogger(debug, background): if debug: streamHandler.setLevel(logging.DEBUG) else: - streamHandler.setLevel(logging.WARNING) + streamHandler.setLevel(logging.INFO) # Don't print exceptions to stdout in non-debug mode f = logging.Filter() diff --git a/manager/__init__.py b/manager/__init__.py new file mode 100644 index 0000000..31406e7 --- /dev/null +++ b/manager/__init__.py @@ -0,0 +1,2 @@ + +from manager import Manager, ManagerError diff --git a/manager/manager.py b/manager/manager.py new file mode 100644 index 0000000..ed104c2 --- /dev/null +++ b/manager/manager.py @@ -0,0 +1,24 @@ + + +class ManagerError(Exception): + pass + + +class Manager(object): + def sendVirtGuests(self, domains): + raise NotImplementedError() + + def hypervisorCheckIn(self, owner, env, mapping, type=None): + raise NotImplementedError() + + @classmethod + def fromOptions(cls, logger, options): + # Imports can't be top-level, it would be circular dependency + import subscriptionmanager + import satellite + + for subcls in cls.__subclasses__(): + if subcls.smType == options.smType: + return subcls(logger, options) + + raise KeyError("Invalid config type: %s" % options.smType) diff --git a/manager/satellite/__init__.py b/manager/satellite/__init__.py new file mode 100644 index 0000000..23b7650 --- /dev/null +++ b/manager/satellite/__init__.py @@ -0,0 +1,2 @@ + +from satellite import Satellite, SatelliteError diff --git a/satellite.py b/manager/satellite/satellite.py similarity index 79% rename from satellite.py rename to manager/satellite/satellite.py index 3c0a195..c1d6484 100644 --- a/satellite.py +++ b/manager/satellite/satellite.py @@ -18,37 +18,46 @@ along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. """ -import sys -import os import xmlrpclib import pickle -class SatelliteError(Exception): +from manager import Manager, ManagerError + + +class SatelliteError(ManagerError): def __init__(self, message): self.message = message def __str__(self): return self.message -class Satellite(object): + +class Satellite(Manager): + smType = "satellite" """ Class for interacting with satellite (RHN Classic). """ - HYPERVISOR_SYSTEMID_FILE="/var/lib/virt-who/hypervisor-systemid-%s" - def __init__(self, logger): + HYPERVISOR_SYSTEMID_FILE = "/var/lib/virt-who/hypervisor-systemid-%s" + + def __init__(self, logger, options): self.logger = logger self.server = None + self.options = options - def connect(self, server, username, password, options=None, force_register=False): - if not server.startswith("http://") and not server.startswith("https://"): - server = "https://%s" % server - if not server.endswith("XMLRPC"): - server = "%s/XMLRPC" % server + def _connect(self): + if not self.options.server.startswith("http://") and not self.options.server.startswith("https://"): + self.options.server = "https://%s" % self.options.server + if not self.options.server.endswith("XMLRPC"): + self.options.server = "%s/XMLRPC" % self.options.server - self.username = username - self.password = password + self.username = self.options.username + self.password = self.options.password + try: + self.force_register = self.options.force_register + except AttributeError: + self.force_register = False - self.logger.debug("Initializing satellite connection to %s" % server) + self.logger.debug("Initializing satellite connection to %s" % self.options.server) try: - self.server = xmlrpclib.Server(server, verbose=0) + self.server = xmlrpclib.Server(self.options.server, verbose=0) except Exception: self.logger.exception("Unable to connect to the Satellite server") raise SatelliteError("Unable to connect to the Satellite server") @@ -58,14 +67,17 @@ class Satellite(object): systemid_filename = self.HYPERVISOR_SYSTEMID_FILE % hypervisor_uuid # attempt to read the existing systemid file for the hypervisor try: + if self.force_register: + raise IOError() self.logger.debug("Loading system id info from %s" % systemid_filename) new_system = pickle.load(open(systemid_filename, "rb")) except IOError: # assume file was not found, create a new hypervisor try: # TODO: what to do here? 6Server will consume subscription - new_system = self.server.registration.new_system_user_pass("%s hypervisor %s" % (type, hypervisor_uuid), - "unknown", "6Server", "x86_64", self.username, self.password, {}) + new_system = self.server.registration.new_system_user_pass( + "%s hypervisor %s" % (type, hypervisor_uuid), + "unknown", "6Server", "x86_64", self.username, self.password, {}) self.server.registration.refresh_hw_profile(new_system['system_id'], []) except Exception, e: self.logger.exception("Unable to refresh HW profile") @@ -95,17 +107,15 @@ class Satellite(object): def _assemble_plan(self, hypervisor_mapping, hypervisor_uuid, type): - # Get rid of dashes from UUID, spacewalk does not like them - #hypervisor_uuid = (str(hypervisor_uuid).replace("-", "")) events = [] # the stub_instance_info is not used by the report. When the guest system checks in, it will provide # actual hardware info stub_instance_info = { - 'vcpus' : 1, - 'memory_size' : 0, - 'virt_type' : 'fully_virtualized', - 'state' : 'running', + 'vcpus': 1, + 'memory_size': 0, + 'virt_type': 'fully_virtualized', + 'state': 'running', } # again, remove dashes @@ -113,7 +123,6 @@ class Satellite(object): for g_uuid in hypervisor_mapping: guest_uuids.append(str(g_uuid).replace("-", "")) - # TODO: spacewalk wants all zeroes for the hypervisor uuid?? events.append([0, 'exists', 'system', {'identity': 'host', 'uuid': '0000000000000000'}]) @@ -131,13 +140,15 @@ class Satellite(object): raise SatelliteError("virt-who does not support sending local hypervisor data to satellite; use rhn-virtualization-host instead") def hypervisorCheckIn(self, owner, env, mapping, type=None): + self._connect() + self.logger.info("Sending update in hosts-to-guests mapping: %s" % mapping) if len(mapping) == 0: self.logger.info("no hypervisors found, not sending data to satellite") for hypervisor_uuid, guest_uuids in mapping.items(): self.logger.debug("Loading systemid for %s" % hypervisor_uuid) - hypervisor_systemid = self._load_hypervisor(hypervisor_uuid, type=type) + hypervisor_systemid = self._load_hypervisor(hypervisor_uuid, type=type) self.logger.debug("Building plan for hypervisor %s: %s" % (hypervisor_uuid, guest_uuids)) plan = self._assemble_plan(guest_uuids, hypervisor_uuid, type=type) diff --git a/manager/subscriptionmanager/__init__.py b/manager/subscriptionmanager/__init__.py new file mode 100644 index 0000000..771d9b9 --- /dev/null +++ b/manager/subscriptionmanager/__init__.py @@ -0,0 +1,2 @@ + +from subscriptionmanager import SubscriptionManager diff --git a/subscriptionmanager.py b/manager/subscriptionmanager/subscriptionmanager.py similarity index 80% rename from subscriptionmanager.py rename to manager/subscriptionmanager/subscriptionmanager.py index 0045d38..3b6ce5a 100644 --- a/subscriptionmanager.py +++ b/manager/subscriptionmanager/subscriptionmanager.py @@ -19,12 +19,14 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. """ import os -import sys import rhsm.connection as rhsm_connection import rhsm.certificate as rhsm_certificate import rhsm.config as rhsm_config +from ..manager import Manager + + class SubscriptionManagerError(Exception): def __init__(self, message): self.message = message @@ -32,10 +34,14 @@ class SubscriptionManagerError(Exception): def __str__(self): return self.message -class SubscriptionManager: + +class SubscriptionManager(Manager): + smType = "sam" + """ Class for interacting subscription-manager. """ - def __init__(self, logger): + def __init__(self, logger, options): self.logger = logger + self.options = options self.cert_uuid = None self.config = rhsm_config.initConfig(rhsm_config.DEFAULT_CONFIG_PATH) @@ -55,17 +61,17 @@ class SubscriptionManager: if not os.access(self.cert_file, os.R_OK): raise SubscriptionManagerError("Unable to read certificate, system is not registered or you are not root") - def connect(self, Connection=rhsm_connection.UEPConnection): + def _connect(self): """ Connect to the subscription-manager. """ - self.connection = Connection( - host=self.config.get('server', 'hostname'), - ssl_port=int(self.config.get('server', 'port')), - handler=self.config.get('server', 'prefix'), - proxy_hostname=self.config.get('server', 'proxy_hostname'), - proxy_port=self.config.get('server', 'proxy_port'), - proxy_user=self.config.get('server', 'proxy_user'), - proxy_password=self.config.get('server', 'proxy_password'), - cert_file=self.cert_file, key_file=self.key_file) + self.connection = rhsm_connection.UEPConnection( + host=self.config.get('server', 'hostname'), + ssl_port=int(self.config.get('server', 'port')), + handler=self.config.get('server', 'prefix'), + proxy_hostname=self.config.get('server', 'proxy_hostname'), + proxy_port=self.config.get('server', 'proxy_port'), + proxy_user=self.config.get('server', 'proxy_user'), + proxy_password=self.config.get('server', 'proxy_password'), + cert_file=self.cert_file, key_file=self.key_file) if not self.connection.ping()['result']: raise SubscriptionManagerError("Unable to obtain status from server, UEPConnection is likely not usable.") @@ -88,6 +94,8 @@ class SubscriptionManager: :type domain: list of str or list of dict domains """ + self._connect() + # Sort the list key = None if len(domains) > 0: @@ -96,17 +104,18 @@ class SubscriptionManager: domains.sort(key=key) if key is not None: - self.logger.debug("Sending list of uuids: %s" % [domain[key] for domain in domains]) + self.logger.info("Sending list of uuids: %s" % [domain[key] for domain in domains]) else: - self.logger.debug("Sending list of uuids: %s" % domains) + self.logger.info("Sending list of uuids: %s" % domains) # Send list of guest uuids to the server self.connection.updateConsumer(self.uuid(), guest_uuids=domains) def hypervisorCheckIn(self, owner, env, mapping, type=None): """ Send hosts to guests mapping to subscription manager. """ + self.logger.info("Sending update in hosts-to-guests mapping: %s" % mapping) - self.logger.debug("Sending update in hosts-to-guests mapping: %s" % mapping) + self._connect() # Send the mapping return self.connection.hypervisorCheckIn(owner, env, mapping) diff --git a/virt-who b/virt-who index 959997b..f2ca438 100755 --- a/virt-who +++ b/virt-who @@ -1,10 +1,10 @@ #!/bin/sh -if [ -f ./virt-who.py ]; +if [ -f ./virtwho.py ]; then # Run it from local directory when available - exec /usr/bin/python ./virt-who.py "$@" + exec /usr/bin/python ./virtwho.py "$@" else # Run it from /usr/share/virt-who - exec /usr/bin/python /usr/share/virt-who/virt-who.py "$@" + exec /usr/bin/python /usr/share/virt-who/virtwho.py "$@" fi diff --git a/virt-who-config.5 b/virt-who-config.5 new file mode 100644 index 0000000..a055912 --- /dev/null +++ b/virt-who-config.5 @@ -0,0 +1,50 @@ +.TH VIRT-WHO-CONFIG "5" "June 2014" "virt-who" +.SH NAME +virt-who-config - configuration for virt-who +.SH SYNOPISIS +/etc/virt-who.d/* +.SH DESCRIPTION +Configuration format is ini-like, where group name (in square brackets) is arbitrary name of the configuration. + +Only required key is \fBtype\fR that has to have one of the allowed virtualization backend names, see virt-who(8). + +Another options that could be supplied are: +.TP +\fBserver\fR +hostname, IP address or URL of the server that provides virtualization information (not applicable for libvirt and vdsm mode). +.TP +\fBusername\fR +username for authentication to the server (not applicable for libvirt and vdsm mode). +.TP +\fBpassword\fR +password for authentication to the server (not applicable for libvirt and vdsm mode). +.TP +\fBencrypted_password\fR +encrypted password that is generated by virt-who-password(8) utility +.TP +\fBowner\fR +owner for use with Subscription Asset Manager (not applicable for Satellite) +.TP +\fBenv\fR +environment for use with Subscription Asset Manager (not applicable for Satellite) + +.SH EXAMPLE +[test-esx] +.br +type=esx +.br +server=1.2.3.4 +.br +username=admin +.br +password=password +.br +owner=test +.br +env=staging + +.SH AUTHOR +Radek Novacek <rnovacek at redhat dot com> + +.SH SEE ALSO +virt-who(8), virt-who-password(8) diff --git a/virt-who.spec b/virt-who.spec index a5ef0b6..81619fb 100644 --- a/virt-who.spec +++ b/virt-who.spec @@ -39,6 +39,8 @@ rm -rf $RPM_BUILD_ROOT make DESTDIR=$RPM_BUILD_ROOT install mkdir -p %{buildroot}/%{_sharedstatedir}/%{name} +mkdir -p %{buildroot}/%{_sysconfdir}/virt-who.d +touch %{buildroot}/%{_sharedstatedir}/%{name}/key # Don't run test suite in check section, because it need the system to be # registered to subscription-manager server @@ -65,11 +67,16 @@ fi %files %doc README LICENSE %{_bindir}/virt-who +%{_bindir}/virt-who-password %{_datadir}/virt-who/ %{_sysconfdir}/rc.d/init.d/virt-who +%attr(600, root, root) %dir %{_sysconfdir}/virt-who.d %attr(600, root, root) %config(noreplace) %{_sysconfdir}/sysconfig/virt-who %{_mandir}/man8/virt-who.8.gz -%{_sharedstatedir}/%{name} +%{_mandir}/man8/virt-who-password.8.gz +%{_mandir}/man5/virt-who-config.5.gz +%attr(600, root, root) %{_sharedstatedir}/%{name} +%ghost %{_sharedstatedir}/%{name}/key %changelog diff --git a/virt/__init__.py b/virt/__init__.py new file mode 100644 index 0000000..aebc492 --- /dev/null +++ b/virt/__init__.py @@ -0,0 +1,3 @@ + + +from virt import Virt, DirectVirt, HypervisorVirt, VirtError, Domain diff --git a/virt/esx/__init__.py b/virt/esx/__init__.py new file mode 100644 index 0000000..87db45d --- /dev/null +++ b/virt/esx/__init__.py @@ -0,0 +1,2 @@ + +from esx import Esx diff --git a/vsphere.py b/virt/esx/esx.py similarity index 90% rename from vsphere.py rename to virt/esx/esx.py index 20febd4..7330299 100644 --- a/vsphere.py +++ b/virt/esx/esx.py @@ -20,6 +20,10 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. import sys import suds +from urllib2 import URLError + +import virt + def get_search_filter_spec(client, begin_entity, property_spec): """ Build a PropertyFilterSpec capable of full inventory traversal. @@ -104,15 +108,17 @@ def get_search_filter_spec(client, begin_entity, property_spec): return pfs -class VSphere: - def __init__(self, logger, url, username, password): +class Esx(virt.HypervisorVirt): + CONFIG_TYPE = "esx" + + def __init__(self, logger, config): self.logger = logger - self.url = url - self.username = username - self.password = password + self.url = config.server + self.username = config.username + self.password = config.password # Url must contain protocol (usualy https://) - if not "://" in self.url: + if "://" not in self.url: self.url = "https://%s" % self.url self.clusters = {} @@ -121,12 +127,16 @@ class VSphere: def scan(self): """ - Scan method does full inventory traversal on the vCenter machine. It finds - all ComputeResources, Hosts and VirtualMachines. + Scan method does full inventory traversal on the vCenter machine. + It finds all ComputeResources, Hosts and VirtualMachines. """ # Connect to the vCenter server - self.client = suds.client.Client("%s/sdk/vimService.wsdl" % self.url) + try: + self.client = suds.client.Client("%s/sdk/vimService.wsdl" % self.url) + except URLError, e: + self.logger.exception("Unable to connect to ESX") + raise virt.VirtError(str(e)) self.client.set_options(location="%s/sdk" % self.url) @@ -138,7 +148,11 @@ class VSphere: self.sc = self.client.service.RetrieveServiceContent(_this=self.moRef) # Login to server using given credentials - self.client.service.Login(_this=self.sc.sessionManager, userName=self.username, password=self.password) + try: + self.client.service.Login(_this=self.sc.sessionManager, userName=self.username, password=self.password) + except suds.WebFault, e: + self.logger.exception("Unable to login to ESX") + raise virt.VirtError(str(e)) # Clear results from last run self.clusters = {} @@ -151,15 +165,17 @@ class VSphere: ts.pathSet = 'name' ts.all = True try: - retrieve_result = self.client.service.RetrievePropertiesEx(_this=self.sc.propertyCollector, - specSet=[get_search_filter_spec(self.client, self.sc.rootFolder, [ts])]) + retrieve_result = self.client.service.RetrievePropertiesEx( + _this=self.sc.propertyCollector, + specSet=[get_search_filter_spec(self.client, self.sc.rootFolder, [ts])]) if retrieve_result is None: object_content = [] else: object_content = retrieve_result[0] except suds.MethodNotFound: - object_content = self.client.service.RetrieveProperties(_this=self.sc.propertyCollector, - specSet=[get_search_filter_spec(self.client, self.sc.rootFolder, [ts])]) + object_content = self.client.service.RetrieveProperties( + _this=self.sc.propertyCollector, + specSet=[get_search_filter_spec(self.client, self.sc.rootFolder, [ts])]) # Get properties of each cluster clusterObjs = [] # List of objs for 'ComputeResource' query @@ -233,7 +249,6 @@ class VSphere: # This means that there is no guest on given host pass - def ping(self): return True @@ -311,16 +326,19 @@ class VSphere: for vm in host.vms: print "\t\tVirtualMachine: %s" % vm.uuid + class Cluster: def __init__(self, name): self.name = name self.hosts = [] + class Host: def __init__(self): self.uuid = None self.vms = [] + class VM: def __init__(self): self.uuid = None @@ -333,6 +351,8 @@ if __name__ == '__main__': import logging logger = logging.Logger("") - vsphere = VSphere(logger, sys.argv[1], sys.argv[2], sys.argv[3]) + from config import Config + config = Config('esx', 'esx', sys.argv[1], sys.argv[2], sys.argv[3]) + vsphere = Esx(logger, config) vsphere.scan() vsphere.printLayout() diff --git a/virt/hyperv/__init__.py b/virt/hyperv/__init__.py new file mode 100644 index 0000000..28ff934 --- /dev/null +++ b/virt/hyperv/__init__.py @@ -0,0 +1,2 @@ + +from hyperv import HyperV diff --git a/hyperv.py b/virt/hyperv/hyperv.py similarity index 86% rename from hyperv.py rename to virt/hyperv/hyperv.py index 189366e..8b48c46 100644 --- a/hyperv.py +++ b/virt/hyperv/hyperv.py @@ -1,13 +1,35 @@ +""" +Module for communcating with Hyper-V, part of virt-who + +Copyright (C) 2014 Radek Novacek <rnovacek(a)redhat.com> + +This program is free software; you can redistribute it and/or +modify it under the terms of the GNU General Public License +as published by the Free Software Foundation; either version 2 +of the License, or (at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +""" import sys import httplib import urlparse import base64 +import virt + try: from uuid import uuid1 except ImportError: import subprocess + def uuid1(): # fallback to calling commandline uuidgen return subprocess.Popen(["uuidgen"], stdout=subprocess.PIPE).communicate()[0].strip() @@ -21,10 +43,10 @@ except ImportError: import ntlm NAMESPACES = { - 's': 'http://www.w3.org/2003/05/soap-envelope', - 'wsa': 'http://schemas.xmlsoap.org/ws/2004/08/addressing', + 's': 'http://www.w3.org/2003/05/soap-envelope', + 'wsa': 'http://schemas.xmlsoap.org/ws/2004/08/addressing', 'wsman': 'http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd', - 'wsen': 'http://schemas.xmlsoap.org/ws/2004/09/enumeration' + 'wsen': 'http://schemas.xmlsoap.org/ws/2004/09/enumeration' } ENVELOPE = """<?xml version="1.0" encoding="UTF-8"?> @@ -33,6 +55,7 @@ ENVELOPE = """<?xml version="1.0" encoding="UTF-8"?> %s </s:Envelope>""" + def getHeader(action): return """<s:Header> <wsa:Action s:mustUnderstand="true">""" + NAMESPACES['wsen'] + "/" + action + """</wsa:Action> @@ -44,21 +67,25 @@ def getHeader(action): </wsa:ReplyTo> </s:Header>""" + ENUMERATE_BODY = """<s:Body> <wsen:Enumerate> <wsman:Filter Dialect="http://schemas.microsoft.com/wbem/wsman/1/WQL">%(query)s</wsman:Filter> </wsen:Enumerate> </s:Body>""" + PULL_BODY = """<s:Body> <wsen:Pull> <wsen:EnumerationContext>%(EnumerationContext)s</wsen:EnumerationContext> </wsen:Pull> </s:Body>""" + ENUMERATE_XML = ENVELOPE % (getHeader("Enumerate"), ENUMERATE_BODY) PULL_XML = ENVELOPE % (getHeader("Pull"), PULL_BODY) + class HyperVSoap(object): def __init__(self, url, connection, headers): self.url = url @@ -92,7 +119,7 @@ class HyperVSoap(object): return properties def Enumerate(self, query, namespace="root/virtualization"): - data = ENUMERATE_XML % { 'url': self.url, 'query': query, 'namespace': namespace } + data = ENUMERATE_XML % {'url': self.url, 'query': query, 'namespace': namespace} response = self.post(data) d = response.read() xml = ElementTree.fromstring(d) @@ -110,7 +137,7 @@ class HyperVSoap(object): return contexts[0].text def _PullOne(self, uuid, namespace): - data = PULL_XML % { 'url': self.url, 'EnumerationContext': uuid, 'namespace': namespace } + data = PULL_XML % {'url': self.url, 'EnumerationContext': uuid, 'namespace': namespace} response = self.post(data) d = response.read() xml = ElementTree.fromstring(d) @@ -140,18 +167,22 @@ class HyperVSoap(object): return instances -class HyperVException(Exception): +class HyperVException(virt.VirtError): pass + class HyperVAuthFailed(HyperVException): pass -class HyperV: - def __init__(self, logger, url, username, password): +class HyperV(virt.HypervisorVirt): + CONFIG_TYPE = "hyperv" + + def __init__(self, logger, config): self.logger = logger - self.username = username - self.password = password + url = config.server + self.username = config.username + self.password = config.password # First try to use old API (root/virtualization namespace) if doesn't # work, go with root/virtualization/v2 @@ -177,7 +208,7 @@ class HyperV: logger.debug("Hyper-V url: %s" % self.url) # Check if we have domain defined and set flags accordingly - user_parts = username.split('\\', 1) + user_parts = self.username.split('\\', 1) if len(user_parts) == 1: self.username = user_parts[0] self.domainname = '' @@ -285,8 +316,10 @@ class HyperV: else: # Filter out Planned VMs and snapshots, see # http://msdn.microsoft.com/en-us/library/hh850257%28v=vs.85%29.aspx - uuid = hypervsoap.Enumerate("select BIOSGUID from Msvm_VirtualSystemSettingData " - "where VirtualSystemType = 'Microsoft:Hyper-V:System:Realized'", "root/virtualization/v2") + uuid = hypervsoap.Enumerate( + "select BIOSGUID from Msvm_VirtualSystemSettingData " + "where VirtualSystemType = 'Microsoft:Hyper-V:System:Realized'", + "root/virtualization/v2") except HyperVException, e: if not self.useNewApi: self.logger.debug("Error when enumerating using root/virtualization namespace, trying root/virtualization/v2 namespace") @@ -300,7 +333,7 @@ class HyperV: host = None for instance in hypervsoap.Pull(uuid, "root/cimv2"): host = HyperV.decodeWinUUID(instance["UUID"]) - return { host: guests } + return {host: guests} def ping(self): return True diff --git a/ntlm.py b/virt/hyperv/ntlm.py similarity index 74% rename from ntlm.py rename to virt/hyperv/ntlm.py index 32d33c3..c8d9c36 100644 --- a/ntlm.py +++ b/virt/hyperv/ntlm.py @@ -29,16 +29,20 @@ from socket import gethostname import M2Crypto + # Use md4 from hashlib or directly from openssl class OpenSslMd4(object): def __init__(self, data): self.data = data + def digest(self): return subprocess.Popen(["openssl", "md4", "-binary"], stdin=subprocess.PIPE, stdout=subprocess.PIPE).communicate(self.data)[0] + class HashlibMd4(object): def __init__(self, data): self.md4 = hashlib.new('md4', data) + def digest(self): return self.md4.digest() @@ -47,6 +51,7 @@ if 'md4' in algorithms: else: md4 = OpenSslMd4 + # DES handling functions def des_encrypt(key_str, plain_text): @@ -55,36 +60,41 @@ def des_encrypt(key_str, plain_text): key_str = '' for i in k: key_str += chr(i & 0xFF) - des = M2Crypto.EVP.Cipher("des_ecb", key=key_str, op=M2Crypto.encrypt, iv='\0'*16) + des = M2Crypto.EVP.Cipher("des_ecb", key=key_str, op=M2Crypto.encrypt, iv='\0' * 16) return des.update(plain_text) + def str_to_key56(key_str): if len(key_str) < 7: key_str = key_str + '\000\000\000\000\000\000\000'[:(7 - len(key_str))] key_56 = [] - for i in key_str[:7]: key_56.append(ord(i)) + for i in key_str[:7]: + key_56.append(ord(i)) return key_56 + def key56_to_key64(key_56): "" key = [] - for i in range(8): key.append(0) - - key[0] = key_56[0]; - key[1] = ((key_56[0] << 7) & 0xFF) | (key_56[1] >> 1); - key[2] = ((key_56[1] << 6) & 0xFF) | (key_56[2] >> 2); - key[3] = ((key_56[2] << 5) & 0xFF) | (key_56[3] >> 3); - key[4] = ((key_56[3] << 4) & 0xFF) | (key_56[4] >> 4); - key[5] = ((key_56[4] << 3) & 0xFF) | (key_56[5] >> 5); - key[6] = ((key_56[5] << 2) & 0xFF) | (key_56[6] >> 6); - key[7] = (key_56[6] << 1) & 0xFF; + for i in range(8): + key.append(0) + + key[0] = key_56[0] + key[1] = ((key_56[0] << 7) & 0xFF) | (key_56[1] >> 1) + key[2] = ((key_56[1] << 6) & 0xFF) | (key_56[2] >> 2) + key[3] = ((key_56[2] << 5) & 0xFF) | (key_56[3] >> 3) + key[4] = ((key_56[3] << 4) & 0xFF) | (key_56[4] >> 4) + key[5] = ((key_56[4] << 3) & 0xFF) | (key_56[5] >> 5) + key[6] = ((key_56[5] << 2) & 0xFF) | (key_56[6] >> 6) + key[7] = (key_56[6] << 1) & 0xFF key = set_key_odd_parity(key) return key + def set_key_odd_parity(key): "" for i in range(len(key)): @@ -132,25 +142,25 @@ NTLM_NegotiateKeyExchange = 0x40000000 NTLM_Negotiate56 = 0x80000000 # we send these flags with our type 1 message -NTLM_TYPE1_FLAGS = (NTLM_NegotiateUnicode | \ - NTLM_NegotiateOEM | \ - NTLM_RequestTarget | \ - NTLM_NegotiateNTLM | \ - NTLM_NegotiateOemDomainSupplied | \ - NTLM_NegotiateOemWorkstationSupplied | \ - NTLM_NegotiateAlwaysSign | \ - NTLM_NegotiateExtendedSecurity | \ - NTLM_NegotiateVersion | \ - NTLM_Negotiate128 | \ - NTLM_Negotiate56 ) -NTLM_TYPE2_FLAGS = (NTLM_NegotiateUnicode | \ - NTLM_RequestTarget | \ - NTLM_NegotiateNTLM | \ - NTLM_NegotiateAlwaysSign | \ - NTLM_NegotiateExtendedSecurity | \ - NTLM_NegotiateTargetInfo | \ - NTLM_NegotiateVersion | \ - NTLM_Negotiate128 | \ +NTLM_TYPE1_FLAGS = (NTLM_NegotiateUnicode | + NTLM_NegotiateOEM | + NTLM_RequestTarget | + NTLM_NegotiateNTLM | + NTLM_NegotiateOemDomainSupplied | + NTLM_NegotiateOemWorkstationSupplied | + NTLM_NegotiateAlwaysSign | + NTLM_NegotiateExtendedSecurity | + NTLM_NegotiateVersion | + NTLM_Negotiate128 | + NTLM_Negotiate56) +NTLM_TYPE2_FLAGS = (NTLM_NegotiateUnicode | + NTLM_RequestTarget | + NTLM_NegotiateNTLM | + NTLM_NegotiateAlwaysSign | + NTLM_NegotiateExtendedSecurity | + NTLM_NegotiateTargetInfo | + NTLM_NegotiateVersion | + NTLM_Negotiate128 | NTLM_Negotiate56) NTLM_MsvAvEOL = 0 # Indicates that this is the last AV_PAIR in the list. AvLen MUST be 0. This type of information MUST be present in the AV pair list. @@ -161,7 +171,7 @@ NTLM_MsvAvDnsDomainName = 4 # The server's Active Directory DNS domain name. T NTLM_MsvAvDnsTreeName = 5 # The server's Active Directory (AD) DNS forest tree name. The name MUST be in Unicode, and is not null-terminated. NTLM_MsvAvFlags = 6 # A field containing a 32-bit value indicating server or client configuration. 0x00000001: indicates to the client that the account authentication is constrained. 0x00000002: indicates that the client is providing message integrity in the MIC field (section 2.2.1.3) in the AUTHENTICATE_MESSAGE. NTLM_MsvAvTimestamp = 7 # A FILETIME structure ([MS-DTYP] section 2.3.1) in little-endian byte order that contains the server local time.<12> -NTLM_MsAvRestrictions = 8 #A Restriction_Encoding structure (section 2.2.2.2). The Value field contains a structure representing the integrity level of the security principal, as well as a MachineID created at computer startup to identify the calling machine. <13> +NTLM_MsAvRestrictions = 8 # A Restriction_Encoding structure (section 2.2.2.2). The Value field contains a structure representing the integrity level of the security principal, as well as a MachineID created at computer startup to identify the calling machine. <13> """ @@ -183,6 +193,8 @@ http://sourceforge.net/projects/ntlmaps/ Optimized Attack for NTLM2 Session Response http://www.blackhat.com/presentations/bh-asia-04/bh-jp-04-pdfs/bh-jp-04-s... """ + + def dump_NegotiateFlags(NegotiateFlags): if NegotiateFlags & NTLM_NegotiateUnicode: print "NTLM_NegotiateUnicode set" @@ -249,14 +261,15 @@ def dump_NegotiateFlags(NegotiateFlags): if NegotiateFlags & NTLM_Negotiate56: print "NTLM_Negotiate56 set" + def create_NTLM_NEGOTIATE_MESSAGE(user, type1_flags=NTLM_TYPE1_FLAGS): BODY_LENGTH = 40 Payload_start = BODY_LENGTH # in bytes - protocol = 'NTLMSSP\0' #name + protocol = 'NTLMSSP\0' # name - type = struct.pack('<I',1) #type 1 + type = struct.pack('<I', 1) # type 1 - flags = struct.pack('<I', type1_flags) + flags = struct.pack('<I', type1_flags) Workstation = gethostname().upper().encode('ascii') user_parts = user.split('\\', 1) DomainName = user_parts[0].upper().encode('ascii') @@ -268,7 +281,7 @@ def create_NTLM_NEGOTIATE_MESSAGE(user, type1_flags=NTLM_TYPE1_FLAGS): Payload_start += len(Workstation) DomainNameLen = struct.pack('<H', len(DomainName)) DomainNameMaxLen = struct.pack('<H', len(DomainName)) - DomainNameBufferOffset = struct.pack('<I',Payload_start) + DomainNameBufferOffset = struct.pack('<I', Payload_start) Payload_start += len(DomainName) ProductMajorVersion = struct.pack('<B', 5) ProductMinorVersion = struct.pack('<B', 1) @@ -279,51 +292,52 @@ def create_NTLM_NEGOTIATE_MESSAGE(user, type1_flags=NTLM_TYPE1_FLAGS): NTLMRevisionCurrent = struct.pack('<B', 15) msg1 = protocol + type + flags + \ - DomainNameLen + DomainNameMaxLen + DomainNameBufferOffset + \ - WorkstationLen + WorkstationMaxLen + WorkstationBufferOffset + \ - ProductMajorVersion + ProductMinorVersion + ProductBuild + \ - VersionReserved1 + VersionReserved2 + VersionReserved3 + NTLMRevisionCurrent - assert BODY_LENGTH==len(msg1), "BODY_LENGTH: %d != msg1: %d" % (BODY_LENGTH,len(msg1)) + DomainNameLen + DomainNameMaxLen + DomainNameBufferOffset + \ + WorkstationLen + WorkstationMaxLen + WorkstationBufferOffset + \ + ProductMajorVersion + ProductMinorVersion + ProductBuild + \ + VersionReserved1 + VersionReserved2 + VersionReserved3 + NTLMRevisionCurrent + assert BODY_LENGTH == len(msg1), "BODY_LENGTH: %d != msg1: %d" % (BODY_LENGTH, len(msg1)) msg1 += Workstation + DomainName msg1 = base64.encodestring(msg1) msg1 = string.replace(msg1, '\n', '') return msg1 + def parse_NTLM_CHALLENGE_MESSAGE(msg2): "" msg2 = base64.decodestring(msg2) Signature = msg2[0:8] - msg_type = struct.unpack("<I",msg2[8:12])[0] - assert(msg_type==2) - TargetNameLen = struct.unpack("<H",msg2[12:14])[0] - TargetNameMaxLen = struct.unpack("<H",msg2[14:16])[0] - TargetNameOffset = struct.unpack("<I",msg2[16:20])[0] - TargetName = msg2[TargetNameOffset:TargetNameOffset+TargetNameMaxLen] - NegotiateFlags = struct.unpack("<I",msg2[20:24])[0] + msg_type = struct.unpack("<I", msg2[8:12])[0] + assert(msg_type == 2) + TargetNameLen = struct.unpack("<H", msg2[12:14])[0] + TargetNameMaxLen = struct.unpack("<H", msg2[14:16])[0] + TargetNameOffset = struct.unpack("<I", msg2[16:20])[0] + TargetName = msg2[TargetNameOffset:TargetNameOffset + TargetNameMaxLen] + NegotiateFlags = struct.unpack("<I", msg2[20:24])[0] ServerChallenge = msg2[24:32] Reserved = msg2[32:40] - TargetInfoLen = struct.unpack("<H",msg2[40:42])[0] - TargetInfoMaxLen = struct.unpack("<H",msg2[42:44])[0] - TargetInfoOffset = struct.unpack("<I",msg2[44:48])[0] - TargetInfo = msg2[TargetInfoOffset:TargetInfoOffset+TargetInfoLen] - i=0 - TimeStamp = '\0'*8 - while(i<TargetInfoLen): - AvId = struct.unpack("<H",TargetInfo[i:i+2])[0] - AvLen = struct.unpack("<H",TargetInfo[i+2:i+4])[0] - AvValue = TargetInfo[i+4:i+4+AvLen] - i = i+4+AvLen + TargetInfoLen = struct.unpack("<H", msg2[40:42])[0] + TargetInfoMaxLen = struct.unpack("<H", msg2[42:44])[0] + TargetInfoOffset = struct.unpack("<I", msg2[44:48])[0] + TargetInfo = msg2[TargetInfoOffset:TargetInfoOffset + TargetInfoLen] + i = 0 + TimeStamp = '\0' * 8 + while i < TargetInfoLen: + AvId = struct.unpack("<H", TargetInfo[i:i + 2])[0] + AvLen = struct.unpack("<H", TargetInfo[i + 2:i + 4])[0] + AvValue = TargetInfo[i + 4:i + 4 + AvLen] + i = i + 4 + AvLen if AvId == NTLM_MsvAvTimestamp: TimeStamp = AvValue - #~ print AvId, AvValue.decode('utf-16') return (ServerChallenge, NegotiateFlags) + def create_NTLM_AUTHENTICATE_MESSAGE(nonce, user, domain, password, NegotiateFlags): "" - is_unicode = NegotiateFlags & NTLM_NegotiateUnicode + is_unicode = NegotiateFlags & NTLM_NegotiateUnicode is_NegotiateExtendedSecurity = NegotiateFlags & NTLM_NegotiateExtendedSecurity - flags = struct.pack('<I',NTLM_TYPE2_FLAGS) + flags = struct.pack('<I', NTLM_TYPE2_FLAGS) BODY_LENGTH = 72 Payload_start = BODY_LENGTH # in bytes @@ -344,10 +358,10 @@ def create_NTLM_AUTHENTICATE_MESSAGE(nonce, user, domain, password, NegotiateFla pwhash = create_NT_hashed_password_v1(password, UserName, DomainName) ClientChallenge = "" for i in range(8): - ClientChallenge+= chr(random.getrandbits(8)) - (NtChallengeResponse, LmChallengeResponse) = ntlm2sr_calc_resp(pwhash, nonce, ClientChallenge) #='\x39 e3 f4 cd 59 c5 d8 60') + ClientChallenge += chr(random.getrandbits(8)) + (NtChallengeResponse, LmChallengeResponse) = ntlm2sr_calc_resp(pwhash, nonce, ClientChallenge) # ='\x39 e3 f4 cd 59 c5 d8 60') Signature = 'NTLMSSP\0' - MessageType = struct.pack('<I',3) #type 3 + MessageType = struct.pack('<I', 3) # type 3 DomainNameLen = struct.pack('<H', len(DomainName)) DomainNameMaxLen = struct.pack('<H', len(DomainName)) @@ -376,8 +390,8 @@ def create_NTLM_AUTHENTICATE_MESSAGE(nonce, user, domain, password, NegotiateFla EncryptedRandomSessionKeyLen = struct.pack('<H', len(EncryptedRandomSessionKey)) EncryptedRandomSessionKeyMaxLen = struct.pack('<H', len(EncryptedRandomSessionKey)) - EncryptedRandomSessionKeyOffset = struct.pack('<I',Payload_start) - Payload_start += len(EncryptedRandomSessionKey) + EncryptedRandomSessionKeyOffset = struct.pack('<I', Payload_start) + Payload_start += len(EncryptedRandomSessionKey) NegotiateFlags = flags ProductMajorVersion = struct.pack('<B', 5) @@ -388,24 +402,25 @@ def create_NTLM_AUTHENTICATE_MESSAGE(nonce, user, domain, password, NegotiateFla VersionReserved3 = struct.pack('<B', 0) NTLMRevisionCurrent = struct.pack('<B', 15) - MIC = struct.pack('<IIII',0,0,0,0) + MIC = struct.pack('<IIII', 0, 0, 0, 0) msg3 = Signature + MessageType + \ - LmChallengeResponseLen + LmChallengeResponseMaxLen + LmChallengeResponseOffset + \ - NtChallengeResponseLen + NtChallengeResponseMaxLen + NtChallengeResponseOffset + \ - DomainNameLen + DomainNameMaxLen + DomainNameOffset + \ - UserNameLen + UserNameMaxLen + UserNameOffset + \ - WorkstationLen + WorkstationMaxLen + WorkstationOffset + \ - EncryptedRandomSessionKeyLen + EncryptedRandomSessionKeyMaxLen + EncryptedRandomSessionKeyOffset + \ - NegotiateFlags + \ - ProductMajorVersion + ProductMinorVersion + ProductBuild + \ - VersionReserved1 + VersionReserved2 + VersionReserved3 + NTLMRevisionCurrent - assert BODY_LENGTH==len(msg3), "BODY_LENGTH: %d != msg3: %d" % (BODY_LENGTH,len(msg3)) + LmChallengeResponseLen + LmChallengeResponseMaxLen + LmChallengeResponseOffset + \ + NtChallengeResponseLen + NtChallengeResponseMaxLen + NtChallengeResponseOffset + \ + DomainNameLen + DomainNameMaxLen + DomainNameOffset + \ + UserNameLen + UserNameMaxLen + UserNameOffset + \ + WorkstationLen + WorkstationMaxLen + WorkstationOffset + \ + EncryptedRandomSessionKeyLen + EncryptedRandomSessionKeyMaxLen + EncryptedRandomSessionKeyOffset + \ + NegotiateFlags + \ + ProductMajorVersion + ProductMinorVersion + ProductBuild + \ + VersionReserved1 + VersionReserved2 + VersionReserved3 + NTLMRevisionCurrent + assert BODY_LENGTH == len(msg3), "BODY_LENGTH: %d != msg3: %d" % (BODY_LENGTH, len(msg3)) Payload = DomainName + UserName + Workstation + LmChallengeResponse + NtChallengeResponse + EncryptedRandomSessionKey msg3 += Payload msg3 = base64.encodestring(msg3) msg3 = string.replace(msg3, '\n', '') return msg3 + def calc_resp(password_hash, server_challenge): """calc_resp generates the LM response given a 16-byte password hash and the challenge from the Type-2 message. @@ -418,28 +433,31 @@ def calc_resp(password_hash, server_challenge): """ # padding with zeros to make the hash 21 bytes long password_hash = password_hash + '\0' * (21 - len(password_hash)) - return des_encrypt(password_hash[ 0: 7], server_challenge[0:8]) + \ - des_encrypt(password_hash[ 7:14], server_challenge[0:8]) + \ - des_encrypt(password_hash[14:21], server_challenge[0:8]) + return des_encrypt(password_hash[0: 7], server_challenge[0:8]) + \ + des_encrypt(password_hash[7:14], server_challenge[0:8]) + \ + des_encrypt(password_hash[14:21], server_challenge[0:8]) + -def ComputeResponse(ResponseKeyNT, ResponseKeyLM, ServerChallenge, ServerName, ClientChallenge='\xaa'*8, Time='\0'*8): - LmChallengeResponse = hmac.new(ResponseKeyLM, ServerChallenge+ClientChallenge).digest() + ClientChallenge +def ComputeResponse(ResponseKeyNT, ResponseKeyLM, ServerChallenge, ServerName, ClientChallenge='\xaa' * 8, Time='\0' * 8): + LmChallengeResponse = hmac.new(ResponseKeyLM, ServerChallenge + ClientChallenge).digest() + ClientChallenge Responserversion = '\x01' HiResponserversion = '\x01' - temp = Responserversion + HiResponserversion + '\0'*6 + Time + ClientChallenge + '\0'*4 + ServerChallenge + '\0'*4 - NTProofStr = hmac.new(ResponseKeyNT, ServerChallenge + temp).digest() + temp = Responserversion + HiResponserversion + '\0' * 6 + Time + ClientChallenge + '\0' * 4 + ServerChallenge + '\0' * 4 + NTProofStr = hmac.new(ResponseKeyNT, ServerChallenge + temp).digest() NtChallengeResponse = NTProofStr + temp SessionBaseKey = hmac.new(ResponseKeyNT, NTProofStr).digest() return (NtChallengeResponse, LmChallengeResponse) -def ntlm2sr_calc_resp(ResponseKeyNT, ServerChallenge, ClientChallenge='\xaa'*8): - LmChallengeResponse = ClientChallenge + '\0'*16 - sess = md5(ServerChallenge+ClientChallenge).digest() + +def ntlm2sr_calc_resp(ResponseKeyNT, ServerChallenge, ClientChallenge='\xaa' * 8): + LmChallengeResponse = ClientChallenge + '\0' * 16 + sess = md5(ServerChallenge + ClientChallenge).digest() NtChallengeResponse = calc_resp(ResponseKeyNT, sess[0:8]) return (NtChallengeResponse, LmChallengeResponse) + def create_LM_hashed_password_v1(passwd): "setup LanManager password" "create LanManager hashed password" @@ -454,66 +472,70 @@ def create_LM_hashed_password_v1(passwd): return des_encrypt(lm_pw[0:7], magic_str) + des_encrypt(lm_pw[7:14], magic_str) + def create_NT_hashed_password_v1(passwd, user=None, domain=None): "create NT hashed password" digest = md4(passwd.encode('utf-16le')).digest() return digest + def create_NT_hashed_password_v2(passwd, user, domain): "create NT hashed password" digest = create_NT_hashed_password_v1(passwd) - return hmac.new(digest, (user.upper()+domain).encode('utf-16le')).digest() + return hmac.new(digest, (user.upper() + domain).encode('utf-16le')).digest() return digest + def create_sessionbasekey(password): return md4(create_NT_hashed_password_v1(password)).digest() + if __name__ == "__main__": - def ByteToHex( byteStr ): + def ByteToHex(byteStr): """ Convert a byte string to it's hex string representation e.g. for output. """ - return ' '.join( [ "%02X" % ord( x ) for x in byteStr ] ) + return ' '.join(["%02X" % ord(x) for x in byteStr]) - def HexToByte( hexStr ): + def HexToByte(hexStr): """ Convert a string hex byte values into a byte string. The Hex Byte values may or may not be space separated. """ bytes = [] - hexStr = ''.join( hexStr.split(" ") ) + hexStr = ''.join(hexStr.split(" ")) for i in range(0, len(hexStr), 2): - bytes.append( chr( int (hexStr[i:i+2], 16 ) ) ) + bytes.append(chr(int(hexStr[i:i + 2], 16))) - return ''.join( bytes ) + return ''.join(bytes) ServerChallenge = HexToByte("01 23 45 67 89 ab cd ef") - ClientChallenge = '\xaa'*8 - Time = '\x00'*8 + ClientChallenge = '\xaa' * 8 + Time = '\x00' * 8 Workstation = "COMPUTER".encode('utf-16-le') ServerName = "Server".encode('utf-16-le') User = "User" Domain = "Domain" Password = "Password" - RandomSessionKey = '\55'*16 + RandomSessionKey = '\55' * 16 assert HexToByte("e5 2c ac 67 41 9a 9a 22 4a 3b 10 8f 3f a6 cb 6d") == create_LM_hashed_password_v1(Password) # [MS-NLMP] page 72 assert HexToByte("a4 f4 9c 40 65 10 bd ca b6 82 4e e7 c3 0f d8 52") == create_NT_hashed_password_v1(Password) # [MS-NLMP] page 73 assert HexToByte("d8 72 62 b0 cd e4 b1 cb 74 99 be cc cd f1 07 84") == create_sessionbasekey(Password) assert HexToByte("67 c4 30 11 f3 02 98 a2 ad 35 ec e6 4f 16 33 1c 44 bd be d9 27 84 1f 94") == calc_resp(create_NT_hashed_password_v1(Password), ServerChallenge) assert HexToByte("98 de f7 b8 7f 88 aa 5d af e2 df 77 96 88 a1 72 de f1 1c 7d 5c cd ef 13") == calc_resp(create_LM_hashed_password_v1(Password), ServerChallenge) - (NTLMv1Response,LMv1Response) = ntlm2sr_calc_resp(create_NT_hashed_password_v1(Password), ServerChallenge, ClientChallenge) + (NTLMv1Response, LMv1Response) = ntlm2sr_calc_resp(create_NT_hashed_password_v1(Password), ServerChallenge, ClientChallenge) assert HexToByte("aa aa aa aa aa aa aa aa 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00") == LMv1Response # [MS-NLMP] page 75 assert HexToByte("75 37 f8 03 ae 36 71 28 ca 45 82 04 bd e7 ca f8 1e 97 ed 26 83 26 72 32") == NTLMv1Response assert HexToByte("0c 86 8a 40 3b fd 7a 93 a3 00 1e f2 2e f0 2e 3f") == create_NT_hashed_password_v2(Password, User, Domain) # [MS-NLMP] page 76 ResponseKeyLM = ResponseKeyNT = create_NT_hashed_password_v2(Password, User, Domain) - (NTLMv2Response,LMv2Response) = ComputeResponse(ResponseKeyNT, ResponseKeyLM, ServerChallenge, ServerName, ClientChallenge, Time) + (NTLMv2Response, LMv2Response) = ComputeResponse(ResponseKeyNT, ResponseKeyLM, ServerChallenge, ServerName, ClientChallenge, Time) assert HexToByte("86 c3 50 97 ac 9c ec 10 25 54 76 4a 57 cc cc 19 aa aa aa aa aa aa aa aa") == LMv2Response # [MS-NLMP] page 76 # expected failure # According to the spec in section '3.3.2 NTLM v2 Authentication' the NTLMv2Response should be longer than the value given on page 77 (this suggests a mistake in the spec) - #~ assert HexToByte("68 cd 0a b8 51 e5 1c 96 aa bc 92 7b eb ef 6a 1c") == NTLMv2Response, "\nExpected: 68 cd 0a b8 51 e5 1c 96 aa bc 92 7b eb ef 6a 1c\nActual: %s" % ByteToHex(NTLMv2Response) # [MS-NLMP] page 77 + # ~ assert HexToByte("68 cd 0a b8 51 e5 1c 96 aa bc 92 7b eb ef 6a 1c") == NTLMv2Response, "\nExpected: 68 cd 0a b8 51 e5 1c 96 aa bc 92 7b eb ef 6a 1c\nActual: %s" % ByteToHex(NTLMv2Response) # [MS-NLMP] page 77 diff --git a/virt/libvirtd/__init__.py b/virt/libvirtd/__init__.py new file mode 100644 index 0000000..4c8b00c --- /dev/null +++ b/virt/libvirtd/__init__.py @@ -0,0 +1,2 @@ + +from libvirtd import Libvirtd diff --git a/virt/libvirtd/libvirtd.py b/virt/libvirtd/libvirtd.py new file mode 100644 index 0000000..b0b8999 --- /dev/null +++ b/virt/libvirtd/libvirtd.py @@ -0,0 +1,168 @@ +""" +Module for communcating with libvirt, part of virt-who + +Copyright (C) 2014 Radek Novacek <rnovacek(a)redhat.com> + +This program is free software; you can redistribute it and/or +modify it under the terms of the GNU General Public License +as published by the Free Software Foundation; either version 2 +of the License, or (at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +""" + +import time +import logging +import libvirt +import threading +from event import virEventLoopPureStart + +import virt + + +eventLoopThread = None + + +class LibvirtMonitor(threading.Thread): + """ Singleton class that performs background event monitoring. """ + _instance = None + + def __new__(cls, *args, **kwargs): + if not cls._instance: + cls._instance = super(LibvirtMonitor, cls).__new__(cls, *args, **kwargs) + cls._instance.event = None + cls._instance.terminate = threading.Event() + cls._instance.domainIds = [] + cls._instance.definedDomains = [] + return cls._instance + + STATUS_NOT_STARTED, STATUS_RUNNING, STATUS_DISABLED = range(3) + + def _prepare(self): + self.logger = logging.getLogger("rhsm-app") + self.running = threading.Event() + self.status = LibvirtMonitor.STATUS_NOT_STARTED + + def run(self): + self._prepare() + while not self.terminate.isSet(): + self._checkChange() + time.sleep(5) + + def _checkChange(self): + try: + virt = libvirt.openReadOnly('') + except libvirt.libvirtError, e: + # Show error only once + if self.status != LibvirtMonitor.STATUS_DISABLED: + self.logger.debug("Unable to connect to libvirtd, disabling event monitoring") + self.logger.exception(e) + self.status = LibvirtMonitor.STATUS_DISABLED + return + + domainIds = virt.listDomainsID() + definedDomains = virt.listDefinedDomains() + + changed = domainIds != self.domainIds or definedDomains != self.definedDomains + + self.domainIds = domainIds + self.definedDomains = definedDomains + + virt.close() + + if changed and self.event is not None and self.status != LibvirtMonitor.STATUS_NOT_STARTED: + self.event.set() + + if self.status == LibvirtMonitor.STATUS_DISABLED: + self.logger.debug("Event monitoring resumed") + self.status = LibvirtMonitor.STATUS_RUNNING + + def set_event(self, event): + self.event = event + + def stop(self): + self.terminate.set() + + +class Libvirtd(virt.DirectVirt): + """ Class for interacting with libvirt. """ + CONFIG_TYPE = "libvirt" + + def __init__(self, logger, config, registerEvents=True): + self.changedCallback = None + self.logger = logger + self.registerEvents = registerEvents + libvirt.registerErrorHandler(lambda ctx, error: None, None) + + def _connect(self): + monitor = LibvirtMonitor() + try: + self.virt = libvirt.openReadOnly('') + except libvirt.libvirtError, e: + self.logger.exception("Error in libvirt backend") + raise virt.VirtError(str(e)) + + def listDomains(self): + """ Get list of all domains. """ + domains = [] + self._connect() + + try: + # Active domains + for domainID in self.virt.listDomainsID(): + domain = self.virt.lookupByID(domainID) + if domain.UUIDString() == "00000000-0000-0000-0000-000000000000": + # Don't send Domain-0 on xen (zeroed uuid) + continue + domains.append(virt.Domain(self.virt, domain)) + self.logger.debug("Virtual machine found: %s: %s" % (domain.name(), domain.UUIDString())) + + # Non active domains + for domainName in self.virt.listDefinedDomains(): + domain = self.virt.lookupByName(domainName) + domains.append(virt.Domain(self.virt, domain)) + self.logger.debug("Virtual machine found: %s: %s" % (domainName, domain.UUIDString())) + except libvirt.libvirtError, e: + raise virt.VirtError(str(e)) + return domains + + def canMonitor(self): + return True + + def startMonitoring(self, event): + monitor = LibvirtMonitor() + if not monitor.isAlive(): + monitor.set_event(event) + monitor.start() + + +def eventToString(event): + eventStrings = ("Defined", "Undefined", "Started", "Suspended", "Resumed", + "Stopped", "Shutdown") + try: + return eventStrings[event] + except IndexError: + return "Unknown (%d)" % event + + +def detailToString(event, detail): + eventStrings = ( + ("Added", "Updated"), # Defined + ("Removed", ), # Undefined + ("Booted", "Migrated", "Restored", "Snapshot", "Wakeup"), # Started + ("Paused", "Migrated", "IOError", "Watchdog", "Restored", "Snapshot"), # Suspended + ("Unpaused", "Migrated", "Snapshot"), # Resumed + ("Shutdown", "Destroyed", "Crashed", "Migrated", "Saved", "Failed", "Snapshot"), # Stopped + ("Finished",), # Shutdown + ) + try: + return eventStrings[event][detail] + except IndexError: + return "Unknown (%d)" % detail diff --git a/virt/rhevm/__init__.py b/virt/rhevm/__init__.py new file mode 100644 index 0000000..3787775 --- /dev/null +++ b/virt/rhevm/__init__.py @@ -0,0 +1,2 @@ + +from rhevm import RhevM diff --git a/rhevm.py b/virt/rhevm/rhevm.py similarity index 95% rename from rhevm.py rename to virt/rhevm/rhevm.py index 188c4fa..570fa7b 100644 --- a/rhevm.py +++ b/virt/rhevm/rhevm.py @@ -23,13 +23,18 @@ import urlparse import urllib2 import base64 +import virt + # Import XML parser try: from elementtree import ElementTree except ImportError: from xml.etree import ElementTree -class RHEVM: + +class RhevM(virt.HypervisorVirt): + CONFIG_TYPE = "rhevm" + def __init__(self, logger, url, username, password): self.logger = logger self.url = url @@ -101,5 +106,5 @@ if __name__ == '__main__': import logging logger = logging.Logger("") - rhevm = RHEVM(logger, sys.argv[1], sys.argv[2], sys.argv[3]) + rhevm = RhevM(logger, sys.argv[1], sys.argv[2], sys.argv[3]) rhevm.getHostGuestMapping() diff --git a/virt/vdsm/__init__.py b/virt/vdsm/__init__.py new file mode 100644 index 0000000..9173255 --- /dev/null +++ b/virt/vdsm/__init__.py @@ -0,0 +1,2 @@ + +from vdsm import Vdsm diff --git a/vdsm.py b/virt/vdsm/vdsm.py similarity index 92% rename from vdsm.py rename to virt/vdsm/vdsm.py index 2802f75..06b27a3 100644 --- a/vdsm.py +++ b/virt/vdsm/vdsm.py @@ -25,10 +25,16 @@ import xmlrpclib from ConfigParser import SafeConfigParser, NoSectionError, NoOptionError import subprocess +from virt import DirectVirt + + class VdsmError(Exception): pass -class VDSM: + +class Vdsm(DirectVirt): + CONFIG_TYPE = "vdsm" + def __init__(self, logger): self.logger = logger self._readConfig("/etc/vdsm/vdsm.conf") @@ -48,8 +54,9 @@ class VDSM: raise VdsmError("Error in vdsm configuration file: %s" % str(e)) def _getLocalVdsName(self, tsPath): - p = subprocess.Popen(['/usr/bin/openssl', 'x509', '-noout', '-subject', '-in', - '%s/certs/vdsmcert.pem' % tsPath], stdout=subprocess.PIPE, close_fds=True) + p = subprocess.Popen([ + '/usr/bin/openssl', 'x509', '-noout', '-subject', '-in', + '%s/certs/vdsmcert.pem' % tsPath], stdout=subprocess.PIPE, close_fds=True) out, err = p.communicate() if p.returncode != 0: return '0' @@ -100,5 +107,5 @@ class VDSM: if __name__ == '__main__': import logging logger = logging.getLogger("rhsm-app." + __name__) - vdsm = VDSM(logger) + vdsm = Vdsm(logger) print vdsm.listDomains() diff --git a/virt/virt.py b/virt/virt.py new file mode 100644 index 0000000..72f924a --- /dev/null +++ b/virt/virt.py @@ -0,0 +1,88 @@ +""" +Module for abstraction of all virtualization backends, part of virt-who + +Copyright (C) 2014 Radek Novacek <rnovacek(a)redhat.com> + +This program is free software; you can redistribute it and/or +modify it under the terms of the GNU General Public License +as published by the Free Software Foundation; either version 2 +of the License, or (at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +""" + + +class VirtError(Exception): + pass + + +class Domain(dict): + def __init__(self, virt, domain): + self['guestId'] = domain.UUIDString() + self['attributes'] = { + 'hypervisorType': virt.getType(), + 'virtWhoType': "libvirt", + 'active': 0 + } + if domain.isActive(): + self['attributes']['active'] = 1 + try: + self['state'] = domain.state(0)[0] + except AttributeError: + # Some versions of libvirt doesn't have domain.state() method + pass + + +class Virt(object): + def __init__(self, config): + self.config = config + + @classmethod + def fromConfig(cls, logger, config): + """ + Create instance of inherited class based on the config. + """ + + # Imports can't be top-level, it would be circular dependency + import libvirtd + import esx + import rhevm + import vdsm + + for subcls in cls.__subclasses__(): + for subsubcls in subcls.__subclasses__(): + if config.type == subsubcls.CONFIG_TYPE: + return subsubcls(logger, config) + raise KeyError("Invalid config type: %s" % config.type) + + def canMonitor(self): + """ + Return true if inherited class can perform background monitoring + for changes in host/guest association. + """ + return False + + def startMonitoring(self, event): + """ + Start the monitoring for changes in host/guest association. + + This should set the 'event' to force resending of host/guest associations. + """ + raise NotImplementedError() + + +class DirectVirt(Virt): + def listDomains(self): + raise NotImplementedError() + + +class HypervisorVirt(Virt): + def getHostGuestMapping(self): + raise NotImplementedError() diff --git a/virt-who.py b/virtwho.py similarity index 59% rename from virt-who.py rename to virtwho.py index 9262ebc..6baac51 100644 --- a/virt-who.py +++ b/virtwho.py @@ -24,21 +24,19 @@ import time import atexit import signal import errno +import threading +from daemon import daemon from virt import Virt, VirtError -from vdsm import VDSM -from vsphere import VSphere -from rhevm import RHEVM -from hyperv import HyperV -from event import virEventLoopPureStart -from subscriptionmanager import SubscriptionManager, SubscriptionManagerError -from satellite import Satellite, SatelliteError +from manager import Manager, ManagerError +from config import Config, ConfigManager import logging import log from optparse import OptionParser, OptionGroup + class OptionParserEpilog(OptionParser): """ Epilog is new in Python 2.5, we need to support Python 2.4. """ def __init__(self, usage="%prog [options]", description=None, epilog=None): @@ -66,6 +64,7 @@ DefaultInterval = 3600 # Once per hour PIDFILE = "/var/run/virt-who.pid" + class VirtWho(object): def __init__(self, logger, options): """ @@ -77,89 +76,16 @@ class VirtWho(object): """ self.logger = logger self.options = options + self.sync_event = threading.Event() - self.virt = None - self.subscriptionManager = None + self.configManager = ConfigManager() + for config in self.configManager.configs: + logger.debug("Using config named '%s'" % config.name) self.unableToRecoverStr = "Unable to recover" if not options.oneshot: self.unableToRecoverStr += ", retry in %d seconds." % RetryInterval - # True if reload is queued - self.doReload = False - - - def initVirt(self): - """ - Connect to the virtualization supervisor (libvirt or VDSM) - """ - if self.options.virtType == "vdsm": - self.virt = VDSM(self.logger) - elif self.options.virtType == "libvirt": - self.virt = Virt(self.logger, registerEvents=self.options.background) - # We can listen for libvirt events - self.tryRegisterEventCallback() - elif self.options.virtType == "rhevm": - self.virt = RHEVM(self.logger, self.options.server, self.options.username, self.options.password) - elif self.options.virtType == "hyperv": - self.virt = HyperV(self.logger, self.options.server, self.options.username, self.options.password) - else: - # ESX - self.virt = VSphere(self.logger, self.options.server, self.options.username, self.options.password) - - def initSM(self): - """ - Connect to the subscription manager (candlepin). - """ - try: - if self.options.smType == "sam": - self.subscriptionManager = SubscriptionManager(self.logger) - self.subscriptionManager.connect() - elif self.options.smType == "satellite": - self.subscriptionManager = Satellite(self.logger) - self.subscriptionManager.connect(self.options.sat_server, self.options.sat_username, self.options.sat_password) - except NoOptionError, e: - self.logger.exception("Error in reading configuration file (/etc/rhsm/rhsm.conf):") - raise - except SubscriptionManagerError, e: - self.logger.exception("Unable to obtain status from server, UEPConnection is likely not usable:") - raise - except SatelliteError, e: - self.logger.exception("Unable to connect to the RHN Satellite:") - raise - except (KeyboardInterrupt, SystemExit): - raise - except Exception, e: - exceptionCheck(e) - self.logger.exception("Unknown error") - raise - - if self.options.virtType == "libvirt": - self.tryRegisterEventCallback() - - def tryRegisterEventCallback(self): - """ - This method register the handler which listen to guest changes - - If virt-who is running in background mode with libvirt backend, it can - monitor virt guests changes and send updates as soon as the change happens, - - """ - if self.options.background and self.options.virtType == "libvirt": - if self.virt is not None and self.subscriptionManager is not None: - # Send list of virt guests when something changes in libvirt - self.virt.domainEventRegisterCallback(self.subscriptionManager.sendVirtGuests) - - def checkConnections(self): - """ - Check if connection to subscription manager and virtualization supervisor - is established and reconnect if needed. - """ - if self.subscriptionManager is None: - self.initSM() - if self.virt is None: - self.initVirt() - def send(self): """ Send list of uuids to subscription manager @@ -167,9 +93,13 @@ class VirtWho(object): return - True if sending is successful, False otherwise """ # Try to send it twice - return self._send(True) + result = True + for config in self.configManager.configs: + if not self._send(config, True): + result = False + return result - def _send(self, retry): + def _send(self, config, retry): """ Send list of uuids to subscription manager. This method will call itself once if sending fails. @@ -177,188 +107,85 @@ class VirtWho(object): retry - Should be True on first run, False on second. return - True if sending is successful, False otherwise """ - logger = self.logger - try: - self.checkConnections() - except (KeyboardInterrupt, SystemExit): - raise - except Exception, e: - exceptionCheck(e) - if retry: - logger.exception("Unable to create connection:") - return self._send(False) - else: - logger.error(self.unableToRecoverStr) - return False - try: - if self.options.virtType not in ["esx", "rhevm", "hyperv"]: - virtualGuests = self.virt.listDomains() - else: - virtualGuests = self.virt.getHostGuestMapping() + virtualGuests = self._readGuests(config) except (KeyboardInterrupt, SystemExit): raise except Exception, e: exceptionCheck(e) - # Communication with virtualization supervisor failed - self.virt = None # Retry once if retry: - logger.exception("Error in communication with virt backend, trying to recover:") - return self._send(False) + self.logger.exception("Error in communication with virtualization backend, trying to recover:") + return self._send(config, False) else: - logger.error(self.unableToRecoverStr) + self.logger.error(self.unableToRecoverStr) return False try: - if self.options.virtType not in ["esx", "rhevm", "hyperv"]: - self.subscriptionManager.sendVirtGuests(virtualGuests) - else: - result = self.subscriptionManager.hypervisorCheckIn(self.options.owner, self.options.env, virtualGuests, type=self.options.virtType) - - # Show the result of hypervisorCheckIn - for fail in result['failedUpdate']: - logger.error("Error during update list of guests: %s", str(fail)) - for updated in result['updated']: - guests = [x['guestId'] for x in updated['guestIds']] - logger.info("Updated host: %s with guests: [%s]", updated['uuid'], ", ".join(guests)) - for created in result['created']: - guests = [x['guestId'] for x in created['guestIds']] - logger.info("Created host: %s with guests: [%s]", created['uuid'], ", ".join(guests)) + self._sendGuests(config, virtualGuests) except (KeyboardInterrupt, SystemExit): raise except Exception, e: - exceptionCheck(e) # Communication with subscription manager failed - self.subscriptionManager = None + exceptionCheck(e) # Retry once if retry: - logger.exception("Error in communication with subscription manager, trying to recover:") - return self._send(False) + self.logger.exception("Error in communication with subscription manager, trying to recover:") + return self._send(config, False) else: - logger.error(self.unableToRecoverStr) + self.logger.error(self.unableToRecoverStr) return False - return True - def ping(self): - """ - Test if connection to virtualization manager is alive. - - return - True if connection is alive, False otherwise - """ - if self.virt is None: - return False - return self.virt.ping() - - def queueReload(self, *p): - """ - Reload virt-who configuration. Called on SIGHUP signal arrival. - """ - self.doReload = True - - def reloadConfig(self): - try: - self.virt.virt.close() - except AttributeError: - pass - self.virt = None - self.subscriptionManager = None - try: - self.checkConnections() - except (KeyboardInterrupt, SystemExit): - raise - except Exception, e: - exceptionCheck(e) - pass - self.logger.debug("virt-who configution reloaded") - self.doReload = False - - -def daemonize(debugMode): - """ Perform double-fork and redirect std* to /dev/null """ + def _readGuests(self, config): + virt = Virt.fromConfig(self.logger, config) + if not self.options.oneshot and virt.canMonitor(): + virt.startMonitoring(self.sync_event) + if config.type not in ["esx", "rhevm", "hyperv"]: + return virt.listDomains() + else: + return virt.getHostGuestMapping() - # First fork - try: - pid = os.fork() - except OSError: - return False + def _sendGuests(self, config, virtualGuests): + manager = Manager.fromOptions(self.logger, self.options) + if config.type not in ["esx", "rhevm", "hyperv"]: + manager.sendVirtGuests(virtualGuests) + else: + result = manager.hypervisorCheckIn(config, virtualGuests) + + # Show the result of hypervisorCheckIn + for fail in result['failedUpdate']: + logger.error("Error during update list of guests: %s", str(fail)) + for updated in result['updated']: + guests = [x['guestId'] for x in updated['guestIds']] + logger.info("Updated host: %s with guests: [%s]", updated['uuid'], ", ".join(guests)) + for created in result['created']: + guests = [x['guestId'] for x in created['guestIds']] + logger.info("Created host: %s with guests: [%s]", created['uuid'], ", ".join(guests)) + + def run(self): + if self.options.background and self.options.virtType == "libvirt": + self.logger.debug("Starting infinite loop with %d seconds interval and event handling" % self.options.interval) + else: + self.logger.debug("Starting infinite loop with %d seconds interval" % self.options.interval) - if pid > 0: - # Parent process - os._exit(0) + while 1: + # Run in infinite loop and send list of UUIDs every 'options.interval' seconds - # First child process + if self.send(): + timeout = self.options.interval + else: + timeout = RetryInterval - # Create session and set process group ID - os.setsid() + self.sync_event.wait(timeout) + self.sync_event.clear() - # Second fork - try: - pid = os.fork() - except OSError: - return False - - if pid > 0: - # Parent process - os._exit(0) - - # Second child process - - # Redirect std* to /dev/null - devnull = os.open("/dev/null", os.O_RDWR) - os.dup2(devnull, 0) - os.dup2(devnull, 1) - # Don't redirect stderr in debug mode, we need to write debugging output there - if not debugMode: - os.dup2(devnull, 2) - - # Reset file creation mask - os.umask(0) - # Forget current working directory - os.chdir("/") - return True - -def checkPidFile(): - try: - f = open(PIDFILE, "r") - pid = int(f.read().strip()) + def reload(self): try: - os.kill(pid, 0) - print >>sys.stderr, "virt-who seems to be already running. If not, remove %s" % PIDFILE - sys.exit(1) - except OSError: - # Process no longer exists - print >>sys.stderr, "PID file exists but associated process does not, deleting PID file" - os.remove(PIDFILE) - except (KeyboardInterrupt, SystemExit): - raise - except Exception: - pass - -def createPidFile(logger=None): - atexit.register(cleanup) - signal.signal(signal.SIGINT, cleanup) - signal.signal(signal.SIGTERM, cleanup) - - # Write pid to pidfile - try: - f = open(PIDFILE, "w") - f.write("%d" % os.getpid()) - f.close() - except (KeyboardInterrupt, SystemExit): - raise - except Exception, e: - if logger is not None: - logger.error("Unable to create pid file: %s" % str(e)) + self.sync_event.set() + except Exception: + raise -def cleanup(sig=None, stack=None): - try: - os.remove(PIDFILE) - except (KeyboardInterrupt, SystemExit): - raise - except Exception: - pass def exceptionCheck(e): try: @@ -370,10 +197,8 @@ def exceptionCheck(e): except Exception: pass -def main(): - checkPidFile() - createPidFile() +def parseOptions(): parser = OptionParserEpilog(usage="virt-who [-d] [-i INTERVAL] [-b] [-o] [--sam|--satellite] [--libvirt|--vdsm|--esx|--rhevm|--hyperv]", description="Agent for reporting virtual guest IDs to subscription manager", epilog="virt-who also reads enviromental variables. They have the same name as command line arguments but uppercased, with underscore instead of dash and prefixed with VIRTWHO_ (e.g. VIRTWHO_ONE_SHOT). Empty variables are considered as disabled, non-empty as enabled") @@ -383,7 +208,7 @@ def main(): parser.add_option("-i", "--interval", type="int", dest="interval", default=0, help="Acquire and send list of virtual guest each N seconds") virtGroup = OptionGroup(parser, "Virtualization backend", "Choose virtualization backend that should be used to gather host/guest associations") - virtGroup.add_option("--libvirt", action="store_const", dest="virtType", const="libvirt", default="libvirt", help="Use libvirt to list virtual guests [default]") + virtGroup.add_option("--libvirt", action="store_const", dest="virtType", const="libvirt", default=None, help="Use libvirt to list virtual guests [default]") virtGroup.add_option("--vdsm", action="store_const", dest="virtType", const="vdsm", help="Use vdsm to list virtual guests") virtGroup.add_option("--esx", action="store_const", dest="virtType", const="esx", help="Register ESX machines using vCenter") virtGroup.add_option("--rhevm", action="store_const", dest="virtType", const="rhevm", help="Register guests using RHEV-M") @@ -458,6 +283,10 @@ def main(): if env in ["1", "true"]: options.smType = "satellite" + env = os.getenv("VIRTWHO_LIBVIRT", "0").strip().lower() + if env in ["1", "true"]: + options.virtType = "libvirt" + env = os.getenv("VIRTWHO_VDSM", "0").strip().lower() if env in ["1", "true"]: options.virtType = "vdsm" @@ -474,7 +303,6 @@ def main(): if env in ["1", "true"]: options.virtType = "hyperv" - def checkEnv(variable, option, name): """ If `option` is empty, check enviromental `variable` and return its value. @@ -533,75 +361,99 @@ def main(): # (e.g. libvirtd restart) options.interval = DefaultInterval + return (logger, options) + + +class PIDLock(object): + def __init__(self, filename): + self.filename = filename + + def is_locked(self): + try: + f = open(self.filename, "r") + pid = int(f.read().strip()) + try: + os.kill(pid, 0) + return True + except OSError: + # Process no longer exists + print >>sys.stderr, "PID file exists but associated process does not, deleting PID file" + os.remove(self.filename) + return False + except (KeyboardInterrupt, SystemExit): + raise + except Exception: + return False + + def __enter__(self): + # Write pid to pidfile + try: + f = open(self.filename, "w") + f.write("%d" % os.getpid()) + f.close() + except (KeyboardInterrupt, SystemExit): + raise + except Exception, e: + if logger is not None: + logger.error("Unable to create pid file: %s" % str(e)) + + def __exit__(self, exc_type, exc_value, traceback): + try: + os.remove(self.filename) + except (KeyboardInterrupt, SystemExit): + raise + except Exception: + pass + + +def main(): + lock = PIDLock(PIDFILE) + if lock.is_locked(): + print >>sys.stderr, "virt-who seems to be already running. If not, remove %s" % PIDFILE + sys.exit(1) + logger, options = parseOptions() + if options.background: - # Do a double-fork and other daemon initialization - if not daemonize(options.debug): - logger.error("Unable to fork, continuing in foreground") - createPidFile(logger) - - if not options.oneshot: - if options.background and options.virtType == "libvirt": - logger.debug("Starting event loop") - virEventLoopPureStart() - else: - logger.warning("Listening for events is not available in VDSM, ESX, RHEV-M or Hyper-V mode") + # Do a daemon initialization + with daemon.DaemonContext(pidfile=lock, files_preserve=[logger.handlers[0].stream]): + _main(logger, options) + else: + with lock: + _main(logger, options) + +def _main(logger, options): global RetryInterval if options.interval < RetryInterval: RetryInterval = options.interval virtWho = VirtWho(logger, options) - signal.signal(signal.SIGHUP, virtWho.queueReload) - try: - virtWho.checkConnections() - except (KeyboardInterrupt, SystemExit): - raise - except Exception: - pass + if options.virtType is not None: + config = Config("virt-who", options.virtType, options.server, + options.username, options.password, options.owner, options.env) + virtWho.configManager.addConfig(config) + if len(virtWho.configManager.configs) == 0: + logger.error("No configurations found") + else: + for config in virtWho.configManager.configs: + logger.info("Using virt-who configuration: %s" % config.name) - logger.debug("Virt-who is running in %s mode" % options.virtType) + def reload(signal, stackframe): + virtWho.reload() + signal.signal(signal.SIGHUP, reload) if options.oneshot: # Send list of virtual guests and exit virtWho.send() else: - if options.background: - logger.debug("Starting infinite loop with %d seconds interval and event handling" % options.interval) - else: - logger.debug("Starting infinite loop with %d seconds interval" % options.interval) - - while 1: - # Run in infinite loop and send list of UUIDs every 'options.interval' seconds + virtWho.run() - if virtWho.send(): - # Check if connection is established each 'RetryInterval' seconds - slept = 0 - while slept < options.interval: - # Sleep 'RetryInterval' or the rest of options.interval - t = min(RetryInterval, options.interval - slept) - # But sleep at least one second - t = max(t, 1) - time.sleep(t) - slept += t - - # Reload configuration if queued - if virtWho.doReload: - virtWho.reloadConfig() - break - - # Check the connection - if not virtWho.ping(): - # End the cycle - break - else: - # If last send fails, new try will be sooner - time.sleep(RetryInterval) if __name__ == '__main__': try: main() except (SystemExit, KeyboardInterrupt): - raise + sys.exit(1) except Exception, e: logger = log.getLogger(False, False) logger.exception("Fatal error:")

9 years, 10 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

virt-who-devel June 2014