November 2012 - vdsm-patches - Fedora mailing-lists

Change in vdsm[master]: Add license
by Pablo Iranzo Gómez 26 Nov '12

26 Nov '12

Pablo Iranzo Gómez has uploaded a new change for review. Change subject: Add license ...................................................................... Add license Change-Id: Ib439e74f0010ca9921351421cabdf286e8fa21f5 Signed-off-by: Pablo <Pablo.Iranzo(a)gmail.com> --- A contrib/forceVMstart/COPYING.GPLv2 1 file changed, 339 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/75/9475/1 diff --git a/contrib/forceVMstart/COPYING.GPLv2 b/contrib/forceVMstart/COPYING.GPLv2 new file mode 100644 index 0000000..d159169 --- /dev/null +++ b/contrib/forceVMstart/COPYING.GPLv2 @@ -0,0 +1,339 @@ + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Lesser General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + <one line to give the program's name and a brief idea of what it does.> + Copyright (C) <year> <name of author> + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License along + with this program; if not, write to the Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) year name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + <signature of Ty Coon>, 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. -- To view, visit http://gerrit.ovirt.org/9475 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib439e74f0010ca9921351421cabdf286e8fa21f5 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Pablo Iranzo Gómez <Pablo.Iranzo(a)redhat.com>

2 4

Change in vdsm[master]: File rename to fulfill naming convention
by Pablo Iranzo Gómez 26 Nov '12

26 Nov '12

Pablo Iranzo Gómez has uploaded a new change for review. Change subject: File rename to fulfill naming convention ...................................................................... File rename to fulfill naming convention Change-Id: Ieab5da7cebc6724088689dd30aaa1a9977db1459 Signed-off-by: Pablo <Pablo.Iranzo(a)gmail.com> --- R contrib/forceVMstart/forceVMstart.py 1 file changed, 0 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/74/9474/1 diff --git a/contrib/forceVMstart/vdsEmergency-1.0.0.py b/contrib/forceVMstart/forceVMstart.py similarity index 100% rename from contrib/forceVMstart/vdsEmergency-1.0.0.py rename to contrib/forceVMstart/forceVMstart.py -- To view, visit http://gerrit.ovirt.org/9474 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ieab5da7cebc6724088689dd30aaa1a9977db1459 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Pablo Iranzo Gómez <Pablo.Iranzo(a)redhat.com>

2 4

Change in vdsm[master]: Remove REST bindings
by Dan Kenigsberg 26 Nov '12

26 Nov '12

Dan Kenigsberg has submitted this change and it was merged. Change subject: Remove REST bindings ...................................................................... Remove REST bindings No one uses them and no one should. We are going in a different direction for a supported API and keeping the rest bindings alive is confusing and cumbersome. Change-Id: I6c2a2909323e943e076db8443ce84f839eab874c Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M Makefile.am M configure.ac M tests/Makefile.am M tests/functional/Makefile.am D tests/functional/restTests.py D tests/restData.py D tests/restTests.py M vdsm.spec.in M vdsm/Makefile.am M vdsm/clientIF.py M vdsm/config.py.in D vdsm/rest/BindingREST.py D vdsm/rest/Controller.py D vdsm/rest/Dispatcher.py D vdsm/rest/Makefile.am D vdsm/rest/__init__.py D vdsm/rest/templates/Makefile.am D vdsm/rest/templates/api.xsd D vdsm/rest/templates/image.json.x D vdsm/rest/templates/image.xml.x D vdsm/rest/templates/images.json.x D vdsm/rest/templates/images.xml.x D vdsm/rest/templates/response.json.x D vdsm/rest/templates/response.xml.x D vdsm/rest/templates/root.json.x D vdsm/rest/templates/root.xml.x D vdsm/rest/templates/rsdl.xml D vdsm/rest/templates/storageconnectionref.json.x D vdsm/rest/templates/storageconnectionref.xml.x D vdsm/rest/templates/storageconnectionrefs.json.x D vdsm/rest/templates/storageconnectionrefs.xml.x D vdsm/rest/templates/storagedomain.json.x D vdsm/rest/templates/storagedomain.xml.x D vdsm/rest/templates/storagedomains.json.x D vdsm/rest/templates/storagedomains.xml.x D vdsm/rest/templates/storagepool.json.x D vdsm/rest/templates/storagepool.xml.x D vdsm/rest/templates/storagepools.json.x D vdsm/rest/templates/storagepools.xml.x D vdsm/rest/templates/task.json.x D vdsm/rest/templates/task.xml.x D vdsm/rest/templates/tasks.json.x D vdsm/rest/templates/tasks.xml.x D vdsm/rest/templates/volume.json.x D vdsm/rest/templates/volume.xml.x D vdsm/rest/templates/volumes.json.x D vdsm/rest/templates/volumes.xml.x 47 files changed, 3 insertions(+), 4,195 deletions(-) Approvals: Adam Litke: Looks good to me, but someone else must approve Dan Kenigsberg: Verified; Looks good to me, approved -- To view, visit http://gerrit.ovirt.org/7754 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: merged Gerrit-Change-Id: I6c2a2909323e943e076db8443ce84f839eab874c Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com> Gerrit-Reviewer: Adam Litke <agl(a)us.ibm.com> Gerrit-Reviewer: Barak Azulay <bazulay(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com> Gerrit-Reviewer: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> Gerrit-Reviewer: oVirt Jenkins CI Server

1 0

Change in vdsm[master]: Remove REST bindings
by Dan Kenigsberg 26 Nov '12

26 Nov '12

Dan Kenigsberg has posted comments on this change. Change subject: Remove REST bindings ...................................................................... Patch Set 2: Verified; Looks good to me, approved alas, farewell. -- To view, visit http://gerrit.ovirt.org/7754 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I6c2a2909323e943e076db8443ce84f839eab874c Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com> Gerrit-Reviewer: Adam Litke <agl(a)us.ibm.com> Gerrit-Reviewer: Barak Azulay <bazulay(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com> Gerrit-Reviewer: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> Gerrit-Reviewer: oVirt Jenkins CI Server

1 0

Change in vdsm[master]: Remove REST bindings
by agl＠us.ibm.com 26 Nov '12

26 Nov '12

Adam Litke has posted comments on this change. Change subject: Remove REST bindings ...................................................................... Patch Set 2: Looks good to me, but someone else must approve -- To view, visit http://gerrit.ovirt.org/7754 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I6c2a2909323e943e076db8443ce84f839eab874c Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com> Gerrit-Reviewer: Adam Litke <agl(a)us.ibm.com> Gerrit-Reviewer: Barak Azulay <bazulay(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com> Gerrit-Reviewer: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> Gerrit-Reviewer: oVirt Jenkins CI Server

1 0

Change in vdsm[master]: mom: Report the status of the mom threads
by agl＠us.ibm.com 26 Nov '12

26 Nov '12

Adam Litke has uploaded a new change for review. Change subject: mom: Report the status of the mom threads ...................................................................... mom: Report the status of the mom threads It will be useful to check on the status of the MOM threads. Add a new field to the HostStats structure to provide this information. Change-Id: Ibd36a5c3a91abfa3fa29c811ce9b3586bf0bda36 Signed-off-by: Adam Litke <agl(a)us.ibm.com> --- M vdsm/API.py M vdsm/momIF.py M vdsm_api/vdsmapi-schema.json 3 files changed, 28 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/87/9287/1 diff --git a/vdsm/API.py b/vdsm/API.py index 202afc8..cb1f4cc 100644 --- a/vdsm/API.py +++ b/vdsm/API.py @@ -1120,8 +1120,10 @@ stats['dateTime'] = '%02d-%02d-%02dT%02d:%02d:%02d GMT' % ( tm_year, tm_mon, tm_day, tm_hour, tm_min, tm_sec) if self._cif.mom: + stats['momStatus'] = self._cif.mom.getStatus() stats.update(self._cif.mom.getKsmStats()) else: + stats['momStatus'] = 'disabled' stats['ksmState'] = self._cif.ksmMonitor.state stats['ksmPages'] = self._cif.ksmMonitor.pages stats['ksmCpu'] = self._cif.ksmMonitor.cpuUsage diff --git a/vdsm/momIF.py b/vdsm/momIF.py index 827e9e4..cbe94d0 100644 --- a/vdsm/momIF.py +++ b/vdsm/momIF.py @@ -55,3 +55,9 @@ if self._mom is not None: self.log.info("Shutting down MOM") self._mom.shutdown() + + def getStatus(self): + if self.isAlive(): + return 'active' + else: + return 'stopped' diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index 6591410..149bfef 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -1420,6 +1420,22 @@ 'key': 'str', 'value': 'PathStats'} ## +# @MOMStatus: +# +# An enumeration of possible MOM statuses. +# +# @disabled: MOM is not enabled on this host +# +# @active: MOM is enabled and running +# +# @stopped: MOM has terminated +# +# Since: 4.10.1 +## +{'enum': 'MOMStatus', 'data': ['disabled', 'active', 'stopped']} + + +## # @HostStats: # # Statistics about this host. @@ -1490,6 +1506,8 @@ # # @generationID: A @UUID that is refreshed each time vdsm starts # +# @momStatus: The current status of the MOM policy engine +# # Since: 4.10.0 ## {'type': 'HostStats', @@ -1506,7 +1524,8 @@ 'swapTotal': 'int', 'swapFree': 'int','vmCount': 'int', 'vmActive': 'int', 'vmMigrating': 'int', 'dateTime': 'str', 'ksmState': 'bool', 'ksmPages': 'int', 'ksmCpu': 'float', - 'netConfigDirty': 'bool', 'generationID': 'UUID'} + 'netConfigDirty': 'bool', 'generationID': 'UUID', + 'momStatus': 'MOMStatus'} ## # @Host.getStats: -- To view, visit http://gerrit.ovirt.org/9287 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ibd36a5c3a91abfa3fa29c811ce9b3586bf0bda36 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <agl(a)us.ibm.com>

7 17

Change in vdsm[master]: spec: Add MOM as a package dependency
by agl＠us.ibm.com 26 Nov '12

26 Nov '12

Adam Litke has uploaded a new change for review. Change subject: spec: Add MOM as a package dependency ...................................................................... spec: Add MOM as a package dependency For oVirt 3.2, we are switching to MOM for KSM tuning. One of the goals of this release is to ensure that MOM is running in all deployed oVirt hosts (ovirt-node and standalone). A recent enough version of MOM is now available upstream for Fedora and RHEL and we build nightly RPMs on ovirt.org. I think we are ready to add this packaging dependency now. Change-Id: I8e8c5a67066f270b8f19881ac5871ff844dbecaa Signed-off-by: Adam Litke <agl(a)us.ibm.com> --- M vdsm.spec.in 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/72/9272/1 diff --git a/vdsm.spec.in b/vdsm.spec.in index 20ad1dc..2fb7254 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -120,6 +120,7 @@ Requires: libselinux-python Requires: %{name}-python = %{version}-%{release} Requires: pyparted +Requires: mom <= 0.3.0 Requires(pre,preun): policycoreutils-python Requires(post): /usr/sbin/saslpasswd2 -- To view, visit http://gerrit.ovirt.org/9272 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8e8c5a67066f270b8f19881ac5871ff844dbecaa Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <agl(a)us.ibm.com>

5 18

Change in vdsm[master]: pep8: fix libvirtconnection.py
by Dan Kenigsberg 26 Nov '12

26 Nov '12

Dan Kenigsberg has uploaded a new change for review. Change subject: pep8: fix libvirtconnection.py ...................................................................... pep8: fix libvirtconnection.py Change-Id: Ie6fef13e80b2330dbc65c3440ceb994c5c6903b5 Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M Makefile.am M vdsm/libvirtconnection.py 2 files changed, 13 insertions(+), 6 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/37/9437/1 diff --git a/Makefile.am b/Makefile.am index 26ce402..a234428 100644 --- a/Makefile.am +++ b/Makefile.am @@ -67,6 +67,7 @@ vdsm/hooks.py \ vdsm/kaxmlrpclib.py \ vdsm/ksm.py \ + vdsm/libvirtconnection.py \ vdsm/libvirtev.py \ vdsm/libvirtvm.py \ vdsm/lsblk.py \ diff --git a/vdsm/libvirtconnection.py b/vdsm/libvirtconnection.py index 41ba933..faf0cd0 100644 --- a/vdsm/libvirtconnection.py +++ b/vdsm/libvirtconnection.py @@ -30,6 +30,7 @@ # TODO: make this internal to libvirtev, and make the thread stoppable libvirtev.virEventLoopPureStart() + def __eventCallback(conn, dom, *args): try: cif, eventid = args[-1] @@ -38,7 +39,7 @@ if not v: cif.log.debug('unknown vm %s eventid %s args %s', vmid, eventid, - args) + args) return if eventid == libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE: @@ -74,6 +75,8 @@ __connections = {} __connectionLock = threading.Lock() + + def get(cif=None): """Return current connection to libvirt or open a new one. @@ -91,8 +94,9 @@ setattr(ret, name, wrapMethod(method)) return ret except libvirt.libvirtError, e: - if (e.get_error_domain() in (libvirt.VIR_FROM_REMOTE, libvirt.VIR_FROM_RPC) - and e.get_error_code() == libvirt.VIR_ERR_SYSTEM_ERROR): + if (e.get_error_domain() in (libvirt.VIR_FROM_REMOTE, + libvirt.VIR_FROM_RPC) and + e.get_error_code() == libvirt.VIR_ERR_SYSTEM_ERROR): cif.log.error('connection to libvirt broken. ' 'taking vdsm down.') cif.prepareForShutdown() @@ -102,14 +106,16 @@ return wrapper def req(credentials, user_data): + passwd = file(constants.P_VDSM_LIBVIRT_PASSWD).readline().rstrip("\n") for cred in credentials: if cred[0] == libvirt.VIR_CRED_AUTHNAME: cred[4] = constants.SASL_USERNAME elif cred[0] == libvirt.VIR_CRED_PASSPHRASE: - cred[4] = file(constants.P_VDSM_LIBVIRT_PASSWD).readline().rstrip("\n") + cred[4] = passwd return 0 - auth = [[libvirt.VIR_CRED_AUTHNAME, libvirt.VIR_CRED_PASSPHRASE], req, None] + auth = [[libvirt.VIR_CRED_AUTHNAME, libvirt.VIR_CRED_PASSPHRASE], + req, None] with __connectionLock: conn = __connections.get(id(cif)) @@ -118,7 +124,7 @@ 'qemu:///system', auth, 0) conn = utils.retry(libvirtOpenAuth, timeout=10, sleep=0.2) __connections[id(cif)] = conn - if cif != None: + if cif is not None: for ev in (libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE, libvirt.VIR_DOMAIN_EVENT_ID_REBOOT, libvirt.VIR_DOMAIN_EVENT_ID_RTC_CHANGE, -- To view, visit http://gerrit.ovirt.org/9437 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie6fef13e80b2330dbc65c3440ceb994c5c6903b5 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

3 10

Change in vdsm[master]: pep8: have a space after all operators, not before colon
by Dan Kenigsberg 26 Nov '12

26 Nov '12

Dan Kenigsberg has uploaded a new change for review. Change subject: pep8: have a space after all operators, not before colon ...................................................................... pep8: have a space after all operators, not before colon Change-Id: I24a0b990395242dff75ed72b42dce944060e0b26 Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M vds_bootstrap/vds_bootstrap.py M vds_bootstrap/vds_bootstrap_complete.py M vdsm/dumpStorageTable.py.in M vdsm/sos/vdsm.py.in M vdsm/storage/blockSD.py M vdsm/storage/image.py M vdsm/storage/iscsi.py M vdsm/storage/resourceManager.py M vdsm/storage/sp.py M vdsm/storage/storage_mailbox.py M vdsm/storage/threadPool.py M vdsm_reg/deployUtil.py.in M vdsm_reg/engine.py.in 13 files changed, 131 insertions(+), 131 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/36/9436/1 diff --git a/vds_bootstrap/vds_bootstrap.py b/vds_bootstrap/vds_bootstrap.py index 867d6f4..502fb2f 100755 --- a/vds_bootstrap/vds_bootstrap.py +++ b/vds_bootstrap/vds_bootstrap.py @@ -149,7 +149,7 @@ MINIMAL_SUPPORTED_PLATFORM = "5.5" # Required packages -REQ_PACK = ('SDL.x86_64','bridge-utils.x86_64','mesa-libGLU.x86_64', +REQ_PACK = ('SDL.x86_64', 'bridge-utils.x86_64', 'mesa-libGLU.x86_64', 'openssl.x86_64', 'rsync.x86_64') if rhel6based: @@ -163,9 +163,9 @@ 'glusterfs-geo-replication') else: # Devel packages - DEVEL_PACK = ('gdb','tcpdump','strace','ltrace','sysstat','ntp', - 'pstack','vim-common','vim-enhanced', - 'systemtap','systemtap-runtime') + DEVEL_PACK = ('gdb', 'tcpdump', 'strace', 'ltrace', 'sysstat', 'ntp', + 'pstack', 'vim-common', 'vim-enhanced', + 'systemtap', 'systemtap-runtime') # VDS packages VDS_PACK = ('kvm', 'kmod-kvm', 'kvm-tools', VDSM_NAME, VDSM_NAME+'-cli', 'qcairo', 'qffmpeg-libs', 'qspice-libs', 'qpixman', 'log4cpp', @@ -494,7 +494,7 @@ self.req_pack.append(pack) for p in self.req_pack: - logging.debug('>>> %s should be installed',p) + logging.debug('>>> %s should be installed', p) logging.debug('Check VDS packages ...') for pack in VDS_PACK: self.res, self.message = deployUtil.getPackageInfo("VDS", pack, 'status') @@ -503,7 +503,7 @@ self.vds_pack.append(pack) for p in self.vds_pack: - logging.debug('>>> %s should be installed',p) + logging.debug('>>> %s should be installed', p) logging.debug('Check development packages ...') for pack in DEVEL_PACK: self.res, self.message = deployUtil.getPackageInfo("DEVEL", pack, 'status') @@ -512,11 +512,11 @@ self.devel_pack.append(pack) for p in self.devel_pack: - logging.debug('>>> %s should be installed',p) + logging.debug('>>> %s should be installed', p) def _installPackage(self, pack, type, update=0): nReturn = 0 - logging.debug('Installing %s %d',pack, update ) + logging.debug('Installing %s %d', pack, update ) if type == "REQ" or type == "DEVEL": self.res, self.message = deployUtil.installAndVerify(type, pack, "install") res = "OK" @@ -578,7 +578,7 @@ # install/update packages while self.req_pack: logging.debug('Install required packages ...') - self.rc = self._installPackage(self.req_pack.pop(),"REQ") + self.rc = self._installPackage(self.req_pack.pop(), "REQ") if self.rc: return @@ -587,17 +587,17 @@ logging.debug('Update VDS packages ... %s', self.vds_pack.__repr__()) for pack in VDS_PACK: if pack not in self.vds_pack: - self.rc = self._installPackage(pack,"VDS", 1) + self.rc = self._installPackage(pack, "VDS", 1) if self.rc: return while self.vds_pack: - self.rc = self._installPackage(self.vds_pack.pop(),"VDS") + self.rc = self._installPackage(self.vds_pack.pop(), "VDS") if self.rc: return while self.devel_pack: logging.debug('Install development packages ...') - self._installPackage(self.devel_pack.pop(),"DEVEL") + self._installPackage(self.devel_pack.pop(), "DEVEL") def packagesExplorer(self): """ @@ -1064,7 +1064,7 @@ installGlusterService = False bridgeName = None opts, args = getopt.getopt(sys.argv[1:], "v:r:O:t:f:S:n:u:B:Vg") - for o,v in opts: + for o, v in opts: if o == "-v": deployUtil.setBootstrapInterfaceVersion(int(v)) if o == "-r": diff --git a/vds_bootstrap/vds_bootstrap_complete.py b/vds_bootstrap/vds_bootstrap_complete.py index 7711d82..eadf74c 100755 --- a/vds_bootstrap/vds_bootstrap_complete.py +++ b/vds_bootstrap/vds_bootstrap_complete.py @@ -72,7 +72,7 @@ installVirtualizationService = True installGlusterService = False opts, args = getopt.getopt(sys.argv[1:], "v:c:Vg") - for o,v in opts: + for o, v in opts: if o == "-v": deployUtil.setBootstrapInterfaceVersion(int(v)) if o == "-c": diff --git a/vdsm/dumpStorageTable.py.in b/vdsm/dumpStorageTable.py.in index ea42ab0..d173e6e 100644 --- a/vdsm/dumpStorageTable.py.in +++ b/vdsm/dumpStorageTable.py.in @@ -48,17 +48,17 @@ if len(cont) < 2: continue - self.vmList[vmUUID] = {'Name':vmName, 'Refs': []} + self.vmList[vmUUID] = {'Name': vmName, 'Refs': []} for x in range(1, len(cont)): image = cont[x].split(' ')[0].split('/')[0] if not image in images: - sd = self.serverConncetion.getImageDomainsList(pool,image)['domainslist'] + sd = self.serverConncetion.getImageDomainsList(pool, image)['domainslist'] if len(sd) > 0: sd = sd[0] images.append(image) self._setSdInfo(sd) - disks.append([sd,image]) - self.vmList[vmUUID]['Refs'].append([sd,image]) + disks.append([sd, image]) + self.vmList[vmUUID]['Refs'].append([sd, image]) self._getImages(disks) @@ -66,7 +66,7 @@ if not sd in self.domainsList.keys(): sd_info = self.serverConncetion.getStorageDomainInfo(sd)['info'] - self.domainsList[sd] = {'Name':sd_info['name'], 'Role':sd_info['role'], 'Refs':[]} + self.domainsList[sd] = {'Name': sd_info['name'], 'Role': sd_info['role'], 'Refs': []} def _getSds(self, pool): @@ -109,7 +109,7 @@ for vol in volumes['uuidlist']: res = self.serverConncetion.getVolumeInfo(sd, pool, image, vol) volParents[vol] = res['info']['parent'] - imageDomVol = '%s:%s:%s' % (sd, image,vol) + imageDomVol = '%s:%s:%s' % (sd, image, vol) self.volumesList[imageDomVol] = [] if res['info']['voltype'] == 'LEAF': @@ -119,7 +119,7 @@ leafvol = vol self.volumesList[imageDomVol].append('Template') - self.imagesList[imageDom] = self._buildVolumesChain(leafvol,vollen,volParents, sd, image) + self.imagesList[imageDom] = self._buildVolumesChain(leafvol, vollen, volParents, sd, image) def _buildVolumesChain(self, leafvol, vollen, volParents, sd, image): @@ -127,7 +127,7 @@ volChain[(vollen-1)]=leafvol next = leafvol - for k in range(1,(vollen+1)): + for k in range(1, (vollen+1)): prev = next next = volParents[prev] if next in volParents.keys(): @@ -138,7 +138,7 @@ else: app = '%s' % (next) - imageDomVol = '%s:%s:%s' % (sd, image,prev) + imageDomVol = '%s:%s:%s' % (sd, image, prev) templList = self.volumesList[imageDomVol] if len(templList) == 0: templList.append(app) diff --git a/vdsm/sos/vdsm.py.in b/vdsm/sos/vdsm.py.in index 031e2a3..5f09f8d 100644 --- a/vdsm/sos/vdsm.py.in +++ b/vdsm/sos/vdsm.py.in @@ -45,7 +45,7 @@ optionList = [("logsize", 'max size (MiB) to collect per log file', '', False)] - def __addCopySpecLogLimit(self,path, logsize=None): + def __addCopySpecLogLimit(self, path, logsize=None): """ Glob the files in path and ensure that the sorted cumulative list of globbed files isn't larger than logsize in MiB if diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index ce45da7..bb98b60 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -108,10 +108,10 @@ # TBD: Do we really need this key? BLOCK_SD_MD_FIELDS.update({ # Key dec, enc - DMDK_PV_REGEX : (decodePVInfo, encodePVInfo), - DMDK_VGUUID : (str, str), - DMDK_LOGBLKSIZE : (functools.partial(sd.intOrDefault, DEFAULT_BLOCKSIZE), str), - DMDK_PHYBLKSIZE : (functools.partial(sd.intOrDefault, DEFAULT_BLOCKSIZE), str), + DMDK_PV_REGEX: (decodePVInfo, encodePVInfo), + DMDK_VGUUID: (str, str), + DMDK_LOGBLKSIZE: (functools.partial(sd.intOrDefault, DEFAULT_BLOCKSIZE), str), + DMDK_PHYBLKSIZE: (functools.partial(sd.intOrDefault, DEFAULT_BLOCKSIZE), str), }) INVALID_CHARS = re.compile(r"[^a-zA-Z0-9_+.\-/=!:#]") @@ -341,8 +341,8 @@ f.seek(self._offset) f.write(data) -LvBasedSDMetadata = lambda vg, lv : DictValidator(PersistentDict(LvMetadataRW(vg, lv, 0, SD_METADATA_SIZE)), BLOCK_SD_MD_FIELDS) -TagBasedSDMetadata = lambda vg : DictValidator(PersistentDict(VGTagMetadataRW(vg)), BLOCK_SD_MD_FIELDS) +LvBasedSDMetadata = lambda vg, lv: DictValidator(PersistentDict(LvMetadataRW(vg, lv, 0, SD_METADATA_SIZE)), BLOCK_SD_MD_FIELDS) +TagBasedSDMetadata = lambda vg: DictValidator(PersistentDict(VGTagMetadataRW(vg)), BLOCK_SD_MD_FIELDS) def selectMetadata(sdUUID): mdProvider = LvBasedSDMetadata(sdUUID, sd.METADATA) @@ -358,7 +358,7 @@ mdathreshold - False if the VG's metadata exceeded its threshold, else True mdavalid - False if the VG's metadata size too small, else True """ - mdaStatus = {'mdavalid':True, 'mdathreshold':True} + mdaStatus = {'mdavalid': True, 'mdathreshold': True} mda_size = int(vg.vg_mda_size) mda_free = int(vg.vg_mda_free) @@ -477,7 +477,7 @@ numOfPVs = len(lvm.listPVNames(vgName)) if version in VERS_METADATA_LV and numOfPVs > MAX_PVS: - cls.log.debug("%d > %d" , numOfPVs, MAX_PVS) + cls.log.debug("%d > %d", numOfPVs, MAX_PVS) raise se.StorageDomainIsMadeFromTooManyPVs() # Create metadata service volume @@ -528,21 +528,21 @@ # Do we really need to keep the VGUUID? # no one reads it from here anyway initialMetadata = { - sd.DMDK_VERSION : version, - sd.DMDK_SDUUID : sdUUID, - sd.DMDK_TYPE : storageType, - sd.DMDK_CLASS : domClass, - sd.DMDK_DESCRIPTION : domainName, - sd.DMDK_ROLE : sd.REGULAR_DOMAIN, - sd.DMDK_POOLS : [], - sd.DMDK_LOCK_POLICY : '', - sd.DMDK_LOCK_RENEWAL_INTERVAL_SEC : sd.DEFAULT_LEASE_PARAMS[sd.DMDK_LOCK_RENEWAL_INTERVAL_SEC], - sd.DMDK_LEASE_TIME_SEC : sd.DEFAULT_LEASE_PARAMS[sd.DMDK_LOCK_RENEWAL_INTERVAL_SEC], - sd.DMDK_IO_OP_TIMEOUT_SEC : sd.DEFAULT_LEASE_PARAMS[sd.DMDK_IO_OP_TIMEOUT_SEC], - sd.DMDK_LEASE_RETRIES : sd.DEFAULT_LEASE_PARAMS[sd.DMDK_LEASE_RETRIES], - DMDK_VGUUID : vgUUID, - DMDK_LOGBLKSIZE : logBlkSize, - DMDK_PHYBLKSIZE : phyBlkSize, + sd.DMDK_VERSION: version, + sd.DMDK_SDUUID: sdUUID, + sd.DMDK_TYPE: storageType, + sd.DMDK_CLASS: domClass, + sd.DMDK_DESCRIPTION: domainName, + sd.DMDK_ROLE: sd.REGULAR_DOMAIN, + sd.DMDK_POOLS: [], + sd.DMDK_LOCK_POLICY: '', + sd.DMDK_LOCK_RENEWAL_INTERVAL_SEC: sd.DEFAULT_LEASE_PARAMS[sd.DMDK_LOCK_RENEWAL_INTERVAL_SEC], + sd.DMDK_LEASE_TIME_SEC: sd.DEFAULT_LEASE_PARAMS[sd.DMDK_LOCK_RENEWAL_INTERVAL_SEC], + sd.DMDK_IO_OP_TIMEOUT_SEC: sd.DEFAULT_LEASE_PARAMS[sd.DMDK_IO_OP_TIMEOUT_SEC], + sd.DMDK_LEASE_RETRIES: sd.DEFAULT_LEASE_PARAMS[sd.DMDK_LEASE_RETRIES], + DMDK_VGUUID: vgUUID, + DMDK_LOGBLKSIZE: logBlkSize, + DMDK_PHYBLKSIZE: phyBlkSize, } initialMetadata.update(mapping) @@ -711,7 +711,7 @@ return self.getFreeMetadataSlot(slotSize) def _getOccupiedMetadataSlots(self): - stripPrefix = lambda s, pfx : s[len(pfx):] + stripPrefix = lambda s, pfx: s[len(pfx):] occupiedSlots = [] for lv in lvm.getLV(self.sdUUID): if lv.name in SPECIAL_LVS: diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 0265dcc..a098489 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -48,9 +48,9 @@ SWAP_DISK_TYPE = 4 TEMP_DISK_TYPE = 5 -DISK_TYPES = {UNKNOWN_DISK_TYPE:'UNKNOWN', SYSTEM_DISK_TYPE:'SYSTEM', - DATA_DISK_TYPE:'DATA', SHARED_DISK_TYPE:'SHARED', - SWAP_DISK_TYPE:'SWAP', TEMP_DISK_TYPE:'TEMP'} +DISK_TYPES = {UNKNOWN_DISK_TYPE: 'UNKNOWN', SYSTEM_DISK_TYPE: 'SYSTEM', + DATA_DISK_TYPE: 'DATA', SHARED_DISK_TYPE: 'SHARED', + SWAP_DISK_TYPE: 'SWAP', TEMP_DISK_TYPE: 'TEMP'} # What volumes to synchronize SYNC_VOLUMES_ALL = 'ALL' @@ -61,7 +61,7 @@ UNKNOWN_OP = 0 COPY_OP = 1 MOVE_OP = 2 -OP_TYPES = {UNKNOWN_OP:'UNKNOWN', COPY_OP:'COPY', MOVE_OP:'MOVE'} +OP_TYPES = {UNKNOWN_OP: 'UNKNOWN', COPY_OP: 'COPY', MOVE_OP: 'MOVE'} RENAME_RANDOM_STRING_LEN = 8 @@ -562,7 +562,7 @@ # only base may have a different parent image pimg = imgUUID - return {'srcChain':srcChain, 'dstChain':dstChain} + return {'srcChain': srcChain, 'dstChain': dstChain} def _interImagesCopy(self, destDom, srcSdUUID, imgUUID, chains): srcLeafVol = chains['srcChain'][-1] diff --git a/vdsm/storage/iscsi.py b/vdsm/storage/iscsi.py index a0c3067..aaf1237 100644 --- a/vdsm/storage/iscsi.py +++ b/vdsm/storage/iscsi.py @@ -429,7 +429,7 @@ def getDefaultInitiatorName(): with open("/etc/iscsi/initiatorname.iscsi", "r") as f: - return f.read().strip().split("=",1)[1] + return f.read().strip().split("=", 1)[1] def findUnderlyingStorage(devPath): diff --git a/vdsm/storage/resourceManager.py b/vdsm/storage/resourceManager.py index ef544eb..b343566 100644 --- a/vdsm/storage/resourceManager.py +++ b/vdsm/storage/resourceManager.py @@ -150,11 +150,11 @@ Internal request object, don't use directly """ _log = logging.getLogger("ResourceManager.Request") - namespace = property(lambda self : self._namespace) - name = property(lambda self : self._name) - fullName = property(lambda self : "%s.%s" % (self._namespace, self._name)) - lockType = property(lambda self : self._lockType) - syncRoot = property(lambda self : self._syncRoot) + namespace = property(lambda self: self._namespace) + name = property(lambda self: self._name) + fullName = property(lambda self: "%s.%s" % (self._namespace, self._name)) + lockType = property(lambda self: self._lockType) + syncRoot = property(lambda self: self._syncRoot) def __init__(self, namespace, name, lockType, callback): self._syncRoot = threading.RLock() @@ -166,7 +166,7 @@ self._doneEvent = threading.Event() self._callback = callback self.reqID = str(uuid4()) - self._log = SimpleLogAdapter(self._log, {"ResName" : self.fullName, "ReqID" : self.reqID}) + self._log = SimpleLogAdapter(self._log, {"ResName": self.fullName, "ReqID": self.reqID}) # Because findCaller is expensive. We make sure it wll be printed before # calculating it @@ -237,17 +237,17 @@ This object will auto release the referenced resource unless autorelease is set to `False` """ _log = logging.getLogger("ResourceManager.ResourceRef") - namespace = property(lambda self : self._namespace) - name = property(lambda self : self._name) - fullName = property(lambda self : "%s.%s" % (self._namespace, self._name)) + namespace = property(lambda self: self._namespace) + name = property(lambda self: self._name) + fullName = property(lambda self: "%s.%s" % (self._namespace, self._name)) # States whether this reference is pointing to an owned reference - isValid = property(lambda self : self._isValid) + isValid = property(lambda self: self._isValid) def __init__(self, namespace, name, wrappedObject=None, resRefID=str(uuid4())): self._namespace = namespace self._name = name - self._log = SimpleLogAdapter(self._log, {"ResName" : self.fullName, "ResRefID" : resRefID}) + self._log = SimpleLogAdapter(self._log, {"ResName": self.fullName, "ResRefID": resRefID}) self.__wrappedObject = wrappedObject if wrappedObject is not None: @@ -259,7 +259,7 @@ def __wrapObj(self): for attr in dir(self.__wrappedObject): - if hasattr(self,attr) or attr in ('close', 'switchLockType'): + if hasattr(self, attr) or attr in ('close', 'switchLockType'): continue setattr(self, attr, partial(self.__methodProxy, attr)) diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index ac5e1db..72443c6 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -86,11 +86,11 @@ SP_MD_FIELDS = { # Key dec, enc - PMDK_DOMAINS : (domainListDecoder, domainListEncoder), - PMDK_POOL_DESCRIPTION : (unicodeDecoder, unicodeEncoder), - PMDK_LVER : (int, str), - PMDK_SPM_ID : (int, str), - PMDK_MASTER_VER : (int, str) + PMDK_DOMAINS: (domainListDecoder, domainListEncoder), + PMDK_POOL_DESCRIPTION: (unicodeDecoder, unicodeEncoder), + PMDK_LVER: (int, str), + PMDK_SPM_ID: (int, str), + PMDK_MASTER_VER: (int, str) } # Calculate how many domains can be in the pool before overflowing the Metadata @@ -1368,24 +1368,24 @@ disktotal, diskfree = st.diskUtilization vgmdtotal, vgmdfree = st.vgMdUtilization res[sdUUID] = { - 'finish' : st.lastCheck, - 'result' : { - 'code' : code, - 'lastCheck' : '%.1f' % (now - st.lastCheck), - 'delay' : str(st.readDelay), - 'valid' : (st.error is None) + 'finish': st.lastCheck, + 'result': { + 'code': code, + 'lastCheck': '%.1f' % (now - st.lastCheck), + 'delay': str(st.readDelay), + 'valid': (st.error is None) }, - 'disktotal' : disktotal, - 'diskfree' : diskfree, + 'disktotal': disktotal, + 'diskfree': diskfree, - 'mdavalid' : st.vgMdHasEnoughFreeSpace, - 'mdathreshold' : st.vgMdFreeBelowThreashold, - 'mdasize' : vgmdtotal, - 'mdafree' : vgmdfree, + 'mdavalid': st.vgMdHasEnoughFreeSpace, + 'mdathreshold': st.vgMdFreeBelowThreashold, + 'mdasize': vgmdtotal, + 'mdafree': vgmdfree, - 'masterValidate' : { - 'mount' : st.masterMounted, - 'valid' : st.masterValid + 'masterValidate': { + 'mount': st.masterMounted, + 'valid': st.masterValid } } return res @@ -1449,14 +1449,14 @@ stats['disktotal'] = repoStats[item]['disktotal'] stats['diskfree'] = repoStats[item]['diskfree'] if not repoStats[item]['mdavalid']: - alerts.append({'code':se.SmallVgMetadata.code, - 'message':se.SmallVgMetadata.message}) + alerts.append({'code': se.SmallVgMetadata.code, + 'message': se.SmallVgMetadata.message}) self.log.warning("VG %s's metadata size too small %s", dom.sdUUID, repoStats[item]['mdasize']) if not repoStats[item]['mdathreshold']: - alerts.append({'code':se.VgMetadataCriticallyFull.code, - 'message':se.VgMetadataCriticallyFull.message}) + alerts.append({'code': se.VgMetadataCriticallyFull.code, + 'message': se.VgMetadataCriticallyFull.message}) self.log.warning("VG %s's metadata size exceeded critical size: \ mdasize=%s mdafree=%s", dom.sdUUID, repoStats[item]['mdasize'], repoStats[item]['mdafree']) @@ -1479,7 +1479,7 @@ for item in domDict: try: dom = sdCache.produce(item) - except se.StorageDomainDoesNotExist : + except se.StorageDomainDoesNotExist: self.log.warn("Storage domain %s does not exist", item) continue diff --git a/vdsm/storage/storage_mailbox.py b/vdsm/storage/storage_mailbox.py index f678a04..c74fccc 100644 --- a/vdsm/storage/storage_mailbox.py +++ b/vdsm/storage/storage_mailbox.py @@ -57,7 +57,7 @@ MESSAGES_PER_MAILBOX = SLOTS_PER_MAILBOX - 1 # Last message slot is reserved for metadata (checksum, extendable mailbox, etc) _zeroCheck = misc.checksum(EMPTYMAILBOX, CHECKSUM_BYTES) -pZeroChecksum = struct.pack('<l',_zeroCheck) # Assumes CHECKSUM_BYTES equals 4!!! +pZeroChecksum = struct.pack('<l', _zeroCheck) # Assumes CHECKSUM_BYTES equals 4!!! def dec2hex(n): return "%x" % n @@ -102,9 +102,9 @@ domain = misc.packUuid(volumeData['domainID']) volume = misc.packUuid(volumeData['volumeID']) # Build base payload - payload = MESSAGE_VERSION + EXTEND_CODE + domain + volume + self.newSize.rjust(SIZE_CHARS,"0") + payload = MESSAGE_VERSION + EXTEND_CODE + domain + volume + self.newSize.rjust(SIZE_CHARS, "0") # Pad payload with zeros - self.payload = payload.ljust(MESSAGE_SIZE,"0") + self.payload = payload.ljust(MESSAGE_SIZE, "0") self.log.debug('new extend msg created: domain: %s, volume: %s', volumeData['domainID'], volumeData['volumeID']) @@ -135,8 +135,8 @@ volume = {} volume['poolID'] = pool.spUUID - volume['domainID'] = misc.unpackUuid(payload[sdOffset : sdOffset+PACKED_UUID_SIZE]) - volume['volumeID'] = misc.unpackUuid(payload[volumeOffset : volumeOffset+PACKED_UUID_SIZE]) + volume['domainID'] = misc.unpackUuid(payload[sdOffset: sdOffset+PACKED_UUID_SIZE]) + volume['volumeID'] = misc.unpackUuid(payload[volumeOffset: volumeOffset+PACKED_UUID_SIZE]) size = int(payload[sizeOffset:sizeOffset + SIZE_CHARS], 16) cls.log.info("processRequest: extending volume %s " @@ -289,18 +289,18 @@ # rc = True - newMsg = newMsgs[start : start + MESSAGE_SIZE] + newMsg = newMsgs[start: start + MESSAGE_SIZE] if newMsg == CLEAN_MESSAGE: del self._activeMessages[i] self._used_slots_array[i] = 0 self._msgCounter -= 1 - self._outgoingMail = self._outgoingMail[0 : start] + MESSAGE_SIZE * "\0" + self._outgoingMail[start + MESSAGE_SIZE : ] + self._outgoingMail = self._outgoingMail[0: start] + MESSAGE_SIZE * "\0" + self._outgoingMail[start + MESSAGE_SIZE: ] continue msg = self._activeMessages[i] self._activeMessages[i] = CLEAN_MESSAGE - self._outgoingMail = self._outgoingMail[0 : start] + CLEAN_MESSAGE + self._outgoingMail[start + MESSAGE_SIZE : ] + self._outgoingMail = self._outgoingMail[0: start] + CLEAN_MESSAGE + self._outgoingMail[start + MESSAGE_SIZE: ] try: self.log.debug("HSM_MailboxMonitor(%s/%s) - Checking reply: %s", self._msgCounter, MESSAGES_PER_MAILBOX, repr(newMsg)) @@ -338,9 +338,9 @@ def _sendMail(self): self.log.info("HSM_MailMonitor sending mail to SPM - " + str(self._outCmd)) - chk = misc.checksum(self._outgoingMail[0 : MAILBOX_SIZE-CHECKSUM_BYTES], CHECKSUM_BYTES) - pChk = struct.pack('<l',chk) # Assumes CHECKSUM_BYTES equals 4!!! - self._outgoingMail = self._outgoingMail[0 : MAILBOX_SIZE-CHECKSUM_BYTES] + pChk + chk = misc.checksum(self._outgoingMail[0: MAILBOX_SIZE-CHECKSUM_BYTES], CHECKSUM_BYTES) + pChk = struct.pack('<l', chk) # Assumes CHECKSUM_BYTES equals 4!!! + self._outgoingMail = self._outgoingMail[0: MAILBOX_SIZE-CHECKSUM_BYTES] + pChk misc.execCmd(self._outCmd, data=self._outgoingMail, sudo=False) @@ -393,7 +393,7 @@ try: #self.log.debug("No requests in queue, going to sleep until new requests arrive") # Check if a new message is waiting to be sent - message = self._queue.get(block=True,timeout=self._monitorInterval) + message = self._queue.get(block=True, timeout=self._monitorInterval) self._handleMessage(message) message = None sendMail = True @@ -542,9 +542,9 @@ def _validateMailbox(self, mailbox, mailboxIndex): chkStart = MAILBOX_SIZE-CHECKSUM_BYTES - chk = misc.checksum(mailbox[0 : chkStart], CHECKSUM_BYTES) - pChk = struct.pack('<l',chk) # Assumes CHECKSUM_BYTES equals 4!!! - if pChk != mailbox[chkStart : chkStart+CHECKSUM_BYTES]: + chk = misc.checksum(mailbox[0: chkStart], CHECKSUM_BYTES) + pChk = struct.pack('<l', chk) # Assumes CHECKSUM_BYTES equals 4!!! + if pChk != mailbox[chkStart: chkStart+CHECKSUM_BYTES]: self.log.error("SPM_MailMonitor: mailbox %s checksum failed, not clearing mailbox, clearing newMail.", str(mailboxIndex)) return False elif pChk == pZeroChecksum: return False # Ignore messages of empty mailbox @@ -573,7 +573,7 @@ # Most mailboxes are probably empty so it costs less to check that all messages start with 0 than # to validate the mailbox, therefor this is done after we find a non empty message in mailbox if not isMailboxValidated: - if not self._validateMailbox(newMail[mailboxStart : mailboxStart + MAILBOX_SIZE], host): + if not self._validateMailbox(newMail[mailboxStart: mailboxStart + MAILBOX_SIZE], host): #Cleaning invalid mbx in newMail newMail = newMail[:mailboxStart] + EMPTYMAILBOX + newMail[mailboxStart + MAILBOX_SIZE:] break @@ -581,13 +581,13 @@ isMailboxValidated = True - newMsg = newMail[msgStart : msgStart+MESSAGE_SIZE] + newMsg = newMail[msgStart: msgStart+MESSAGE_SIZE] msgOffset = msgId * MESSAGE_SIZE if newMsg == CLEAN_MESSAGE: # Should probably put a setter on outgoingMail which would take the lock self._outLock.acquire() try: - self._outgoingMail = self._outgoingMail[0:msgOffset] + CLEAN_MESSAGE + self._outgoingMail[msgOffset+MESSAGE_SIZE : self._outMailLen] + self._outgoingMail = self._outgoingMail[0:msgOffset] + CLEAN_MESSAGE + self._outgoingMail[msgOffset+MESSAGE_SIZE: self._outMailLen] finally: self._outLock.release() send = True @@ -605,14 +605,14 @@ # We only get here if there is a novel request try: - msgType = newMail[msgStart+1 : msgStart+5] + msgType = newMail[msgStart+1: msgStart+5] if msgType in self._messageTypes: # Use message class to process request according to message specific logic id = str(uuid.uuid4()) - self.log.debug("SPM_MailMonitor: processing request: %s" % repr(newMail[msgStart : msgStart+MESSAGE_SIZE])) + self.log.debug("SPM_MailMonitor: processing request: %s" % repr(newMail[msgStart: msgStart+MESSAGE_SIZE])) res = self.tp.queueTask(id, runTask, (self._messageTypes[msgType], msgId, - newMail[msgStart : msgStart+MESSAGE_SIZE]) + newMail[msgStart: msgStart+MESSAGE_SIZE]) ) if not res: raise Exception() @@ -663,9 +663,9 @@ self._outLock.acquire() try: msgOffset = msgID*MESSAGE_SIZE - self._outgoingMail = self._outgoingMail[0:msgOffset] + msg.payload + self._outgoingMail[msgOffset+MESSAGE_SIZE : self._outMailLen] + self._outgoingMail = self._outgoingMail[0:msgOffset] + msg.payload + self._outgoingMail[msgOffset+MESSAGE_SIZE: self._outMailLen] mailboxOffset = (msgID / SLOTS_PER_MAILBOX) * MAILBOX_SIZE - mailbox = self._outgoingMail[mailboxOffset : mailboxOffset + MAILBOX_SIZE] + mailbox = self._outgoingMail[mailboxOffset: mailboxOffset + MAILBOX_SIZE] cmd = self._outCmd + ['bs=' + str(MAILBOX_SIZE), 'seek=' + str(mailboxOffset / MAILBOX_SIZE)] #self.log.debug("Running command: %s, for message id: %s", str(cmd), str(msgID)) (rc, out, err) = misc.execCmd(cmd, sudo=False, data=mailbox) diff --git a/vdsm/storage/threadPool.py b/vdsm/storage/threadPool.py index 22633b7..7edaeb2 100644 --- a/vdsm/storage/threadPool.py +++ b/vdsm/storage/threadPool.py @@ -252,13 +252,13 @@ pool = ThreadPool(100) # Insert tasks into the queue and let them run - print "Running tasks: ", pool.getRunningTasks(),"\n" + print "Running tasks: ", pool.getRunningTasks(), "\n" pool.queueTask(sortTask, (1000, 100000), taskCallback) - print "Running tasks: ", pool.getRunningTasks(),"\n" + print "Running tasks: ", pool.getRunningTasks(), "\n" pool.queueTask(waitTask, 5, taskCallback) pool.queueTask(sortTask, (200, 200000), taskCallback) pool.queueTask(waitTask, 2, taskCallback) - print "Running tasks: ", pool.getRunningTasks(),"\n" + print "Running tasks: ", pool.getRunningTasks(), "\n" pool.queueTask(sortTask, (3, 30000), taskCallback) pool.queueTask(waitTask, 7, taskCallback) diff --git a/vdsm_reg/deployUtil.py.in b/vdsm_reg/deployUtil.py.in index 0fe1275..6613ad3 100644 --- a/vdsm_reg/deployUtil.py.in +++ b/vdsm_reg/deployUtil.py.in @@ -90,8 +90,8 @@ CORE_DUMP_PATH = '/var/lib/vdsm/core' CORE_PATTERN = '/proc/sys/kernel/core_pattern' XML_QUOTES = { - "\n":' ', - "'":'~' + "\n": ' ', + "'": '~' } # Regular expression used to validate content of SSH public keys: @@ -136,7 +136,7 @@ if input is not None: logging.debug(input) stdin = subprocess.PIPE - p = subprocess.Popen(argv , stdin=stdin, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + p = subprocess.Popen(argv, stdin=stdin, stdout=subprocess.PIPE, stderr=subprocess.PIPE) out, err = p.communicate(input) rc = p.returncode logging.debug(out) @@ -340,7 +340,7 @@ elif arch == "ppc64": if os.path.exists('/proc/device-tree/system-id'): #eg. output IBM,03061C14A - return file('/proc/device-tree/system-id').readline().replace(",","") + return file('/proc/device-tree/system-id').readline().replace(",", "") logging.error("getMachineUUID: Could not find machine's UUID.") @@ -810,12 +810,12 @@ #Create temp file fh, abs_path = mkstemp() - new_file = open(abs_path,'w') + new_file = open(abs_path, 'w') old_file = open(fileName) #Iterate over the existing file, while replacing if needed. for line in old_file: #Note: the line below must not have spaces before or after the '=', since it'll fail bash scripts ! - newLine = re.sub('^[#\s]*%s\s*=.*' % key , '%s=%s' % (key, value), line) + newLine = re.sub('^[#\s]*%s\s*=.*' % key, '%s=%s' % (key, value), line) if re.match('^%s=%s$' % (re.escape(key), re.escape(value)), newLine): logging.debug("_updateFileLine: replacing " + str(line) + " with: " + newLine) fReplaced = True @@ -1023,7 +1023,7 @@ if fReturn: try: lstBridgeOptions.append('blockingdhcp=true') - out, err, ret = _logExec([os.path.join(vdsmDir, SCRIPT_NAME_ADD) , bridgeName, vlan, bonding, nic] + lstBridgeOptions) + out, err, ret = _logExec([os.path.join(vdsmDir, SCRIPT_NAME_ADD), bridgeName, vlan, bonding, nic] + lstBridgeOptions) if ret: raise Exception('Failed to add bridge') except: diff --git a/vdsm_reg/engine.py.in b/vdsm_reg/engine.py.in index f29d129..d03d978 100644 --- a/vdsm_reg/engine.py.in +++ b/vdsm_reg/engine.py.in @@ -133,7 +133,7 @@ config = {} config["vdc_host_port"] = VDC_HOST_PORT for line in vdsm_config: - line = line.strip().replace(" ","").split("=") + line = line.strip().replace(" ", "").split("=") if "vdc_host_name" in line: item, config["vdc_host_name"] = line[0], line[1] if "vdc_host_port" in line: @@ -161,7 +161,7 @@ self.ncs.screen.setColor(customColorset(1), "black", "magenta") heading.setColors(customColorset(1)) elements.setField(heading, 0, 0, anchorLeft = 1) - engine_grid = Grid(2,2) + engine_grid = Grid(2, 2) engine_grid.setField(Label("Management Server:"), 0, 0, anchorLeft = 1) self.engine_server = Entry(25, "") self.engine_server.setCallback(self.validEngineServerCallback) @@ -170,14 +170,14 @@ self.engine_server_port.setCallback(self.validEngineServerPortCallback) engine_grid.setField(self.engine_server, 1, 0, anchorLeft = 1, padding=(2, 0, 0, 1)) engine_grid.setField(self.engine_server_port, 1, 1, anchorLeft = 1, padding=(2, 0, 0, 0)) - elements.setField(engine_grid, 0, 1, anchorLeft = 1, padding = (0,0,0,0)) + elements.setField(engine_grid, 0, 1, anchorLeft = 1, padding = (0, 0, 0, 0)) elements.setField(Label(""), 0, 2, anchorLeft = 1) self.verify_engine_cert = Checkbox("Connect to @ENGINENAME@ and Validate Certificate", isOn=True) - elements.setField(self.verify_engine_cert, 0, 3, anchorLeft = 1, padding = (0,0,0,0)) + elements.setField(self.verify_engine_cert, 0, 3, anchorLeft = 1, padding = (0, 0, 0, 0)) elements.setField(Label(""), 0, 4, anchorLeft = 1) elements.setField(Label("Optional password for adding node through @ENGINENAME@ UI"), 0, 5, anchorLeft = 1) - pw_elements = Grid(3,3) + pw_elements = Grid(3, 3) pw_elements.setField(Label("Password: "), 0, 1, anchorLeft = 1) self.root_password_1 = Entry(15, password=1) @@ -192,7 +192,7 @@ self.pw_resp = PASSWORD_MATCH elements.setField(pw_elements, 0, 6, anchorLeft=1) - elements.setField(self.pw_msg, 0, 7, padding = (0,0,0,0)) + elements.setField(self.pw_msg, 0, 7, padding = (0, 0, 0, 0)) inputFields = [self.engine_server, self.engine_server_port, self.verify_engine_cert, self.root_password_1, self.root_password_2] @@ -202,7 +202,7 @@ try: engine_server = getEngineConfig() - engine_server,engine_port = engine_server.split(":") + engine_server, engine_port = engine_server.split(":") if engine_server.startswith("None"): self.engine_server.set("") else: -- To view, visit http://gerrit.ovirt.org/9436 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I24a0b990395242dff75ed72b42dce944060e0b26 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

3 7

Change in vdsm[master]: Fixing pep8 error that rises with rhel6.3 pep8 version
by ybronhei＠redhat.com 26 Nov '12

26 Nov '12

Yaniv Bronhaim has uploaded a new change for review. Change subject: Fixing pep8 error that rises with rhel6.3 pep8 version ...................................................................... Fixing pep8 error that rises with rhel6.3 pep8 version deprecated form of raising exception error Change-Id: I7b973beec494f452d66df646b2af6b2f9e8e15bf Signed-off-by: Yaniv Bronhaim <ybronhei(a)redhat.com> --- M vdsm/storage/misc.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/51/9451/1 diff --git a/vdsm/storage/misc.py b/vdsm/storage/misc.py index 6360dcf..bc32590 100644 --- a/vdsm/storage/misc.py +++ b/vdsm/storage/misc.py @@ -738,7 +738,7 @@ # re-raise the earliest exception if firstException is not None: - raise firstException, None, traceback + raise (firstException, None, traceback) def defer(self, func, *args, **kwargs): self._finally.append((func, args, kwargs)) -- To view, visit http://gerrit.ovirt.org/9451 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7b973beec494f452d66df646b2af6b2f9e8e15bf Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim <ybronhei(a)redhat.com>

3 5

Change in vdsm[master]: nwfilter, vmChannels: pep8 fixes
by Dan Kenigsberg 26 Nov '12

26 Nov '12

Dan Kenigsberg has uploaded a new change for review. Change subject: nwfilter, vmChannels: pep8 fixes ...................................................................... nwfilter, vmChannels: pep8 fixes Change-Id: Icf34f7d99d969cc08e3aa2da779769ae796300d1 Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M Makefile.am M vdsm/nwfilter.py M vdsm/vmChannels.py 3 files changed, 12 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/34/9434/1 diff --git a/Makefile.am b/Makefile.am index e15d709..26ce402 100644 --- a/Makefile.am +++ b/Makefile.am @@ -74,6 +74,7 @@ vdsm/momIF.py \ vdsm/neterrors.py \ vdsm/netinfo.py \ + vdsm/nwfilter.py \ vdsm/parted_utils.py \ vdsm/rest/BindingREST.py \ vdsm/rest/Controller.py \ @@ -110,6 +111,7 @@ vdsm/vdsm \ vdsm/vdsmDebugPlugin.py \ vdsm/vm.py \ + vdsm/vmChannels.py \ vdsm_api \ vdsm_cli \ vdsm_hooks/fileinject \ diff --git a/vdsm/nwfilter.py b/vdsm/nwfilter.py index 4129f83..7fcff29 100755 --- a/vdsm/nwfilter.py +++ b/vdsm/nwfilter.py @@ -25,6 +25,7 @@ from vdsm import libvirtconnection + def main(): """ Defines network filters on libvirt @@ -33,6 +34,7 @@ NoMacSpoofingFilter().defineNwFilter(conn) conn.close() + class NwFilter(object): """ Base class for custom network filters diff --git a/vdsm/vmChannels.py b/vdsm/vmChannels.py index 92f67a0..5879afa 100644 --- a/vdsm/vmChannels.py +++ b/vdsm/vmChannels.py @@ -22,6 +22,7 @@ import time import select + class Listener(threading.Thread): """ An events driven listener which handle messages from virtual machines. @@ -68,7 +69,7 @@ """ Add new channels to unconnected channels list. """ for (fileno, obj) in self._add_channels.items(): self.log.debug("fileno %d was added to unconnected channels.", - fileno) + fileno) self._unconnected[fileno] = obj self._add_channels.clear() @@ -139,13 +140,16 @@ self.log.info("Setting channels' timeout to %d seconds.", seconds) self._timeout = seconds - def register(self, fileno, connect_callback, read_callback, timeout_callback, opaque): + def register(self, fileno, connect_callback, read_callback, + timeout_callback, opaque): """ Register a new file descriptor to the listener. """ self.log.debug("Add fileno %d to listener's channels.", fileno) with self._update_lock: - self._add_channels[fileno] = { 'connect_cb': connect_callback, + self._add_channels[fileno] = { + 'connect_cb': connect_callback, 'read_cb': read_callback, 'timeout_cb': timeout_callback, - 'opaque': opaque, 'read_time': 0.0 } + 'opaque': opaque, 'read_time': 0.0, + } def unregister(self, fileno): """ Unregister an exist file descriptor from the listener. """ -- To view, visit http://gerrit.ovirt.org/9434 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Icf34f7d99d969cc08e3aa2da779769ae796300d1 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

3 6

Change in vdsm[master]: Allow break bond with attached network while VM is running.
by Igor Lvovsky 22 Nov '12

22 Nov '12

Igor Lvovsky has uploaded a new change for review. Change subject: Allow break bond with attached network while VM is running. ...................................................................... Allow break bond with attached network while VM is running. The problem was that we counted tap device on the bridge as part of bond because of wrong behaviour of getNicsVlanAndBondingForNetwork. Change-Id: I183e2f4cf780a0eb79a416cdc4afc387893ee106 Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=875487 Signed-off-by: Igor Lvovsky <ilvovsky(a)redhat.com> --- M vdsm/netinfo.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/84/9384/1 diff --git a/vdsm/netinfo.py b/vdsm/netinfo.py index b6d1a1f..80b0925 100644 --- a/vdsm/netinfo.py +++ b/vdsm/netinfo.py @@ -488,7 +488,7 @@ assert bonding is None bonding = port lnics += self.bondings[bonding]['slaves'] - else: + elif port in self.nics: lnics.append(port) return lnics, vlan, bonding -- To view, visit http://gerrit.ovirt.org/9384 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I183e2f4cf780a0eb79a416cdc4afc387893ee106 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Igor Lvovsky <ilvovsky(a)redhat.com>

6 12

Change in vdsm[master]: Bz#839919 remoteFileHandler.py: Add storage into python path
by wudxw＠linux.vnet.ibm.com 22 Nov '12

22 Nov '12

Mark Wu has uploaded a new change for review. Change subject: Bz#839919 remoteFileHandler.py: Add storage into python path ...................................................................... Bz#839919 remoteFileHandler.py: Add storage into python path pickle need import modules in vdsm/storage, so add it to python path. Actually it's already done for RPC server side. This patch adds it for client side, and merge the code. Change-Id: Id23c3d0be6eb2e2212172613ea8f3b0c58edce06 Signed-off-by: Mark Wu <wudxw(a)linux.vnet.ibm.com> --- M vdsm/storage/remoteFileHandler.py 1 file changed, 3 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/54/6354/1 -- To view, visit http://gerrit.ovirt.org/6354 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id23c3d0be6eb2e2212172613ea8f3b0c58edce06 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Mark Wu <wudxw(a)linux.vnet.ibm.com>

3 5

Change in vdsm[master]: change misc.retry to make it more general
by lvroyce＠linux.vnet.ibm.com 22 Nov '12

22 Nov '12

Royce Lv has uploaded a new change for review. Change subject: change misc.retry to make it more general ...................................................................... change misc.retry to make it more general add argument list to misc.retry to make it usable for function with arguments. Change-Id: I2b7d87629762d4d8f248d50fa4da71a0af3645d8 Signed-off-by: Royce Lv <lvroyce(a)linux.vnet.ibm.com> --- M vdsm/storage/misc.py M vdsm/supervdsm.py 2 files changed, 4 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/32/4032/1 -- To view, visit http://gerrit.ovirt.org/4032 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2b7d87629762d4d8f248d50fa4da71a0af3645d8 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Royce Lv <lvroyce(a)linux.vnet.ibm.com>

3 4

Change in vdsm[master]: Add local storage xmlrpc functional test
by lvroyce＠linux.vnet.ibm.com 22 Nov '12

22 Nov '12

Hello Adam Litke, I'd like you to do a code review. Please visit http://gerrit.ovirt.org/5860 to review the following change. Change subject: Add local storage xmlrpc functional test ...................................................................... Add local storage xmlrpc functional test Add defintion parser for functional test case reusable, add local storage xmlrpc tests it can be extend to nfs and iscsi type Change-Id: I4e2e80fa88d66d9bc1a4ba855e2df51ba8165e0a Signed-off-by: Royce Lv <lvroyce(a)linux.vnet.ibm.com> Signed-off-by: Adam Litke <agl(a)us.ibm.com> --- M configure.ac M tests/functional/Makefile.am A tests/functional/xmlrpcTests.py M vdsm.spec.in 4 files changed, 198 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/60/5860/1 -- To view, visit http://gerrit.ovirt.org/5860 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I4e2e80fa88d66d9bc1a4ba855e2df51ba8165e0a Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Royce Lv <lvroyce(a)linux.vnet.ibm.com> Gerrit-Reviewer: Adam Litke <agl(a)us.ibm.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com>

3 6

Change in vdsm[master]: misc: use approriate raise form in RollbackContext
by Federico Simoncelli 21 Nov '12

21 Nov '12

Federico Simoncelli has uploaded a new change for review. Change subject: misc: use approriate raise form in RollbackContext ...................................................................... misc: use approriate raise form in RollbackContext To avoid a pep8 error (W602 deprecated form of raising exception) the RollbackContext.__exit__ method should use the appropriate form to raise an exception. Change-Id: I390771ad2465069400bd777c4463019609ae17d8 Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M vdsm/storage/misc.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/60/9360/1 diff --git a/vdsm/storage/misc.py b/vdsm/storage/misc.py index 6360dcf..9126080 100644 --- a/vdsm/storage/misc.py +++ b/vdsm/storage/misc.py @@ -738,7 +738,7 @@ # re-raise the earliest exception if firstException is not None: - raise firstException, None, traceback + raise firstException(None, traceback) def defer(self, func, *args, **kwargs): self._finally.append((func, args, kwargs)) -- To view, visit http://gerrit.ovirt.org/9360 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I390771ad2465069400bd777c4463019609ae17d8 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

4 11

Change in vdsm[master]: remoteFileHandler pep8 fixes
by smizrahi＠redhat.com 21 Nov '12

21 Nov '12

Saggi Mizrahi has uploaded a new change for review. Change subject: remoteFileHandler pep8 fixes ...................................................................... remoteFileHandler pep8 fixes Change-Id: I849c858597e55a8366e5e1a3f964048da43e80dd Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M vdsm/storage/remoteFileHandler.py 1 file changed, 8 insertions(+), 7 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/98/9398/1 diff --git a/vdsm/storage/remoteFileHandler.py b/vdsm/storage/remoteFileHandler.py index d54bdcc..a47764b 100644 --- a/vdsm/storage/remoteFileHandler.py +++ b/vdsm/storage/remoteFileHandler.py @@ -146,7 +146,7 @@ raise Timeout() if not self._poll(self._myRead, select.POLLIN | select.POLLPRI, - timeLeft): + timeLeft): raise Timeout() try: @@ -166,7 +166,7 @@ raise Timeout() if not self._poll(self._myWrite, select.POLLOUT, - timeLeft): + timeLeft): raise Timeout() l += os.write(self._myWrite, data[l:]) @@ -187,7 +187,7 @@ # If for some reason the connection drops\gets out of sync we treat # it as a timeout so we only have one error path self.log.error("Problem with handler, treating as timeout", - exc_info=True) + exc_info=True) raise Timeout() res, err = pickle.loads(rawResponse) @@ -223,7 +223,8 @@ env['PYTHONPATH'] = "%s:%s" % ( constants.P_VDSM, env.get("PYTHONPATH", "")) self.process = BetterPopen([constants.EXT_PYTHON, __file__, - str(hisRead), str(hisWrite)], close_fds=False, env=env) + str(hisRead), str(hisWrite)], + close_fds=False, env=env) self.proxy = CrabRPCProxy(myRead, myWrite) @@ -273,14 +274,14 @@ handler = self.handlers[i] = PoolHandler() return handler.proxy.callCrabRPCFunction(timeout, name, - *args, **kwargs) + *args, **kwargs) except Timeout: try: self.handlers[i] = None handler.stop() except: self.log.error("Could not signal stuck handler (PID:%d)", - handler.process.pid, exc_info=True) + handler.process.pid, exc_info=True) self.handlers[i] = None raise @@ -386,7 +387,7 @@ try: server = CrabRPCServer(myRead, myWrite) for func in (writeLines, readLines, createSparseFile, echo, sleep, - directWriteLines, directReadLines, simpleWalk): + directWriteLines, directReadLines, simpleWalk): server.registerFunction(func) -- To view, visit http://gerrit.ovirt.org/9398 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I849c858597e55a8366e5e1a3f964048da43e80dd Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

3 5

Change in vdsm[master]: Remove REST bindings
by Jenkins CI RO 21 Nov '12

21 Nov '12

oVirt Jenkins CI Server has posted comments on this change. Change subject: Remove REST bindings ...................................................................... Patch Set 2: Build Successful http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/97/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_unit_tests_manual_gerrit/131/ : SUCCESS -- To view, visit http://gerrit.ovirt.org/7754 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I6c2a2909323e943e076db8443ce84f839eab874c Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com> Gerrit-Reviewer: Adam Litke <agl(a)us.ibm.com> Gerrit-Reviewer: Barak Azulay <bazulay(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com> Gerrit-Reviewer: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> Gerrit-Reviewer: oVirt Jenkins CI Server

1 0

Change in vdsm[master]: Remove REST bindings
by Jenkins CI RO 21 Nov '12

21 Nov '12

oVirt Jenkins CI Server has posted comments on this change. Change subject: Remove REST bindings ...................................................................... Patch Set 2: Build Started http://jenkins.ovirt.org/job/vdsm_unit_tests_manual_gerrit/131/ (2/2) -- To view, visit http://gerrit.ovirt.org/7754 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I6c2a2909323e943e076db8443ce84f839eab874c Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com> Gerrit-Reviewer: Adam Litke <agl(a)us.ibm.com> Gerrit-Reviewer: Barak Azulay <bazulay(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com> Gerrit-Reviewer: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> Gerrit-Reviewer: oVirt Jenkins CI Server

1 0

Change in vdsm[master]: Remove REST bindings
by Jenkins CI RO 21 Nov '12

21 Nov '12

oVirt Jenkins CI Server has posted comments on this change. Change subject: Remove REST bindings ...................................................................... Patch Set 2: Build Started http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/97/ (1/2) -- To view, visit http://gerrit.ovirt.org/7754 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I6c2a2909323e943e076db8443ce84f839eab874c Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com> Gerrit-Reviewer: Adam Litke <agl(a)us.ibm.com> Gerrit-Reviewer: Barak Azulay <bazulay(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com> Gerrit-Reviewer: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> Gerrit-Reviewer: oVirt Jenkins CI Server

1 0

Change in vdsm[master]: configNet: support PREFIX option
by Dan Kenigsberg 21 Nov '12

21 Nov '12

Dan Kenigsberg has uploaded a new change for review. Change subject: configNet: support PREFIX option ...................................................................... configNet: support PREFIX option PREFIX=16 means NETMASK=255.255.0.0. Change-Id: I1d87818642aa2092533f4864728c5ecd5d64f740 Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M vdsm/configNetwork.py 1 file changed, 26 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/22/9322/1 diff --git a/vdsm/configNetwork.py b/vdsm/configNetwork.py index 3940446..4bdb160 100755 --- a/vdsm/configNetwork.py +++ b/vdsm/configNetwork.py @@ -29,6 +29,8 @@ from xml.sax.saxutils import escape import glob import shutil +import socket +import struct import libvirt import selinux @@ -865,6 +867,21 @@ _validateInterNetworkCompatibility(_netinfo, vlan, nic, bridged) +def _prefix2netmask(prefix): + return socket.inet_ntoa( + struct.pack( + "!I", + int( + ( + ''.ljust(prefix, '1') + + ''.ljust(32 - prefix, '0') + ), + 2 + ) + ) + ) + + def addNetwork(network, vlan=None, bonding=None, nics=None, ipaddr=None, netmask=None, mtu=None, gateway=None, force=False, configWriter=None, bondingOptions=None, bridged=True, @@ -876,6 +893,15 @@ if mtu: mtu = int(mtu) + prefix = int(options.get('PREFIX')) + if prefix: + if netmask is None: + netmask = _prefix2netmask(prefix) + del options['PREFIX'] + else: + raise ConfigNetworkError(ne.ERR_BAD_PARAMS, + 'Both PREFIX and NETMASK supplied') + # Validation if not utils.tobool(force): logging.debug('validating network...') -- To view, visit http://gerrit.ovirt.org/9322 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I1d87818642aa2092533f4864728c5ecd5d64f740 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

5 34

Change in vdsm[master]: confNet: report missing net params as ERR_BAD_PARAMS
by Dan Kenigsberg 21 Nov '12

21 Nov '12

Dan Kenigsberg has uploaded a new change for review. Change subject: confNet: report missing net params as ERR_BAD_PARAMS ...................................................................... confNet: report missing net params as ERR_BAD_PARAMS Change-Id: I6e6b48332f27d1b976a8cd8069d252133ff32378 Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M vdsm/configNetwork.py 1 file changed, 12 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/59/6959/1 -- To view, visit http://gerrit.ovirt.org/6959 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I6e6b48332f27d1b976a8cd8069d252133ff32378 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

3 4

Change in vdsm[master]: vdsmd.init: retire ifconfig
by Dan Kenigsberg 21 Nov '12

21 Nov '12

Dan Kenigsberg has uploaded a new change for review. Change subject: vdsmd.init: retire ifconfig ...................................................................... vdsmd.init: retire ifconfig Replace ifconfig with iproute2's /sbin/ip also in our init script. Drop an old irrelevant comment that mentions ifconfig. Change-Id: I27385f74653b209c18aa8594b89a409287b4f259 Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M vdsm/netinfo.py M vdsm/vdsmd.init.in 2 files changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/85/9385/1 diff --git a/vdsm/netinfo.py b/vdsm/netinfo.py index b6d1a1f..7b28fae 100644 --- a/vdsm/netinfo.py +++ b/vdsm/netinfo.py @@ -274,9 +274,9 @@ def get(): d = {} routes = getRoutes() - # FIXME handle bridge/nic missing from ifconfig d['networks'] = {} nets = networks() + for netname in nets.iterkeys(): if nets[netname]['bridged']: devname = netname diff --git a/vdsm/vdsmd.init.in b/vdsm/vdsmd.init.in index 53ad8af..e1df196 100755 --- a/vdsm/vdsmd.init.in +++ b/vdsm/vdsmd.init.in @@ -179,7 +179,7 @@ } test_lo() { - if ! LC_ALL=C /sbin/ifconfig lo | /bin/grep -q UP; + if ! LC_ALL=C /sbin/ip lo | /bin/grep -q UP; then log_failure_msg "VDSMD: lo interface is down, can't run !" echo "VDSMD: lo interface is down, can't run !" > /dev/kmsg -- To view, visit http://gerrit.ovirt.org/9385 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I27385f74653b209c18aa8594b89a409287b4f259 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

5 14

Change in vdsm[master]: pep8 fixes
by lhornyak＠redhat.com 21 Nov '12

21 Nov '12

Laszlo Hornyak has uploaded a new change for review. Change subject: pep8 fixes ...................................................................... pep8 fixes fixes in the code to make vdsm build on rhel 6.3 Change-Id: Ifbbba925fa0c9be78b0eb5eb3d07066dc3b3c5ab Signed-off-by: Laszlo Hornyak <lhornyak(a)redhat.com> --- M vdsm/storage/misc.py M vdsm/vm.py 2 files changed, 4 insertions(+), 5 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/66/9366/1 diff --git a/vdsm/storage/misc.py b/vdsm/storage/misc.py index 6360dcf..9f3d1bc 100644 --- a/vdsm/storage/misc.py +++ b/vdsm/storage/misc.py @@ -733,12 +733,10 @@ # keep the earliest exception info if not firstException: firstException = e - # keep the original traceback info - traceback = sys.exc_info()[2] # re-raise the earliest exception if firstException is not None: - raise firstException, None, traceback + raise firstException def defer(self, func, *args, **kwargs): self._finally.append((func, args, kwargs)) diff --git a/vdsm/vm.py b/vdsm/vm.py index e6d5947..73e3f3f 100644 --- a/vdsm/vm.py +++ b/vdsm/vm.py @@ -363,7 +363,7 @@ SOUND_DEVICES: [], VIDEO_DEVICES: [], CONTROLLER_DEVICES: [], GENERAL_DEVICES: [], BALLOON_DEVICES: [], REDIR_DEVICES: [], - WATCHDOG_DEVICES: [],} + WATCHDOG_DEVICES: []} def _get_lastStatus(self): PAUSED_STATES = ('Powering down', 'RebootInProgress', 'Up') @@ -495,7 +495,8 @@ if not 'specParams' in devices[WATCHDOG_DEVICES][0]: devices[WATCHDOG_DEVICES][0]['specParams'] = {} if not 'model' in devices[WATCHDOG_DEVICES][0]['specParams']: - devices[WATCHDOG_DEVICES][0]['specParams']['model'] = 'i6300esb' + devices[WATCHDOG_DEVICES][0]['specParams']['model'] = \ + 'i6300esb' if not 'action' in devices[WATCHDOG_DEVICES][0]['specParams']: devices[WATCHDOG_DEVICES][0]['specParams']['action'] = 'none' -- To view, visit http://gerrit.ovirt.org/9366 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ifbbba925fa0c9be78b0eb5eb3d07066dc3b3c5ab Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Laszlo Hornyak <lhornyak(a)redhat.com>

5 13

Change in vdsm[master]: tc: retire ifconfig
by Dan Kenigsberg 21 Nov '12

21 Nov '12

Dan Kenigsberg has uploaded a new change for review. Change subject: tc: retire ifconfig ...................................................................... tc: retire ifconfig ifconfig was deprecated quite a few years ago, and some modern distribution do not ship it by default. In this patch, we dump its only use, by setting promisc flag directly. We implement set_flags() in hope to see ethtool add it. Change-Id: Ifb4f289efb5754d7c49ac0a08e3a9adc09b12bb7 Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M configure.ac M vdsm/constants.py.in M vdsm/tc.py M vdsm_reg/deployUtil.py.in 4 files changed, 30 insertions(+), 9 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/70/9370/1 diff --git a/configure.ac b/configure.ac index 5d8b238..b8f1825 100644 --- a/configure.ac +++ b/configure.ac @@ -144,7 +144,6 @@ AC_PATH_PROG([FUSER_PATH], [fuser], [/sbin/fuser]) AC_PATH_PROG([GRUBBY_PATH], [grubby], [/sbin/grubby]) AC_PATH_PROG([HWCLOCK_PATH], [hwclock], [/usr/sbin/hwclock]) -AC_PATH_PROG([IFCONFIG_PATH], [ifconfig], [/sbin/ifconfig]) AC_PATH_PROG([IFDOWN_PATH], [ifdown], [/sbin/ifdown]) AC_PATH_PROG([IFUP_PATH], [ifup], [/sbin/ifup]) AC_PATH_PROG([IONICE_PATH], [ionice], [/usr/bin/ionice]) diff --git a/vdsm/constants.py.in b/vdsm/constants.py.in index 2e9eb99..07dd431 100644 --- a/vdsm/constants.py.in +++ b/vdsm/constants.py.in @@ -97,7 +97,6 @@ EXT_FSCK = '@FSCK_PATH@' EXT_FUSER = '@FUSER_PATH@' -EXT_IFCONFIG = '@IFCONFIG_PATH@' EXT_IFDOWN = '@IFDOWN_PATH@' EXT_IFUP = '@IFUP_PATH@' EXT_IONICE = '@IONICE_PATH@' diff --git a/vdsm/tc.py b/vdsm/tc.py index a0a612f..7630b9b 100644 --- a/vdsm/tc.py +++ b/vdsm/tc.py @@ -19,11 +19,15 @@ # from collections import namedtuple +from contextlib import closing +import ctypes +import fcntl +import socket import ethtool import storage.misc -from vdsm.constants import EXT_TC, EXT_IFCONFIG +from vdsm.constants import EXT_TC ERR_DEV_NOEXIST = 2 @@ -155,12 +159,32 @@ raise +def set_flags(dev, flags): + "Set device flags. We need this local definition until ethtool has it" + + SIOCSIFFLAGS = 0x8914 + + class ifreq(ctypes.Structure): + _fields_ = [("ifr_ifrn", ctypes.c_char * 16), + ("ifr_flags", ctypes.c_short)] + + with closing(socket.socket(socket.AF_INET, socket.SOCK_DGRAM)) as s: + ifr = ifreq() + ifr.ifr_ifrn = dev + ifr.ifr_flags = flags + + fcntl.ioctl(s.fileno(), SIOCSIFFLAGS, ifr) + + def set_promisc(dev, on=True): - promisc = 'promisc' - if not on: - promisc = '-promisc' - command = [EXT_IFCONFIG, dev, promisc] - _process_request(command) + flags = ethtool.get_flags(dev) + + if on: + flags |= ethtool.IFF_PROMISC + else: + flags &= ~ethtool.IFF_PROMISC + + set_flags(dev, flags) Filter = namedtuple('Filter', 'prio handle actions') diff --git a/vdsm_reg/deployUtil.py.in b/vdsm_reg/deployUtil.py.in index dec03c2..0fe1275 100644 --- a/vdsm_reg/deployUtil.py.in +++ b/vdsm_reg/deployUtil.py.in @@ -65,7 +65,6 @@ EX_ECHO = '@ECHO_PATH@' EX_GRUBBY = '@GRUBBY_PATH@' EX_HWCLOCK = '@HWCLOCK_PATH@' -EX_IFCONFIG = '@IFCONFIG_PATH@' EX_NTPQ = '@NTPQ_PATH@' EX_OPENSSL = '@OPENSSL_PATH@' EX_REBOOT = '@REBOOT_PATH@' -- To view, visit http://gerrit.ovirt.org/9370 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ifb4f289efb5754d7c49ac0a08e3a9adc09b12bb7 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

6 9

Change in vdsm[master]: add a constants.py file for HSM service standalone
by shaohef＠linux.vnet.ibm.com 21 Nov '12

21 Nov '12

Hello Shu Ming, I'd like you to do a code review. Please visit http://gerrit.ovirt.org/9348 to review the following change. Change subject: add a constants.py file for HSM service standalone ...................................................................... add a constants.py file for HSM service standalone constants.py is installed in a separate python site-packages for hsm Remame P_VDSM_RUN as P_HSM_RUN This is one of step to make HSM service standalone wiki-page: http://wiki.ovirt.org/wiki/HSM_service_stand_alone Change-Id: I092c11d8ee022c7d2954d52352ff44ea37233f37 Signed-off-by: Shu Ming <shuming(a)linux.vnet.ibm.com> Signed-off-by: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> --- M .gitignore M build-aux/Makefile.subs M configure.ac M vdsm.spec.in M vdsm/storage/Makefile.am M vdsm/storage/blockSD.py A vdsm/storage/constants.py.in M vdsm/storage/devicemapper.py M vdsm/storage/fileSD.py M vdsm/storage/fileUtils.py M vdsm/storage/fuser.py M vdsm/storage/hsm.py M vdsm/storage/iscsi.py M vdsm/storage/iscsiadm.py M vdsm/storage/lvm.py M vdsm/storage/misc.py M vdsm/storage/mount.py M vdsm/storage/multipath.py M vdsm/storage/remoteFileHandler.py M vdsm/storage/safelease.py M vdsm/storage/sd.py M vdsm/storage/sp.py M vdsm/storage/storage_mailbox.py M vdsm/storage/volume.py 24 files changed, 172 insertions(+), 23 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/48/9348/1 diff --git a/.gitignore b/.gitignore index b5b17e4..577453f 100644 --- a/.gitignore +++ b/.gitignore @@ -31,6 +31,7 @@ vdsm/mom.conf vdsm/sos/vdsm.py vdsm/storage/config.py +vdsm/storage/constants.py vdsm/storage/protect/safelease vdsm/sudoers.vdsm vdsm/vdsm-gencerts.sh diff --git a/build-aux/Makefile.subs b/build-aux/Makefile.subs index 74b7153..5f88984 100644 --- a/build-aux/Makefile.subs +++ b/build-aux/Makefile.subs @@ -34,7 +34,8 @@ -e "s,[@]VDSMLOGDIR[@],$(vdsmlogdir),g" \ -e "s,[@]VDSMREPO[@],$(vdsmrepo),g" \ -e "s,[@]VDSMRUNDIR[@],$(vdsmrundir),g" \ - -e "s,[@]VDSMUPGRADEDIR[@],$(vdsmupgradedir),g" + -e "s,[@]VDSMUPGRADEDIR[@],$(vdsmupgradedir),g" \ + -e "s,[@]HSMRUNDIR[@],$(hsmrundir),g" CONFIGSUBST = $(top_builddir)/config.status --file=- diff --git a/configure.ac b/configure.ac index a384823..2412a01 100644 --- a/configure.ac +++ b/configure.ac @@ -82,6 +82,8 @@ # HSM default paths AC_SUBST([hsmpylibdir], ['${pyexecdir}/hsmd']) +# FIXME: this path should be modified when HSM service standalone was done +AC_SUBST([hsmrundir], ['${localstatedir}/run/vdsm']) # VDSM registration default paths AC_SUBST([vdsmregdir], ['${datarootdir}/vdsm-reg']) diff --git a/vdsm.spec.in b/vdsm.spec.in index a9f1379..c5997e8 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -779,6 +779,7 @@ %{python_sitearch}/%{vdsm_name}/tool/load_needed_modules.py* %{python_sitearch}/%{hsm_name}/__init__.py* %{python_sitearch}/%{hsm_name}/config.py* +%{python_sitearch}/%{hsm_name}/constants.py* %files tests %doc %{_datadir}/%{vdsm_name}/tests/README diff --git a/vdsm/storage/Makefile.am b/vdsm/storage/Makefile.am index 9af470e..09028a5 100644 --- a/vdsm/storage/Makefile.am +++ b/vdsm/storage/Makefile.am @@ -70,7 +70,8 @@ __init__.py nodist_hsmpylib_PYTHON = \ - config.py + config.py \ + constants.py nodist_noinst_DATA = \ hsm.conf.sample @@ -81,7 +82,8 @@ EXTRA_DIST = \ 12-vdsm-lvm.rules \ - config.py.in + config.py.in \ + constants.py.in hsm.conf.sample: Makefile config.py @echo " CONFIG $@"; $(PYTHON) $(builddir)/config.py > $@ diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index d59aa05..e0e2e95 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -32,7 +32,7 @@ from operator import itemgetter from hsmd.config import config -from vdsm import constants +from hsmd import constants import misc import fileUtils import sd diff --git a/vdsm/storage/constants.py.in b/vdsm/storage/constants.py.in new file mode 100644 index 0000000..292d8a5 --- /dev/null +++ b/vdsm/storage/constants.py.in @@ -0,0 +1,142 @@ +# +# Copyright 2009-2012 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# +# +# Description: Constants definitions for hsm and utilities. + +# +# file ownership definitions +# +DISKIMAGE_USER = '@VDSMUSER@' +DISKIMAGE_GROUP = '@QEMUGROUP@' +METADATA_USER = '@METADATAUSER@' +METADATA_GROUP = '@METADATAGROUP@' +QEMU_PROCESS_USER = '@QEMUUSER@' + +# This is the domain version translation list +# DO NOT CHANGE OLD VALUES ONLY APPEND +DOMAIN_VERSIONS = (0, 2, 3) +SUPPORTED_BLOCKSIZE = (512,) + +# This contains the domains versions that this HSM +# accepts currently its all of the version but in the +# future we might slice it (eg. tuple(DOMAIN_VERSION[1:])) +SUPPORTED_DOMAIN_VERSIONS = DOMAIN_VERSIONS + +UUID_GLOB_PATTERN = '*-*-*-*-*' + +MEGAB = 2 ** 20 # = 1024 ** 2 = 1 MiB + +# +# Path definitions +# +P_HSM_RUN = '@HSMRUNDIR@/' + + +# +# External programs (sorted, please keep in order). +# + +EXT_CHOWN = '@CHOWN_PATH@' +EXT_CP = '@CP_PATH@' + +EXT_DD = '@DD_PATH@' +EXT_DMSETUP = '@DMSETUP_PATH@' + +EXT_FSCK = '@FSCK_PATH@' +EXT_FUSER = '@FUSER_PATH@' + +EXT_IONICE = '@IONICE_PATH@' +EXT_ISCSIADM = '@ISCSIADM_PATH@' +EXT_TC = '@TC_PATH@' + +EXT_KILL = '@KILL_PATH@' + +EXT_LSBLK = '@LSBLK_PATH@' +EXT_LVM = '@LVM_PATH@' + +EXT_MKFS = '@MKFS_PATH@' +EXT_MKFS_MSDOS = '@MKFS_MSDOS_PATH@' +EXT_MKISOFS = '@MKISOFS_PATH@' +EXT_MOUNT = '@MOUNT_PATH@' +EXT_MULTIPATH = '@MULTIPATH_PATH@' +EXT_MV = '@MV_PATH@' + +EXT_NICE = '@NICE_PATH@' + +EXT_PERSIST = '@PERSIST_PATH@' +EXT_PGREP = '@PGREP_PATH@' +EXT_PYTHON = '@PYTHON@' + +EXT_QEMUIMG = '@QEMUIMG_PATH@' + +EXT_REBOOT = '@REBOOT_PATH@' +EXT_RSYNC = '@RSYNC_PATH@' + +EXT_SCSI_ID = '@SCSI_ID_PATH@' # TBD ! +EXT_SERVICE = '@SERVICE_PATH@' +EXT_SETSID = '@SETSID_PATH@' +EXT_SH = '/bin/sh' # The shell path is invariable +EXT_SHOWMOUNT = '@SHOWMOUNT_PATH@' +EXT_SU = '@SU_PATH@' +EXT_SUDO = '@SUDO_PATH@' + +EXT_TAR = '@TAR_PATH@' +EXT_TUNE2FS = '@TUNE2FS_PATH@' + +EXT_UDEVADM = '@UDEVADM_PATH@' + +EXT_UMOUNT = '@UMOUNT_PATH@' +EXT_UNPERSIST = '@UNPERSIST_PATH@' + +EXT_IPROUTE = '@IPROUTE_PATH@' + +EXT_WGET = '@WGET_PATH@' + +CMD_LOWPRIO = [EXT_NICE, '-n', '19', EXT_IONICE, '-c', '3'] + +# +# Storage constants +# +STRG_ISCSI_HOST = "iscsi_host/" +STRG_SCSI_HOST = "scsi_host/" +STRG_ISCSI_SESSION = "iscsi_session/" +STRG_MPATH_CONF = ( +"\n\n" +"defaults {\n" +" polling_interval 5\n" +" getuid_callout \"@SCSI_ID_PATH@ --whitelisted " + "--replace-whitespace --device=/dev/%n\"\n" +" no_path_retry fail\n" +" user_friendly_names no\n" +" flush_on_last_del yes\n" +" fast_io_fail_tmo 5\n" +" dev_loss_tmo 30\n" +" max_fds 4096\n" +"}\n" +"\n" +"devices {\n" +"device {\n" +" vendor \"HITACHI\"\n" +" product \"DF.*\"\n" +" getuid_callout \"@SCSI_ID_PATH@ --whitelisted " + "--replace-whitespace --device=/dev/%n\"\n" +"}\n" +"}" +) diff --git a/vdsm/storage/devicemapper.py b/vdsm/storage/devicemapper.py index a1651e0..c931085 100644 --- a/vdsm/storage/devicemapper.py +++ b/vdsm/storage/devicemapper.py @@ -25,7 +25,7 @@ import re from supervdsm import getProxy -from vdsm.constants import EXT_DMSETUP +from hsmd.constants import EXT_DMSETUP DMPATH_FORMAT = "/dev/mapper/%s" diff --git a/vdsm/storage/fileSD.py b/vdsm/storage/fileSD.py index 90ffecd..7ec3718 100644 --- a/vdsm/storage/fileSD.py +++ b/vdsm/storage/fileSD.py @@ -32,7 +32,7 @@ import outOfProcess as oop from remoteFileHandler import Timeout from persistentDict import PersistentDict, DictValidator -from vdsm import constants +from hsmd import constants import time import supervdsm import mount diff --git a/vdsm/storage/fileUtils.py b/vdsm/storage/fileUtils.py index 13a8cfa..834fd3b 100644 --- a/vdsm/storage/fileUtils.py +++ b/vdsm/storage/fileUtils.py @@ -33,7 +33,7 @@ import subprocess import shutil -from vdsm import constants +from hsmd import constants import logging import errno libc = ctypes.CDLL("libc.so.6", use_errno=True) diff --git a/vdsm/storage/fuser.py b/vdsm/storage/fuser.py index d9becbc..6de3032 100644 --- a/vdsm/storage/fuser.py +++ b/vdsm/storage/fuser.py @@ -18,7 +18,7 @@ # Refer to the README and COPYING files for full details of the license # import misc -from vdsm import constants +from hsmd import constants def fuser(path, mountPoint=False): diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 476eed8..8b29707 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -56,7 +56,7 @@ import safelease import storage_exception as se from threadLocal import vars -from vdsm import constants +from hsmd import constants from storageConstants import STORAGE from task import Job from resourceFactories import IMAGE_NAMESPACE @@ -84,7 +84,7 @@ # FIXME: moved from spm.py but this should be somewhere else SECTOR_SIZE = 512 -STORAGE_CONNECTION_DIR = os.path.join(constants.P_VDSM_RUN, "connections/") +STORAGE_CONNECTION_DIR = os.path.join(constants.P_HSM_RUN, "connections/") def public(f=None, **kwargs): if f is None: diff --git a/vdsm/storage/iscsi.py b/vdsm/storage/iscsi.py index 4723746..57dfe4a 100644 --- a/vdsm/storage/iscsi.py +++ b/vdsm/storage/iscsi.py @@ -30,7 +30,7 @@ import time from collections import namedtuple -from vdsm import constants +from hsmd import constants import misc from hsmd.config import config import devicemapper diff --git a/vdsm/storage/iscsiadm.py b/vdsm/storage/iscsiadm.py index 6b58d15..6182d15 100644 --- a/vdsm/storage/iscsiadm.py +++ b/vdsm/storage/iscsiadm.py @@ -1,6 +1,6 @@ from threading import Lock import misc -from vdsm import constants +from hsmd import constants # iscsiadm exit statuses ISCSI_ERR_SESS_EXISTS = 15 diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py index 8c77a6a..8543a26 100644 --- a/vdsm/storage/lvm.py +++ b/vdsm/storage/lvm.py @@ -37,7 +37,7 @@ from itertools import chain from subprocess import list2cmdline -from vdsm import constants +from hsmd import constants import misc import multipath import storage_exception as se @@ -120,7 +120,7 @@ } """ -VAR_RUN_VDSM = constants.P_VDSM_RUN +VAR_RUN_VDSM = constants.P_HSM_RUN VDSM_LVM_SYSTEM_DIR = os.path.join(VAR_RUN_VDSM, "lvm") VDSM_LVM_CONF = os.path.join(VDSM_LVM_SYSTEM_DIR, "lvm.conf") diff --git a/vdsm/storage/misc.py b/vdsm/storage/misc.py index bf3ae79..9de1e93 100644 --- a/vdsm/storage/misc.py +++ b/vdsm/storage/misc.py @@ -56,7 +56,7 @@ import inspect sys.path.append("../") -from vdsm import constants +from hsmd import constants import storage_exception as se from vdsm.betterPopen import BetterPopen import logUtils diff --git a/vdsm/storage/mount.py b/vdsm/storage/mount.py index 94cfae0..1659a89 100644 --- a/vdsm/storage/mount.py +++ b/vdsm/storage/mount.py @@ -24,7 +24,7 @@ import os import stat -from vdsm import constants +from hsmd import constants import misc # Common vfs types diff --git a/vdsm/storage/multipath.py b/vdsm/storage/multipath.py index c43cb09..627f2ce 100644 --- a/vdsm/storage/multipath.py +++ b/vdsm/storage/multipath.py @@ -29,7 +29,7 @@ import re from collections import namedtuple -from vdsm import constants +from hsmd import constants import misc import iscsi import supervdsm diff --git a/vdsm/storage/remoteFileHandler.py b/vdsm/storage/remoteFileHandler.py index edf8e68..47f4f1c 100644 --- a/vdsm/storage/remoteFileHandler.py +++ b/vdsm/storage/remoteFileHandler.py @@ -44,7 +44,7 @@ # don't need anything when the child spawns. Plus anything you don't have # to import is less memory taken by each helper. from vdsm.betterPopen import BetterPopen - from vdsm import constants + from hsmd import constants else: # We add the parent directory so that imports that import the storage # package would work even though CWD is inside the storage package. diff --git a/vdsm/storage/safelease.py b/vdsm/storage/safelease.py index 5184224..9ce880c 100644 --- a/vdsm/storage/safelease.py +++ b/vdsm/storage/safelease.py @@ -24,7 +24,7 @@ import subprocess import sanlock from contextlib import nested -from vdsm import constants +from hsmd import constants import storage_exception as se import threading import logging diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index 52f80aa..7bfbcd3 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -30,7 +30,7 @@ import resourceFactories from resourceFactories import IMAGE_NAMESPACE, VOLUME_NAMESPACE import resourceManager as rm -from vdsm import constants +from hsmd import constants import safelease import outOfProcess as oop from persistentDict import unicodeEncoder, unicodeDecoder diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index 77452e9..d3f5c82 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -32,7 +32,7 @@ from imageRepository.formatConverter import DefaultFormatConverter -from vdsm import constants +from hsmd import constants import storage_mailbox import blockSD import fileSD diff --git a/vdsm/storage/storage_mailbox.py b/vdsm/storage/storage_mailbox.py index 4743940..a14f603 100644 --- a/vdsm/storage/storage_mailbox.py +++ b/vdsm/storage/storage_mailbox.py @@ -34,7 +34,7 @@ from threadLocal import vars from threadPool import ThreadPool from storage_exception import InvalidParameterException -from vdsm import constants +from hsmd import constants __author__="ayalb" __date__ ="$Mar 9, 2009 5:25:07 PM$" diff --git a/vdsm/storage/volume.py b/vdsm/storage/volume.py index b97e6f3..0c854e4 100644 --- a/vdsm/storage/volume.py +++ b/vdsm/storage/volume.py @@ -25,7 +25,7 @@ from contextlib import contextmanager import image -from vdsm import constants +from hsmd import constants import storage_exception as se import sd from sdc import sdCache -- To view, visit http://gerrit.ovirt.org/9348 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I092c11d8ee022c7d2954d52352ff44ea37233f37 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> Gerrit-Reviewer: Shu Ming <shuming(a)linux.vnet.ibm.com>

2 6

Change in vdsm[master]: move betterPopen to storage for HSM service standalone
by shaohef＠linux.vnet.ibm.com 21 Nov '12

21 Nov '12

ShaoHe Feng has uploaded a new change for review. Change subject: move betterPopen to storage for HSM service standalone ...................................................................... move betterPopen to storage for HSM service standalone move betterPopen to the hsm python lib This is one of step to make HSM service standalone wiki-page: http://wiki.ovirt.org/wiki/HSM_service_stand_alone Change-Id: Ide9c63ec0072d44c717dd78e8613352b5e3e5e12 Signed-off-by: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> --- M configure.ac M tests/betterPopenTests.py M tests/testrunner.py M vdsm.spec.in M vdsm/Makefile.am M vdsm/storage/Makefile.am R vdsm/storage/betterPopen/.gitignore R vdsm/storage/betterPopen/Makefile.am R vdsm/storage/betterPopen/__init__.py R vdsm/storage/betterPopen/createprocess.c R vdsm/storage/betterPopen/setup.py M vdsm/storage/misc.py M vdsm/storage/remoteFileHandler.py 13 files changed, 10 insertions(+), 10 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/56/9356/1 diff --git a/configure.ac b/configure.ac index 2412a01..b6de932 100644 --- a/configure.ac +++ b/configure.ac @@ -218,9 +218,9 @@ vdsm_hooks/vmdisk/Makefile vdsm/Makefile vdsm_reg/Makefile - vdsm/betterPopen/Makefile vdsm/gluster/Makefile vdsm/sos/Makefile + vdsm/storage/betterPopen/Makefile vdsm/storage/Makefile vdsm/storage/imageRepository/Makefile vdsm/storage/protect/Makefile diff --git a/tests/betterPopenTests.py b/tests/betterPopenTests.py index 6347460..b48288a 100644 --- a/tests/betterPopenTests.py +++ b/tests/betterPopenTests.py @@ -28,7 +28,7 @@ if __name__ != "__main__": # This will not be available when we use this module as a subprocess - from vdsm.betterPopen import BetterPopen + from hsmd.betterPopen import BetterPopen class TestBetterPopen(TestCaseBase): diff --git a/tests/testrunner.py b/tests/testrunner.py index 596ced1..73cd629 100644 --- a/tests/testrunner.py +++ b/tests/testrunner.py @@ -235,7 +235,7 @@ sys.modules['vdsm'] = mod = vdsm() for name in ('config', 'constants', 'utils', 'define', 'netinfo', - 'SecureXMLRPCServer', 'libvirtconnection', 'betterPopen', + 'SecureXMLRPCServer', 'libvirtconnection', 'exception', 'vdscli', 'qemuImg'): sub = __import__(name, globals(), locals(), [], -1) setattr(mod, name, sub) diff --git a/vdsm.spec.in b/vdsm.spec.in index c5997e8..c1140ce 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -771,13 +771,13 @@ %{python_sitearch}/%{vdsm_name}/SecureXMLRPCServer.py* %{python_sitearch}/%{vdsm_name}/utils.py* %{python_sitearch}/%{vdsm_name}/vdscli.py* -%{python_sitearch}/%{vdsm_name}/betterPopen/__init__.py* -%{python_sitearch}/%{vdsm_name}/betterPopen/createprocess.so %{python_sitearch}/%{vdsm_name}/tool/__init__.py* %{python_sitearch}/%{vdsm_name}/tool/passwd.py* %{python_sitearch}/%{vdsm_name}/tool/validate_ovirt_certs.py* %{python_sitearch}/%{vdsm_name}/tool/load_needed_modules.py* %{python_sitearch}/%{hsm_name}/__init__.py* +%{python_sitearch}/%{hsm_name}/betterPopen/__init__.py* +%{python_sitearch}/%{hsm_name}/betterPopen/createprocess.so %{python_sitearch}/%{hsm_name}/config.py* %{python_sitearch}/%{hsm_name}/constants.py* diff --git a/vdsm/Makefile.am b/vdsm/Makefile.am index 7c79036..b127168 100644 --- a/vdsm/Makefile.am +++ b/vdsm/Makefile.am @@ -18,7 +18,7 @@ # Refer to the README and COPYING files for full details of the license # -SUBDIRS = sos storage betterPopen rest gluster +SUBDIRS = sos storage rest gluster include $(top_srcdir)/build-aux/Makefile.subs diff --git a/vdsm/storage/Makefile.am b/vdsm/storage/Makefile.am index 09028a5..188943f 100644 --- a/vdsm/storage/Makefile.am +++ b/vdsm/storage/Makefile.am @@ -18,7 +18,7 @@ # Refer to the README and COPYING files for full details of the license # -SUBDIRS = protect imageRepository +SUBDIRS = betterPopen protect imageRepository include $(top_srcdir)/build-aux/Makefile.subs diff --git a/vdsm/betterPopen/.gitignore b/vdsm/storage/betterPopen/.gitignore similarity index 100% rename from vdsm/betterPopen/.gitignore rename to vdsm/storage/betterPopen/.gitignore diff --git a/vdsm/betterPopen/Makefile.am b/vdsm/storage/betterPopen/Makefile.am similarity index 96% rename from vdsm/betterPopen/Makefile.am rename to vdsm/storage/betterPopen/Makefile.am index 9a14e86..b76c3db 100644 --- a/vdsm/betterPopen/Makefile.am +++ b/vdsm/storage/betterPopen/Makefile.am @@ -18,7 +18,7 @@ # Refer to the README and COPYING files for full details of the license # -betterpopendir = $(vdsmpylibdir)/betterPopen +betterpopendir = $(hsmpylibdir)/betterPopen dist_betterpopen_PYTHON = \ __init__.py diff --git a/vdsm/betterPopen/__init__.py b/vdsm/storage/betterPopen/__init__.py similarity index 100% rename from vdsm/betterPopen/__init__.py rename to vdsm/storage/betterPopen/__init__.py diff --git a/vdsm/betterPopen/createprocess.c b/vdsm/storage/betterPopen/createprocess.c similarity index 100% rename from vdsm/betterPopen/createprocess.c rename to vdsm/storage/betterPopen/createprocess.c diff --git a/vdsm/betterPopen/setup.py b/vdsm/storage/betterPopen/setup.py similarity index 100% rename from vdsm/betterPopen/setup.py rename to vdsm/storage/betterPopen/setup.py diff --git a/vdsm/storage/misc.py b/vdsm/storage/misc.py index 9de1e93..ea874df 100644 --- a/vdsm/storage/misc.py +++ b/vdsm/storage/misc.py @@ -58,7 +58,7 @@ sys.path.append("../") from hsmd import constants import storage_exception as se -from vdsm.betterPopen import BetterPopen +from hsmd.betterPopen import BetterPopen import logUtils diff --git a/vdsm/storage/remoteFileHandler.py b/vdsm/storage/remoteFileHandler.py index 47f4f1c..d28ef23 100644 --- a/vdsm/storage/remoteFileHandler.py +++ b/vdsm/storage/remoteFileHandler.py @@ -43,7 +43,7 @@ # If you don't have the vdsm package installed this will fail. Luckily we # don't need anything when the child spawns. Plus anything you don't have # to import is less memory taken by each helper. - from vdsm.betterPopen import BetterPopen + from hsmd.betterPopen import BetterPopen from hsmd import constants else: # We add the parent directory so that imports that import the storage -- To view, visit http://gerrit.ovirt.org/9356 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ide9c63ec0072d44c717dd78e8613352b5e3e5e12 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com>

2 6

Change in vdsm[master]: split config file for HSM service standalone
by shaohef＠linux.vnet.ibm.com 21 Nov '12

21 Nov '12

Hello Shu Ming, I'd like you to do a code review. Please visit http://gerrit.ovirt.org/9345 to review the following change. Change subject: split config file for HSM service standalone ...................................................................... split config file for HSM service standalone Split config.py of vdsm, irs section was moved to hsm config.py Create a separate python site-packages for hsm config.py Split vdsm.conf, irs section was moved to hsm.conf The hsm.conf was still install in the vdsm config path The hsm.conf.sample was still install in the vdsm doc patch In order to run the locole test, hackVdsmModule of testrunner This is one of step to make HSM service standalone wiki-page: http://wiki.ovirt.org/wiki/HSM_service_stand_alone Change-Id: I8f62a4108275a1295f122025973a767f79b75780 Signed-off-by: Shu Ming <shuming(a)linux.vnet.ibm.com> Signed-off-by: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> --- M .gitignore M configure.ac M tests/testrunner.py M vdsm.spec.in M vdsm/clientIF.py M vdsm/config.py.in M vdsm/libvirtvm.py M vdsm/storage/Makefile.am M vdsm/storage/blockSD.py M vdsm/storage/blockVolume.py A vdsm/storage/config.py.in M vdsm/storage/dispatcher.py M vdsm/storage/domainMonitor.py M vdsm/storage/fileVolume.py M vdsm/storage/hsm.py M vdsm/storage/image.py M vdsm/storage/iscsi.py M vdsm/storage/lvm.py M vdsm/storage/outOfProcess.py M vdsm/storage/resourceFactories.py M vdsm/storage/safelease.py M vdsm/storage/sd.py M vdsm/storage/sdc.py M vdsm/storage/sp.py M vdsm/storage/storage_mailbox.py M vdsm/storage/task.py M vdsm/storage/taskManager.py M vdsm/supervdsmServer.py M vdsm/utils.py M vdsm/vm.py 30 files changed, 217 insertions(+), 120 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/45/9345/1 diff --git a/.gitignore b/.gitignore index 49a78ef..b5b17e4 100644 --- a/.gitignore +++ b/.gitignore @@ -30,6 +30,7 @@ vdsm/mk_sysprep_floppy vdsm/mom.conf vdsm/sos/vdsm.py +vdsm/storage/config.py vdsm/storage/protect/safelease vdsm/sudoers.vdsm vdsm/vdsm-gencerts.sh diff --git a/configure.ac b/configure.ac index 7cb12ef..a384823 100644 --- a/configure.ac +++ b/configure.ac @@ -80,6 +80,9 @@ AC_SUBST([vdsmtooldir], ['${vdsmpylibdir}/tool']) AC_SUBST([vdsmtestsdir], ['${datarootdir}/vdsm/tests']) +# HSM default paths +AC_SUBST([hsmpylibdir], ['${pyexecdir}/hsmd']) + # VDSM registration default paths AC_SUBST([vdsmregdir], ['${datarootdir}/vdsm-reg']) AC_SUBST([vdsmbootstrapdir], ['${datarootdir}/vdsm-bootstrap']) diff --git a/tests/testrunner.py b/tests/testrunner.py index 9851f3e..596ced1 100644 --- a/tests/testrunner.py +++ b/tests/testrunner.py @@ -228,6 +228,10 @@ def hackVdsmModule(): + # this hack should be removed after the hsm service standalone was done + sub = __import__('storage', globals(), locals(), [], -1) + sys.modules['hsmd'] = sub + sys.modules['vdsm'] = mod = vdsm() for name in ('config', 'constants', 'utils', 'define', 'netinfo', diff --git a/vdsm.spec.in b/vdsm.spec.in index c117ee5..a9f1379 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -1,5 +1,6 @@ # Packages names %global vdsm_name vdsm +%global hsm_name hsmd %global vdsm_bootstrap vdsm-bootstrap %global vdsm_reg vdsm-reg @@ -403,6 +404,8 @@ # Install the configuration sample install -Dm 0644 vdsm/vdsm.conf.sample \ %{buildroot}%{_sysconfdir}/vdsm/vdsm.conf +install -Dm 0644 vdsm/storage/hsm.conf.sample \ + %{buildroot}%{_sysconfdir}/vdsm/hsm.conf # Install the systemd scripts install -Dm 0755 vdsm/vdsmd.init %{buildroot}/lib/systemd/systemd-vdsmd @@ -589,6 +592,7 @@ %files %defattr(-, root, root, -) %doc COPYING README vdsm/vdsm.conf.sample +%doc COPYING README vdsm/storage/hsm.conf.sample %if 0%{?rhel} %{_initrddir}/vdsmd %else @@ -754,6 +758,7 @@ %{_bindir}/vdsm-tool %if !0%{?rhel} %config(noreplace) %{_sysconfdir}/%{vdsm_name}/vdsm.conf +%config(noreplace) %{_sysconfdir}/%{vdsm_name}/hsm.conf %endif %{python_sitearch}/%{vdsm_name}/__init__.py* %{python_sitearch}/%{vdsm_name}/config.py* @@ -772,6 +777,8 @@ %{python_sitearch}/%{vdsm_name}/tool/passwd.py* %{python_sitearch}/%{vdsm_name}/tool/validate_ovirt_certs.py* %{python_sitearch}/%{vdsm_name}/tool/load_needed_modules.py* +%{python_sitearch}/%{hsm_name}/__init__.py* +%{python_sitearch}/%{hsm_name}/config.py* %files tests %doc %{_datadir}/%{vdsm_name}/tests/README diff --git a/vdsm/clientIF.py b/vdsm/clientIF.py index ed2a3c1..481f0d1 100644 --- a/vdsm/clientIF.py +++ b/vdsm/clientIF.py @@ -28,6 +28,7 @@ from storage.dispatcher import Dispatcher from storage.hsm import HSM from vdsm.config import config +from hsmd.config import config as hsmconfig import ksm from momIF import MomThread from vdsm import netinfo @@ -238,7 +239,7 @@ def _initIRS(self): self.irs = None - if config.getboolean('irs', 'irs_enable'): + if hsmconfig.getboolean('irs', 'irs_enable'): try: self.irs = Dispatcher(HSM()) except: diff --git a/vdsm/config.py.in b/vdsm/config.py.in index 038d9bd..49fd3a2 100644 --- a/vdsm/config.py.in +++ b/vdsm/config.py.in @@ -152,98 +152,6 @@ ]), - # Section: [irs] - ('irs', [ - - ('irs_enable', 'true', None), - - ('repository', '@VDSMREPO@', - 'Image repository.'), - - ('hsm_tasks', '%(repository)s/hsm-tasks', None), - - ('images', '/images', None), - - ('irsd', '%(images)s/irsd', None), - - ('images_check_times', '0', - 'Image repository check period (seconds).'), - - ('volume_utilization_percent', '50', None), - - ('volume_utilization_chunk_mb', '1024', None), - - ('vol_size_sample_interval', '60', - 'How often should the volume size be checked (seconds).'), - - ('sd_validate_timeout', '80', - 'Storage domain validate timeout, the maximum number of seconds ' - 'to wait until all the domains will be validated.'), - - ('scsi_rescan_minimal_timeout', '2', - 'The minimum number of seconds to wait for scsi scan to return.'), - - ('scsi_rescan_maximal_timeout', '30', - 'The maximal number of seconds to wait for scsi scan to return.'), - - ('sd_health_check_delay', '10', - 'Storage domain health check delay, the amount of seconds to ' - 'wait between two successive run of the domain health check.'), - - ('nfs_mount_options', 'soft,nosharecache', - 'NFS mount options, comma-separated list (NB: no white space ' - 'allowed!)'), - - ('pools_data_dir', '@POOLSDIR@', None), - - ('vol_extend_policy', 'ON', None), - - ('lock_util_path', '@LIBEXECDIR@', None), - - ('lock_cmd', 'spmprotect.sh', None), - - ('free_lock_cmd', 'spmstop.sh', None), - - ('thread_pool_size', '10', None), - - ('max_tasks', '500', None), - - ('lvm_dev_whitelist', '', None), - - ('md_backup_versions', '30', None), - - ('md_backup_dir', '@BACKUPDIR@', None), - - ('maximum_allowed_pvs', '8', - 'The number of PVs per VG has a hard-coded limit of 10.'), - - ('repo_stats_cache_refresh_timeout', '300', None), - - ('task_resource_default_timeout', '120000', None), - - ('prepare_image_timeout', '600000', None), - - ('gc_blocker_force_collect_interval', '60', None), - - ('maximum_domains_in_pool', '100', - 'Process pool configuration.'), - - ('process_pool_size', '100', None), - - ('process_pool_timeout', '60', None), - - ('process_pool_grace_period', '2', None), - - ('process_pool_max_slots_per_domain', '10', None), - - ('iscsi_default_ifaces', 'default', - 'Comma seperated ifaces to connect with. ' - 'i.e. iser,default'), - - ('use_volume_leases', 'false', - 'Whether to use the volume leases or not.'), - ]), - # Section: [addresses] ('addresses', [ diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py index ca31a43..524eb81 100644 --- a/vdsm/libvirtvm.py +++ b/vdsm/libvirtvm.py @@ -32,6 +32,7 @@ import libvirtev from vdsm import libvirtconnection from vdsm.config import config +from hsmd.config import config as hsmconfig import hooks import caps from vdsm import netinfo @@ -67,7 +68,7 @@ self.updateVolumes = ( utils.AdvancedStatsFunction( self._updateVolumes, - config.getint('irs', 'vol_size_sample_interval'))) + hsmconfig.getint('irs', 'vol_size_sample_interval'))) self.sampleCpu = ( utils.AdvancedStatsFunction( @@ -1198,8 +1199,8 @@ # TODO find a better idea how to calculate this constant only after # config is initialized self._MIN_DISK_REMAIN = \ - ((100 - config.getint('irs', 'volume_utilization_percent')) * - config.getint('irs', 'volume_utilization_chunk_mb') * + ((100 - hsmconfig.getint('irs', 'volume_utilization_percent')) * + hsmconfig.getint('irs', 'volume_utilization_chunk_mb') * 2 ** 20 / 100) self._lastXMLDesc = '<domain><uuid>%s</uuid></domain>' % self.id self._devXmlHash = '0' diff --git a/vdsm/storage/Makefile.am b/vdsm/storage/Makefile.am index cff09be..9af470e 100644 --- a/vdsm/storage/Makefile.am +++ b/vdsm/storage/Makefile.am @@ -20,6 +20,8 @@ SUBDIRS = protect imageRepository +include $(top_srcdir)/build-aux/Makefile.subs + vdsmstoragedir = $(vdsmdir)/storage dist_vdsmstorage_PYTHON = \ __init__.py \ @@ -64,8 +66,29 @@ threadPool.py \ volume.py +dist_hsmpylib_PYTHON = \ + __init__.py + +nodist_hsmpylib_PYTHON = \ + config.py + +nodist_noinst_DATA = \ + hsm.conf.sample + +CLEANFILES = \ + $(nodist_noinst_DATA) \ + $(nodist_hsmpylib_PYTHON) + EXTRA_DIST = \ - 12-vdsm-lvm.rules + 12-vdsm-lvm.rules \ + config.py.in + +hsm.conf.sample: Makefile config.py + @echo " CONFIG $@"; $(PYTHON) $(builddir)/config.py > $@ + +all-local: \ + $(nodist_noinst_DATA) \ + $(nodist_hsmpylib_PYTHON) check-local: PYTHONDONTWRITEBYTECODE=1 $(PYTHON) $(srcdir)/storage_exception.py diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index 26d4380..d59aa05 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -31,7 +31,7 @@ from collections import namedtuple from operator import itemgetter -from vdsm.config import config +from hsmd.config import config from vdsm import constants import misc import fileUtils diff --git a/vdsm/storage/blockVolume.py b/vdsm/storage/blockVolume.py index 98c471b..aaed6c5 100644 --- a/vdsm/storage/blockVolume.py +++ b/vdsm/storage/blockVolume.py @@ -22,7 +22,7 @@ import threading import sanlock -from vdsm.config import config +from hsmd.config import config import storage_exception as se import volume import image diff --git a/vdsm/storage/config.py.in b/vdsm/storage/config.py.in new file mode 100644 index 0000000..160c022 --- /dev/null +++ b/vdsm/storage/config.py.in @@ -0,0 +1,148 @@ +# +# Copyright 2011 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +import os +import textwrap +import ConfigParser + +parameters = [ + # Section: [irs] + ('irs', [ + + ('irs_enable', 'true', None), + + ('repository', '@VDSMREPO@', + 'Image repository.'), + + ('hsm_tasks', '%(repository)s/hsm-tasks', None), + + ('images', '/images', None), + + ('irsd', '%(images)s/irsd', None), + + ('images_check_times', '0', + 'Image repository check period (seconds).'), + + ('volume_utilization_percent', '50', None), + + ('volume_utilization_chunk_mb', '1024', None), + + ('vol_size_sample_interval', '60', + 'How often should the volume size be checked (seconds).'), + + ('sd_validate_timeout', '80', + 'Storage domain validate timeout, the maximum number of seconds ' + 'to wait until all the domains will be validated.'), + + ('scsi_rescan_minimal_timeout', '2', + 'The minimum number of seconds to wait for scsi scan to return.'), + + ('scsi_rescan_maximal_timeout', '30', + 'The maximal number of seconds to wait for scsi scan to return.'), + + ('sd_health_check_delay', '10', + 'Storage domain health check delay, the amount of seconds to ' + 'wait between two successive run of the domain health check.'), + + ('nfs_mount_options', 'soft,nosharecache', + 'NFS mount options, comma-separated list (NB: no white space ' + 'allowed!)'), + + ('pools_data_dir', '@POOLSDIR@', None), + + ('vol_extend_policy', 'ON', None), + + ('lock_util_path', '@LIBEXECDIR@', None), + + ('lock_cmd', 'spmprotect.sh', None), + + ('free_lock_cmd', 'spmstop.sh', None), + + ('thread_pool_size', '10', None), + + ('max_tasks', '500', None), + + ('lvm_dev_whitelist', '', None), + + ('md_backup_versions', '30', None), + + ('md_backup_dir', '@BACKUPDIR@', None), + + ('maximum_allowed_pvs', '8', + 'The number of PVs per VG has a hard-coded limit of 10.'), + + ('repo_stats_cache_refresh_timeout', '300', None), + + ('task_resource_default_timeout', '120000', None), + + ('prepare_image_timeout', '600000', None), + + ('gc_blocker_force_collect_interval', '60', None), + + ('maximum_domains_in_pool', '100', + 'Process pool configuration.'), + + ('process_pool_size', '100', None), + + ('process_pool_timeout', '60', None), + + ('process_pool_grace_period', '2', None), + + ('process_pool_max_slots_per_domain', '10', None), + + ('iscsi_default_ifaces', 'default', + 'Comma seperated ifaces to connect with. ' + 'i.e. iser,default'), + + ('use_volume_leases', 'false', + 'Whether to use the volume leases or not.'), + ]), +] + + +def __prepare_config(): + global config + + for section, keylist in parameters: + config.add_section(section) + for key, value, comment in keylist: + config.set(section, key, value) + +def print_config(): + twp = textwrap.TextWrapper(initial_indent='# ', subsequent_indent='# ') + + print twp.fill("HSM Sample Configuration") + print + + for section, keylist in parameters: + print "[%s]\n" % section + + for key, value, comment in keylist: + if comment: + print twp.fill(comment) + print twp.fill("%s = %s" % (key, value)) + print + +config = ConfigParser.ConfigParser() +__prepare_config() +config.read([os.path.join('@CONFDIR@', 'hsm.conf')]) + +if __name__ == '__main__': + print_config() diff --git a/vdsm/storage/dispatcher.py b/vdsm/storage/dispatcher.py index 7b2ba83..3e27803 100644 --- a/vdsm/storage/dispatcher.py +++ b/vdsm/storage/dispatcher.py @@ -19,7 +19,7 @@ # import logging -from vdsm.config import config +from hsmd.config import config import task import storage_exception as se diff --git a/vdsm/storage/domainMonitor.py b/vdsm/storage/domainMonitor.py index 6f333ee..858a450 100644 --- a/vdsm/storage/domainMonitor.py +++ b/vdsm/storage/domainMonitor.py @@ -24,7 +24,7 @@ import logging import misc -from vdsm.config import config +from hsmd.config import config from sdc import sdCache diff --git a/vdsm/storage/fileVolume.py b/vdsm/storage/fileVolume.py index d36b6fb..2bf41cc 100644 --- a/vdsm/storage/fileVolume.py +++ b/vdsm/storage/fileVolume.py @@ -23,7 +23,7 @@ import sanlock import storage_exception as se -from vdsm.config import config +from hsmd.config import config from sdc import sdCache import outOfProcess as oop import volume diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 4d696bf..476eed8 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -37,7 +37,7 @@ import math import stat -from vdsm.config import config +from hsmd.config import config import sp import sd import blockSD diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 4cd9785..174f9b7 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -29,7 +29,7 @@ import sd import misc import fileUtils -from vdsm.config import config +from hsmd.config import config import storage_exception as se import task from threadLocal import vars diff --git a/vdsm/storage/iscsi.py b/vdsm/storage/iscsi.py index a0c3067..4723746 100644 --- a/vdsm/storage/iscsi.py +++ b/vdsm/storage/iscsi.py @@ -32,7 +32,7 @@ from vdsm import constants import misc -from vdsm.config import config +from hsmd.config import config import devicemapper from threading import RLock diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py index b3abc2c..8c77a6a 100644 --- a/vdsm/storage/lvm.py +++ b/vdsm/storage/lvm.py @@ -41,7 +41,7 @@ import misc import multipath import storage_exception as se -from vdsm.config import config +from hsmd.config import config import devicemapper log = logging.getLogger("Storage.LVM") diff --git a/vdsm/storage/outOfProcess.py b/vdsm/storage/outOfProcess.py index c0d85d4..71ec11a 100644 --- a/vdsm/storage/outOfProcess.py +++ b/vdsm/storage/outOfProcess.py @@ -19,7 +19,7 @@ # import types -from vdsm.config import config +from hsmd.config import config import threading from functools import partial diff --git a/vdsm/storage/resourceFactories.py b/vdsm/storage/resourceFactories.py index 02aba0f..f7225a0 100644 --- a/vdsm/storage/resourceFactories.py +++ b/vdsm/storage/resourceFactories.py @@ -19,7 +19,7 @@ # import os -from vdsm.config import config +from hsmd.config import config import logging import lvm import resourceManager as rm diff --git a/vdsm/storage/safelease.py b/vdsm/storage/safelease.py index 88a4eae..5184224 100644 --- a/vdsm/storage/safelease.py +++ b/vdsm/storage/safelease.py @@ -19,7 +19,7 @@ # import os -from vdsm.config import config +from hsmd.config import config import misc import subprocess import sanlock diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index 17b6ada..52f80aa 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -35,7 +35,7 @@ import outOfProcess as oop from persistentDict import unicodeEncoder, unicodeDecoder -from vdsm.config import config +from hsmd.config import config DOMAIN_MNT_POINT = 'mnt' DOMAIN_META_DATA = 'dom_md' diff --git a/vdsm/storage/sdc.py b/vdsm/storage/sdc.py index f2f4534..816f95b 100644 --- a/vdsm/storage/sdc.py +++ b/vdsm/storage/sdc.py @@ -24,7 +24,7 @@ """ import logging import threading -from vdsm.config import config +from hsmd.config import config import multipath import lvm diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index c9489b8..77452e9 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -39,7 +39,7 @@ import sd import misc import fileUtils -from vdsm.config import config +from hsmd.config import config from sdc import sdCache import storage_exception as se from persistentDict import DictValidator, unicodeEncoder, unicodeDecoder diff --git a/vdsm/storage/storage_mailbox.py b/vdsm/storage/storage_mailbox.py index f678a04..4743940 100644 --- a/vdsm/storage/storage_mailbox.py +++ b/vdsm/storage/storage_mailbox.py @@ -27,7 +27,7 @@ import logging import uuid -from vdsm.config import config +from hsmd.config import config import sd import misc import task diff --git a/vdsm/storage/task.py b/vdsm/storage/task.py index a596494..0ea8a4b 100644 --- a/vdsm/storage/task.py +++ b/vdsm/storage/task.py @@ -56,7 +56,7 @@ import resourceManager from threadLocal import vars from weakref import proxy -from vdsm.config import config +from hsmd.config import config import outOfProcess as oop from logUtils import SimpleLogAdapter diff --git a/vdsm/storage/taskManager.py b/vdsm/storage/taskManager.py index 3bc12f3..6ce82f2 100644 --- a/vdsm/storage/taskManager.py +++ b/vdsm/storage/taskManager.py @@ -21,7 +21,7 @@ import os import logging -from vdsm.config import config +from hsmd.config import config import storage_exception as se from task import Task, Job, TaskCleanType from threadPool import ThreadPool diff --git a/vdsm/supervdsmServer.py b/vdsm/supervdsmServer.py index a1bf8e6..39ff023 100755 --- a/vdsm/supervdsmServer.py +++ b/vdsm/supervdsmServer.py @@ -50,7 +50,7 @@ from storage.devicemapper import _removeMapping, _getPathsStatus import storage.misc import configNetwork -from vdsm.config import config +from hsmd.config import config as hsmconfig import tc import ksm import mkimage @@ -62,7 +62,7 @@ _UDEV_RULE_FILE_NAME = _UDEV_RULE_FILE_DIR + _UDEV_RULE_FILE_PREFIX + \ "%s-%s" + _UDEV_RULE_FILE_EXT -RUN_AS_TIMEOUT = config.getint("irs", "process_pool_timeout") +RUN_AS_TIMEOUT = hsmconfig.getint("irs", "process_pool_timeout") class Timeout(RuntimeError): diff --git a/vdsm/utils.py b/vdsm/utils.py index b4614e7..9d07ce7 100644 --- a/vdsm/utils.py +++ b/vdsm/utils.py @@ -43,7 +43,7 @@ import ethtool import constants -from config import config +from hsmd.config import config as hsmconfig _THP_STATE_PATH = '/sys/kernel/mm/transparent_hugepage/enabled' if not os.path.exists(_THP_STATE_PATH): @@ -644,7 +644,7 @@ class ImagePathStatus(threading.Thread): def __init__(self, cif, interval=None): if interval is None: - interval = config.getint('irs', 'images_check_times') + interval = hsmconfig.getint('irs', 'images_check_times') self._interval = interval self._cif = cif self.storageDomains = {} diff --git a/vdsm/vm.py b/vdsm/vm.py index 7a421bf..af681ed 100644 --- a/vdsm/vm.py +++ b/vdsm/vm.py @@ -30,6 +30,7 @@ from vdsm import utils from vdsm.define import NORMAL, ERROR, doneCode, errCode from vdsm.config import config +from hsmd.config import config as hsmconfig import kaxmlrpclib from logUtils import SimpleLogAdapter import libvirt @@ -800,7 +801,7 @@ if newSize is None: # newSize is always in megabytes - newSize = (config.getint('irs', 'volume_utilization_chunk_mb') + + newSize = (hsmconfig.getint('irs', 'volume_utilization_chunk_mb') + ((vmDrive.apparentsize + constants.MEGAB - 1) / constants.MEGAB)) -- To view, visit http://gerrit.ovirt.org/9345 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8f62a4108275a1295f122025973a767f79b75780 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> Gerrit-Reviewer: Shu Ming <shuming(a)linux.vnet.ibm.com>

3 8

Change in vdsm[master]: PEP 8: fix "E302 expected 2 blank lines" in tests/apiTests.p...
by zhshzhou＠linux.vnet.ibm.com 20 Nov '12

20 Nov '12

Zhou Zheng Sheng has uploaded a new change for review. Change subject: PEP 8: fix "E302 expected 2 blank lines" in tests/apiTests.py ...................................................................... PEP 8: fix "E302 expected 2 blank lines" in tests/apiTests.py On my pep8 version 1.3.3, it reports error on this file and fails the build. The patch fixes this. Change-Id: I27608d7d00c7d8e72a043eba387cd4e9f03fa837 Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- M tests/apiTests.py 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/46/9346/1 diff --git a/tests/apiTests.py b/tests/apiTests.py index 4b1a56b..ff0a785 100644 --- a/tests/apiTests.py +++ b/tests/apiTests.py @@ -156,6 +156,7 @@ raise server.start() + class APITest(TestCaseBase): def expectAPI(self, obj, meth, retval): global _fakeret -- To view, visit http://gerrit.ovirt.org/9346 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I27608d7d00c7d8e72a043eba387cd4e9f03fa837 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com>

3 8

Change in vdsm[master]: Added an option to create a watchdog device.
by shaohef＠linux.vnet.ibm.com 20 Nov '12

20 Nov '12

ShaoHe Feng has uploaded a new change for review. Change subject: Added an option to create a watchdog device. ...................................................................... Added an option to create a watchdog device. A support for a watchdog device was added to the "devices" configuration. Also add the watchdog event callback. model of watchdog: 'i6300esb' default model, emulating a PCI Intel 6300ESB 'ib700' emulating an ISA iBase IB700 action of watchdog: 'reset' default, forcefully reset the guest 'shutdown' gracefully shutdown the guest (not recommended) 'poweroff' forcefully power off the guest 'pause' pause the guest 'none' do nothing 'dump' automatically dump the guest The watchdog device can used to detect guest crash, and if choose 'dump' action, the libvirt will dump the guest automatically. The directory to save dump files can be configured by auto_dump_path in file /etc/libvirt/qemu.conf. The watchdog device requires an additional driver and management daemon in the guest. Just enabling the watchdog in the vdsm "devices" configuration does not do anything useful on its own. Change-Id: I2b7970a9050ab0279fe03371b9a77692fba30af8 Signed-off-by: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> --- M vdsm/libvirtconnection.py M vdsm/libvirtev.py M vdsm/libvirtvm.py M vdsm/vm.py 4 files changed, 102 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/35/7535/1 diff --git a/vdsm/libvirtconnection.py b/vdsm/libvirtconnection.py index e525996..25e2494 100644 --- a/vdsm/libvirtconnection.py +++ b/vdsm/libvirtconnection.py @@ -63,6 +63,12 @@ elif eventid == libvirt.VIR_DOMAIN_EVENT_ID_BLOCK_JOB: path, type, status = args[:-1] v._onBlockJobEvent(path, type, status) + elif eventid == libvirt.VIR_DOMAIN_EVENT_ID_WATCHDOG: + wdargs = args[:-1] + action = libvirtev.watchdogActionToString(wdargs[0]) + cif.log.debug("watchdog event: Domain %s(%s). Action: %s", + dom.name(), dom.ID(), action) + v._onWatchdogEvent(dom.name(), dom.ID(), action) else: v.log.warning('unknown eventid %s args %s', eventid, args) except: @@ -118,7 +124,8 @@ libvirt.VIR_DOMAIN_EVENT_ID_RTC_CHANGE, libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON, libvirt.VIR_DOMAIN_EVENT_ID_GRAPHICS, - libvirt.VIR_DOMAIN_EVENT_ID_BLOCK_JOB): + libvirt.VIR_DOMAIN_EVENT_ID_BLOCK_JOB, + libvirt.VIR_DOMAIN_EVENT_ID_WATCHDOG): conn.domainEventRegisterAny(None, ev, __eventCallback, (cif, ev)) for name in dir(libvirt.virConnect): diff --git a/vdsm/libvirtev.py b/vdsm/libvirtev.py index 92a7a0e..b12b916 100644 --- a/vdsm/libvirtev.py +++ b/vdsm/libvirtev.py @@ -444,6 +444,16 @@ return eventStrings[event] +def watchdogActionToString(action): + actionStrings = ("No action, watchdog ignored", + "Guest CPUs are paused", + "Guest CPUs are reset", + "Guest is forcably powered off", + "Guest is requested to gracefully shutdown", + "No action, a debug message logged") + return actionStrings[action] + + def myDomainEventCallback1(conn, dom, event, detail, opaque): print ("myDomainEventCallback1 EVENT: Domain %s(%s) %s %d" % (dom.name(), dom.ID(), eventToString(event), detail)) diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py index e95e94b..cc75d78 100644 --- a/vdsm/libvirtvm.py +++ b/vdsm/libvirtvm.py @@ -1188,6 +1188,20 @@ m.setAttribute('model', self.specParams['model']) return m +class WatchdogDevice(LibvirtVmDevice): + def getXML(self): + """ + Create domxml for a watchdog device. + + <watchdog model='i6300esb' action='dump'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> + </watchdog> + """ + m = self.createXmlElem(self.type, None, ['address']) + m.setAttribute('model', self.specParams['model']) + if 'action' in self.specParams: + m.setAttribute('action', self.specParams['action']) + return m class RedirDevice(LibvirtVmDevice): def getXML(self): @@ -1331,6 +1345,7 @@ self._getUnderlyingVideoDeviceInfo() self._getUnderlyingControllerDeviceInfo() self._getUnderlyingBalloonDeviceInfo() + self._getUnderlyingWatchdogDeviceInfo() # Obtain info of all unknown devices. Must be last! self._getUnderlyingUnknownDeviceInfo() @@ -1420,6 +1435,7 @@ vm.CONTROLLER_DEVICES: ControllerDevice, vm.GENERAL_DEVICES: GeneralDevice, vm.BALLOON_DEVICES: BalloonDevice, + vm.WATCHDOG_DEVICES: WatchdogDevice, vm.REDIR_DEVICES: RedirDevice} for devType, devClass in devMap.items(): @@ -2031,6 +2047,10 @@ self.saveState() + def _onWatchdogEvent(self, name, vmid, action): + self.log.debug("Watchdog event comes from guest %s(%s). " + "Action: %s", name, vmid, action) + def changeCD(self, drivespec): return self._changeBlockDev('cdrom', 'hdc', drivespec) @@ -2374,6 +2394,38 @@ dev['address'] = address dev['alias'] = alias + def _getUnderlyingWatchdogDeviceInfo(self): + """ + Obtain watchdog device info from libvirt. + """ + watchdogxml = _domParseStr(self._lastXMLDesc).childNodes[0]. \ + getElementsByTagName('devices')[0]. \ + getElementsByTagName('watchdog') + for x in watchdogxml: + action = x.getAttribute('action') + # Ignore watchdog device without address. + if not x.getElementsByTagName('address'): + continue + + address = self._getUnderlyingDeviceAddress(x) + alias = x.getElementsByTagName('alias')[0].getAttribute('name') + + for dev in self._devices[vm.WATCHDOG_DEVICES]: + if not hasattr(dev, 'address'): + dev.address = address + dev.alias = alias + if not "action" in dev.specParams: + dev.specParams['action'] = action + + for dev in self.conf['devices']: + if ((dev['type'] == vm.WATCHDOG_DEVICES) and + not dev.get('address')): + dev['address'] = address + dev['alias'] = alias + if ((dev['type'] == vm.WATCHDOG_DEVICES) and + not "action" in dev['specParams']): + dev['specParams']['action'] = action + def _getUnderlyingVideoDeviceInfo(self): """ Obtain video devices info from libvirt. diff --git a/vdsm/vm.py b/vdsm/vm.py index 949a3ee..d24d1a4 100644 --- a/vdsm/vm.py +++ b/vdsm/vm.py @@ -45,6 +45,7 @@ GENERAL_DEVICES = 'general' BALLOON_DEVICES = 'balloon' REDIR_DEVICES = 'redir' +WATCHDOG_DEVICES = 'watchdog' """ A module containing classes needed for VM communication. @@ -310,7 +311,8 @@ self._devices = {DISK_DEVICES: [], NIC_DEVICES: [], SOUND_DEVICES: [], VIDEO_DEVICES: [], CONTROLLER_DEVICES: [], GENERAL_DEVICES: [], - BALLOON_DEVICES: [], REDIR_DEVICES: []} + BALLOON_DEVICES: [], REDIR_DEVICES: [], + WATCHDOG_DEVICES: [],} def _get_lastStatus(self): SHOW_PAUSED_STATES = ('Powering down', 'RebootInProgress', 'Up') @@ -381,11 +383,24 @@ 'index': 0, 'truesize': 0}) return removables + def _removeRedundantDevices(self, devices): + counters = 0 + for dev in self.conf['devices']: + if (dev['type'] == devices): + counters += 1 + if (counters > 1): + self.conf['devices'].remove(dev) + if (counters > 1): + self.log.warning("%d redundant %s devices are removed", + counters-1, devices) + return counters + def getConfDevices(self): devices = {DISK_DEVICES: [], NIC_DEVICES: [], SOUND_DEVICES: [], VIDEO_DEVICES: [], CONTROLLER_DEVICES: [], GENERAL_DEVICES: [], - BALLOON_DEVICES: [], REDIR_DEVICES: []} + BALLOON_DEVICES: [], REDIR_DEVICES: [], + WATCHDOG_DEVICES: []} for dev in self.conf.get('devices'): try: devices[dev['type']].append(dev) @@ -417,8 +432,23 @@ devices[CONTROLLER_DEVICES] = self.getConfController() devices[GENERAL_DEVICES] = [] devices[BALLOON_DEVICES] = [] + devices[WATCHDOG_DEVICES] = [] devices[REDIR_DEVICES] = [] else: + # libvirt only support one watchdog device + # before getting devices list, check WATCHDOG_DEVICES and + # try to remove redundant watchdog devices + self._removeRedundantDevices(WATCHDOG_DEVICES) + # must assign the model for watchdog device to libvirt + # but the action can be assigned default by libvirt + # so ignore the action + for dev in self.conf['devices']: + if (dev['type'] == WATCHDOG_DEVICES): + if not 'specParams' in dev: + dev['specParams'] = {} + if not 'model' in dev['specParams']: + dev['specParams']['model'] = 'i6300esb' + devices = self.getConfDevices() # Normalize vdsm images -- To view, visit http://gerrit.ovirt.org/7535 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2b7970a9050ab0279fe03371b9a77692fba30af8 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com>

12 35

Change in vdsm[master]: avoid creating hangover in storage domain cache after format...
by zhshzhou＠linux.vnet.ibm.com 20 Nov '12

20 Nov '12

Zhou Zheng Sheng has uploaded a new change for review. Change subject: avoid creating hangover in storage domain cache after formatting ...................................................................... avoid creating hangover in storage domain cache after formatting HSM._recycle(self, dom) tries to delete the storage domain from cache by calling "sdCache.manuallyRemoveDomain(dom.sdUUID)". This is OK, but when it invokes "dom.format(dom.sdUUID)" to format storage domain, the "dom" is a proxy object and have to look for the true storage domain object by calling 'StorageDomainCache._realProduce()', thus cause the storage domain added to the cache again. This bug will prevent us re-using the storage domain UUID even though we format it. This patch invokes the format method of the proxy object first, then delete the storage domain from cache, and can avoid creating hangovers. Change-Id: Ie13a97ad6830553cbd099189dfb82d6f02ce82fe Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=806774 Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- M vdsm/storage/hsm.py 1 file changed, 7 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/44/8144/1 diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index a89274a..8512f08 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -2192,11 +2192,14 @@ #TODO: Remove this function when formatStorageDomain() is removed. def _recycle(self, dom): try: - sdCache.manuallyRemoveDomain(dom.sdUUID) - except KeyError: - self.log.warn("Storage domain %s doesn't exist in cache. Trying recycle leftovers ...", dom.sdUUID) + dom.format(dom.sdUUID) + finally: + try: + sdCache.manuallyRemoveDomain(dom.sdUUID) + except KeyError: + self.log.warn("Storage domain %s doesn't exist in cache. " + "Trying recycle leftovers ...", dom.sdUUID) - dom.format(dom.sdUUID) @public def formatStorageDomain(self, sdUUID, autoDetach = False, options = None): -- To view, visit http://gerrit.ovirt.org/8144 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie13a97ad6830553cbd099189dfb82d6f02ce82fe Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com>

8 17

Re: [oVirt Jenkins] vdsm_unit_tests_manual_gerrit - Build # 87 - Still Failing!
by eedri＠redhat.com 20 Nov '12

20 Nov '12

Hi, This error was caused by jenkins job misconfiguration, and was fixed now. all failed patches were re-triggered. Eyal. ----- Original Message ----- > From: "Eduardo Warszawski" <ewarszaw(a)redhat.com> > To: "Eyal Edri" <eedri(a)redhat.com> > Sent: Tuesday, November 20, 2012 1:46:00 PM > Subject: Fwd: [oVirt Jenkins] vdsm_unit_tests_manual_gerrit - Build # 87 - Still Failing! > > > > ----- Forwarded Message ----- > From: "Jenkins oVirt Server" <jenkins(a)ovirt.org> > To: abaron(a)redhat.com, danken(a)redhat.com, > vdsm-patches(a)lists.fedorahosted.org, ewarszaw(a)redhat.com > Sent: Tuesday, November 20, 2012 1:36:13 PM > Subject: [oVirt Jenkins] vdsm_unit_tests_manual_gerrit - Build # 87 - > Still Failing! > > > BUILD FAILURE > Build URL: > http://jenkins.ovirt.org/job/vdsm_unit_tests_manual_gerrit/87/ > Project: vdsm_unit_tests_manual_gerrit > Date of build: Tue, 20 Nov 2012 06:36:10 -0500 > Build duration: 2.7 sec > > CHANGE SET > Revision aa8a607d9b9d4e9ce5b851334d4da7af0ec46389 by ewarszaw: (Fix > block volume atomicity creation issue.) > change: edit vdsm/storage/blockVolume.py >

1 0

Change in vdsm[master]: pep8: fix a flaw introduced by commit 2c0fd0796
by Dan Kenigsberg 20 Nov '12

20 Nov '12

Dan Kenigsberg has uploaded a new change for review. Change subject: pep8: fix a flaw introduced by commit 2c0fd0796 ...................................................................... pep8: fix a flaw introduced by commit 2c0fd0796 Change-Id: Idf75d82ef9c54b09b2b7279783610713d7f165f0 Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M tests/apiTests.py 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/57/9357/1 diff --git a/tests/apiTests.py b/tests/apiTests.py index 4b1a56b..ff0a785 100644 --- a/tests/apiTests.py +++ b/tests/apiTests.py @@ -156,6 +156,7 @@ raise server.start() + class APITest(TestCaseBase): def expectAPI(self, obj, meth, retval): global _fakeret -- To view, visit http://gerrit.ovirt.org/9357 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Idf75d82ef9c54b09b2b7279783610713d7f165f0 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

2 5

Change in vdsm[master]: Adding hooks support for NIC hotplug
by itzikb＠mellanox.com 20 Nov '12

20 Nov '12

Itzik Brown has uploaded a new change for review. Change subject: Adding hooks support for NIC hotplug ...................................................................... Adding hooks support for NIC hotplug Adding the ability to write hooks for events of NIC hotplug and hotunplug. Hooks are: before_nic_plug and after_nic_unplug Change-Id: I253104ee9831a881e2fb06f0a658631662611d77 Signed-off-by: Itzik Brown <itzikb(a)mellanox.com> --- M vdsm.spec.in M vdsm/hooks.py M vdsm/libvirtvm.py M vdsm_hooks/Makefile.am 4 files changed, 15 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/24/7224/1 diff --git a/vdsm.spec.in b/vdsm.spec.in index 23afd6b..0d5f82d 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -658,6 +658,8 @@ %dir %{_libexecdir}/%{vdsm_name}/hooks/after_vm_destroy %dir %{_libexecdir}/%{vdsm_name}/hooks/before_vm_set_ticket %dir %{_libexecdir}/%{vdsm_name}/hooks/after_vm_set_ticket +%dir %{_libexecdir}/%{vdsm_name}/hooks/before_nic_plug +%dir %{_libexecdir}/%{vdsm_name}/hooks/after_nic_unplug %dir %{_libexecdir}/%{vdsm_name}/hooks/before_vdsm_start %dir %{_libexecdir}/%{vdsm_name}/hooks/after_vdsm_stop %{_datadir}/%{vdsm_name}/addNetwork diff --git a/vdsm/hooks.py b/vdsm/hooks.py index 5b79cee..64263b3 100644 --- a/vdsm/hooks.py +++ b/vdsm/hooks.py @@ -174,6 +174,14 @@ raiseError=False, params=params) +def before_nic_hotplug(domxml, vmconf={}): + return _runHooksDir(domxml, 'before_vm_hotplug', vmconf=vmconf) + + +def after_nic_unplug(domxml, vmconf={}): + return _runHooksDir(domxml, 'after_vm_unplug', vmconf=vmconf) + + def before_vdsm_start(): return _runHooksDir(None, 'before_vdsm_start', raiseError=False) diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py index 3b7cfc5..4ae54f7 100644 --- a/vdsm/libvirtvm.py +++ b/vdsm/libvirtvm.py @@ -1395,12 +1395,14 @@ nicParams = params.get('nic', {}) nic = NetworkInterfaceDevice(self.conf, self.log, **nicParams) nicXml = nic.getXML().toprettyxml(encoding='utf-8') + nicXml = hooks.before_nic_hotplug(nicXml, self.conf) self.log.debug("Hotplug NIC xml: %s" % (nicXml)) try: self._dom.attachDevice(nicXml) except libvirt.libvirtError, e: self.log.error("Hotplug failed", exc_info=True) + nicXml = hooks.after_nic_unplug(nicXml, self.conf) if e.get_error_code() == libvirt.VIR_ERR_NO_DOMAIN: return errCode['noVM'] return {'status' : {'code': errCode['hotplugNic']['status']['code'], @@ -1454,6 +1456,7 @@ try: self._dom.detachDevice(nicXml) + hooks.after_nic_unplug(nicXml, self.conf) except libvirt.libvirtError, e: self.log.error("Hotunplug failed", exc_info=True) if e.get_error_code() == libvirt.VIR_ERR_NO_DOMAIN: diff --git a/vdsm_hooks/Makefile.am b/vdsm_hooks/Makefile.am index e6a8280..36919d9 100644 --- a/vdsm_hooks/Makefile.am +++ b/vdsm_hooks/Makefile.am @@ -74,6 +74,8 @@ after_vm_destroy \ before_vm_set_ticket \ after_vm_set_ticket \ + before_nic_plug \ + after_nic_unplug \ before_vdsm_start \ after_vdsm_stop -- To view, visit http://gerrit.ovirt.org/7224 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I253104ee9831a881e2fb06f0a658631662611d77 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Itzik Brown <itzikb(a)mellanox.com>

5 20

Change in vdsm[master]: hsm: use misc.parseBool instead of ad-hoc bool convertion
by zhshzhou＠linux.vnet.ibm.com 20 Nov '12

20 Nov '12

Zhou Zheng Sheng has uploaded a new change for review. Change subject: hsm: use misc.parseBool instead of ad-hoc bool convertion ...................................................................... hsm: use misc.parseBool instead of ad-hoc bool convertion Change-Id: I2ce0ec34d10380c8e1e8bd94e17ef436deb28579 Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- M vdsm/storage/hsm.py 1 file changed, 1 insertion(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/82/8482/1 diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 50d6ce1..e1f61c7 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -1705,9 +1705,7 @@ lvm.createVG(vgname, devices, blockSD.STORAGE_UNREADY_DOMAIN_TAG, metadataSize=blockSD.VG_METADATASIZE, - force=(force == True) or \ - (type(force) == type("") and \ - (force.capitalize() == "True"))) + force=misc.parseBool(force)) return dict(uuid=lvm.getVG(vgname).uuid) -- To view, visit http://gerrit.ovirt.org/8482 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2ce0ec34d10380c8e1e8bd94e17ef436deb28579 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com>

3 7

Change in vdsm[master]: tests: Support parallel testing in apiTests
by agl＠us.ibm.com 19 Nov '12

19 Nov '12

Adam Litke has uploaded a new change for review. Change subject: tests: Support parallel testing in apiTests ...................................................................... tests: Support parallel testing in apiTests The apiTests module tests the JSON-RPC server by creating a server instance on a specific port and then connecting clients to that port. Unfortunately, this means that only one instance of the test can be running on a host at any given time. Our Jenkins infrastructure would like to run parallel tests. To enable this, we allow the test to find an open port to use. Change-Id: Icee7a7cac15ba271374b622d4227d5c15e190341 Signed-off-by: Adam Litke <agl(a)us.ibm.com> --- M tests/apiTests.py 1 file changed, 17 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/42/9342/1 diff --git a/tests/apiTests.py b/tests/apiTests.py index c51c9fc..4b1a56b 100644 --- a/tests/apiTests.py +++ b/tests/apiTests.py @@ -22,6 +22,7 @@ import os import os.path import socket +import errno import json import struct @@ -123,6 +124,7 @@ 1. Override the API so we can program our own return values 2. Start an embedded server to process our requests """ + global port log = logging.getLogger('apiTests') handler = logging.StreamHandler() fmt_str = "%(asctime)s - %(name)s - %(levelname)s - %(message)s" @@ -137,9 +139,22 @@ # Bridge imports the API module so we must set up the fake API first import Bridge bridge = Bridge.DynamicBridge(schema) - server = BindingJsonRpc.BindingJsonRpc(bridge, ip, port) - server.start() + # Support parallel testing. Try hard to find an open port to use + while True: + try: + server = BindingJsonRpc.BindingJsonRpc(bridge, ip, port) + break + except socket.error as ex: + if ex.errno == errno.EADDRINUSE: + port += 1 + if port > 65535: + raise socket.error( + errno.EADDRINUSE, + "Can not find available port to bind") + else: + raise + server.start() class APITest(TestCaseBase): def expectAPI(self, obj, meth, retval): -- To view, visit http://gerrit.ovirt.org/9342 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Icee7a7cac15ba271374b622d4227d5c15e190341 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <agl(a)us.ibm.com>

3 5

Change in vdsm[master]: Verify cleaning of supervdsm subprocesses after terminated t...
by ybronhei＠redhat.com 19 Nov '12

19 Nov '12

Yaniv Bronhaim has uploaded a new change for review. Change subject: Verify cleaning of supervdsm subprocesses after terminated to avoid defunct ...................................................................... Verify cleaning of supervdsm subprocesses after terminated to avoid defunct After killing subprocess register its pid to zombieRepear to verify that svdsm process will wait and clear the pid when the process is terminated. Change-Id: I21837f6f707e5e6f64fbb6968ac25294622bd22b Signed-off-by: Yaniv Bronhaim <ybronhei(a)redhat.com> --- M vdsm/supervdsmServer.py 1 file changed, 2 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/36/9236/1 diff --git a/vdsm/supervdsmServer.py b/vdsm/supervdsmServer.py index 34fefdb..0979ba0 100755 --- a/vdsm/supervdsmServer.py +++ b/vdsm/supervdsmServer.py @@ -30,6 +30,7 @@ from multiprocessing import Pipe, Process import storage.misc as misc from vdsm import utils +import zombieReaper try: from gluster import cli as gcli @@ -199,6 +200,7 @@ res, err = pipe.recv() pipe.send("Bye") proc.terminate() + zombieReaper.autoReapPID(proc.pid) if err is not None: raise err -- To view, visit http://gerrit.ovirt.org/9236 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I21837f6f707e5e6f64fbb6968ac25294622bd22b Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim <ybronhei(a)redhat.com>

2 3

Change in vdsm[master]: [WIP] used gluster cli xml output
by barumuga＠redhat.com 19 Nov '12

19 Nov '12

Hello Ayal Baron, Timothy Asir, Saggi Mizrahi, Federico Simoncelli, Dan Kenigsberg, I'd like you to do a code review. Please visit http://gerrit.ovirt.org/7951 to review the following change. Change subject: [WIP] used gluster cli xml output ...................................................................... [WIP] used gluster cli xml output Below verbs are using gluster cli xml output glusterVolumesList glusterVolumeCreate glusterVolumeStart glusterVolumeStop glusterVolumeDelete glusterVolumeSet glusterVolumeReset glusterVolumeAddBrick glusterVolumeRemoveBrickForce glusterHostAdd glusterHostRemove Change-Id: I364930675dcc433dfc0432a343fc66e91721e801 Signed-off-by: Bala.FA <barumuga(a)redhat.com> --- M tests/gluster_cli_tests.py M vdsm/gluster/api.py M vdsm/gluster/cli.py 3 files changed, 195 insertions(+), 179 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/51/7951/1 diff --git a/tests/gluster_cli_tests.py b/tests/gluster_cli_tests.py index bb88918..e5fe1ce 100644 --- a/tests/gluster_cli_tests.py +++ b/tests/gluster_cli_tests.py @@ -39,41 +39,98 @@ raise SkipTest("vdsm-gluster not found") def _parseVolumeInfo_empty_test(self): - out = ['No volumes present'] - self.assertFalse(gcli._parseVolumeInfo(out)) + out = """<?xml version="1.0" encoding="UTF-8" standalone="yes"?> +<cliOutput> + <opRet>0</opRet> + <opErrno>0</opErrno> + <opErrstr/> + <volInfo/> +</cliOutput> +""" + tree = etree.fromstring(out) + self.assertFalse(gcli._parseVolumeInfo(tree)) def _parseVolumeInfo_test(self): - out = [' ', - 'Volume Name: music', - 'Type: Distribute', - 'Volume ID: 1e7d2638-fb77-4325-94fd-3242650a013c', - 'Status: Stopped', - 'Number of Bricks: 2', - 'Transport-type: tcp', - 'Bricks:', - 'Brick1: 192.168.122.167:/tmp/music-b1', - 'Brick2: 192.168.122.167:/tmp/music-b2', - 'Options Reconfigured:', - 'auth.allow: *'] - volumeInfo = gcli._parseVolumeInfo(out) - for volumeName in volumeInfo: - if volumeName == 'music': - self.assertEquals(volumeInfo['music']['volumeName'], - volumeName) - self.assertEquals(volumeInfo['music']['uuid'], - '1e7d2638-fb77-4325-94fd-3242650a013c') - self.assertEquals(volumeInfo['music']['volumeType'], - 'DISTRIBUTE') - self.assertEquals(volumeInfo['music']['volumeStatus'], - 'STOPPED') - self.assertEquals(volumeInfo['music']['transportType'], - ['TCP']) - self.assertEquals(volumeInfo['music']['bricks'], - ['192.168.122.167:/tmp/music-b1', - '192.168.122.167:/tmp/music-b2']) - self.assertEquals(volumeInfo['music']['brickCount'], '2') - self.assertEquals(volumeInfo['music']['options'], - {'auth.allow': '*'}) + out = """<?xml version="1.0" encoding="UTF-8" standalone="yes"?> +<cliOutput> + <opRet>0</opRet> + <opErrno>0</opErrno> + <opErrstr/> + <volInfo> + <volumes> + <volume> + <name>music</name> + <id>b3114c71-741b-4c6f-a39e-80384c4ea3cf</id> + <type>2</type> + <status>1</status> + <brickCount>2</brickCount> + <distCount>2</distCount> + <stripeCount>1</stripeCount> + <replicaCount>2</replicaCount> + <transport>0</transport> + <bricks> + <brick>192.168.122.2:/tmp/music-b1</brick> + <brick>192.168.122.2:/tmp/music-b2</brick> + </bricks> + <optCount>1</optCount> + <options> + <option> + <name>auth.allow</name> + <value>*</value> + </option> + </options> + </volume> + <volume> + <name>test1</name> + <id>2e562614-00ad-4acb-a630-4429049a9818</id> + <type>0</type> + <status>2</status> + <brickCount>2</brickCount> + <distCount>1</distCount> + <stripeCount>1</stripeCount> + <replicaCount>1</replicaCount> + <transport>1</transport> + <bricks> + <brick>192.168.122.2:/tmp/test1-b1</brick> + <brick>192.168.122.2:/tmp/test1-b2</brick> + </bricks> + <optCount>0</optCount> + <options/> + </volume> + <count>2</count> + </volumes> + </volInfo> +</cliOutput> +""" + tree = etree.fromstring(out) + oVolumeInfo = {'music': + {'brickCount': '2', + 'bricks': ['192.168.122.2:/tmp/music-b1', + '192.168.122.2:/tmp/music-b2'], + 'distCount': '2', + 'options': {'auth.allow': '*'}, + 'replicaCount': '2', + 'stripeCount': '1', + 'transportType': [gcli.TransportType.TCP], + 'uuid': 'b3114c71-741b-4c6f-a39e-80384c4ea3cf', + 'volumeName': 'music', + 'volumeStatus': gcli.VolumeStatus.ONLINE, + 'volumeType': '2'}, + 'test1': + {'brickCount': '2', + 'bricks': ['192.168.122.2:/tmp/test1-b1', + '192.168.122.2:/tmp/test1-b2'], + 'distCount': '1', + 'options': {}, + 'replicaCount': '1', + 'stripeCount': '1', + 'transportType': [gcli.TransportType.RDMA], + 'uuid': '2e562614-00ad-4acb-a630-4429049a9818', + 'volumeName': 'test1', + 'volumeStatus': gcli.VolumeStatus.OFFLINE, + 'volumeType': '0'}} + volumeInfo = gcli._parseVolumeInfo(tree) + self.assertEquals(volumeInfo, oVolumeInfo) def test_parseVolumeInfo(self): self._parseVolumeInfo_empty_test() diff --git a/vdsm/gluster/api.py b/vdsm/gluster/api.py index ba1f374..3c1bc58 100644 --- a/vdsm/gluster/api.py +++ b/vdsm/gluster/api.py @@ -42,11 +42,6 @@ return wrapper -class VolumeStatus(): - ONLINE = 'ONLINE' - OFFLINE = 'OFFLINE' - - class GlusterApi(object): """ The gluster interface of vdsm. @@ -120,35 +115,14 @@ @exportAsVerb def volumesList(self, volumeName=None, options=None): - """ - Returns: - {'status' : {'code': CODE, 'message': MESSAGE}, - 'volumes': {VOLUMENAME: {'brickCount': BRICKCOUNT, - 'bricks': [BRICK1, BRICK2, ...], - 'options': {OPTION: VALUE, ...}, - 'transportType': [TCP, RDMA, ...], - 'uuid': UUID, - 'volumeName': NAME, - 'volumeStatus': STATUS, - 'volumeType': TYPE}, ...}} - """ - volumeInfoDict = self.svdsmProxy.glusterVolumeInfo(volumeName) - for name, info in volumeInfoDict.iteritems(): - info["volumeType"] = info["volumeType"].replace("-", "_") - if info["volumeStatus"] == "STARTED": - info["volumeStatus"] = VolumeStatus.ONLINE - else: - info["volumeStatus"] = VolumeStatus.OFFLINE - return {'volumes': volumeInfoDict} + return {'volumes': self.svdsmProxy.glusterVolumeInfo(volumeName)} @exportAsVerb def volumeCreate(self, volumeName, brickList, replicaCount=0, stripeCount=0, transportList=[], options=None): - self.svdsmProxy.glusterVolumeCreate(volumeName, brickList, - replicaCount, stripeCount, - transportList) - volumeList = self.svdsmProxy.glusterVolumeInfo() - return {'uuid': volumeList[volumeName]['uuid']} + return self.svdsmProxy.glusterVolumeCreate(volumeName, brickList, + replicaCount, stripeCount, + transportList) @exportAsVerb def volumeStart(self, volumeName, force=False, options=None): diff --git a/vdsm/gluster/cli.py b/vdsm/gluster/cli.py index e31c3f4..2ead7c9 100644 --- a/vdsm/gluster/cli.py +++ b/vdsm/gluster/cli.py @@ -18,7 +18,6 @@ # Refer to the README and COPYING files for full details of the license # -import re import xml.etree.cElementTree as etree from functools import wraps @@ -30,7 +29,6 @@ _glusterCommandPath = utils.CommandPath("gluster", "/usr/sbin/gluster", ) -_brickCountRegEx = re.compile('(\d+) x (\d+) = (\d+)') def _getGlusterVolCmd(): @@ -74,6 +72,16 @@ RUNNING = 'RUNNING' FAILED = 'FAILED' COMPLETED = 'COMPLETED' + + +class VolumeStatus: + ONLINE = 'ONLINE' + OFFLINE = 'OFFLINE' + + +class TransportType: + TCP = 'TCP' + RDMA = 'RDMA' def _execGluster(cmd): @@ -354,76 +362,47 @@ raise ge.GlusterXmlErrorException(err=[etree.tostring(xmltree)]) -def _parseVolumeInfo(out): - if not out[0].strip(): - del out[0] - if out[-1].strip(): - out += [""] - - if out[0].strip().upper() == "NO VOLUMES PRESENT": - return {} - - volumeInfoDict = {} - volumeInfo = {} - volumeName = None - brickList = [] - volumeOptions = {} - for line in out: - line = line.strip() - if not line: - if volumeName and volumeInfo: - volumeInfo["bricks"] = brickList - volumeInfo["options"] = volumeOptions - volumeInfoDict[volumeName] = volumeInfo - volumeInfo = {} - volumeName = None - brickList = [] - volumeOptions = {} - continue - - tokens = line.split(":", 1) - key = tokens[0].strip().upper() - if key == "BRICKS": - continue - elif key == "OPTIONS RECONFIGURED": - continue - elif key == "VOLUME NAME": - volumeName = tokens[1].strip() - volumeInfo["volumeName"] = volumeName - elif key == "VOLUME ID": - volumeInfo["uuid"] = tokens[1].strip() - elif key == "TYPE": - volumeInfo["volumeType"] = tokens[1].strip().upper() - elif key == "STATUS": - volumeInfo["volumeStatus"] = tokens[1].strip().upper() - elif key == "TRANSPORT-TYPE": - volumeInfo["transportType"] = tokens[1].strip().upper().split(',') - elif key.startswith("BRICK"): - brickList.append(tokens[1].strip()) - elif key == "NUMBER OF BRICKS": - volumeInfo["brickCount"] = tokens[1].strip() +def _parseVolumeInfo(tree): + """ + {VOLUMENAME: {'brickCount': BRICKCOUNT, + 'bricks': [BRICK1, BRICK2, ...], + 'options': {OPTION: VALUE, ...}, + 'transportType': [TCP,RDMA, ...], + 'uuid': UUID, + 'volumeName': NAME, + 'volumeStatus': STATUS, + 'volumeType': TYPE}, ...} + """ + volumes = {} + for el in tree.findall('volInfo/volumes/volume'): + value = {} + value['volumeName'] = el.find('name').text + value['uuid'] = el.find('id').text + value['volumeType'] = el.find('type').text + status = el.find('status').text + if status == '1': + value["volumeStatus"] = VolumeStatus.ONLINE else: - volumeOptions[tokens[0].strip()] = tokens[1].strip() - - for volumeName, volumeInfo in volumeInfoDict.iteritems(): - if volumeInfo["volumeType"] == "REPLICATE": - volumeInfo["replicaCount"] = volumeInfo["brickCount"] - elif volumeInfo["volumeType"] == "STRIPE": - volumeInfo["stripeCount"] = volumeInfo["brickCount"] - elif volumeInfo["volumeType"] == "DISTRIBUTED-REPLICATE": - m = _brickCountRegEx.match(volumeInfo["brickCount"]) - if m: - volumeInfo["replicaCount"] = m.groups()[1] - else: - volumeInfo["replicaCount"] = "" - - elif volumeInfo["volumeType"] == "DISTRIBUTED-STRIPE": - m = _brickCountRegEx.match(volumeInfo["brickCount"]) - if m: - volumeInfo["stripeCount"] = m.groups()[1] - else: - volumeInfo["stripeCount"] = "" - return volumeInfoDict + value["volumeStatus"] = VolumeStatus.OFFLINE + value['brickCount'] = el.find('brickCount').text + value['distCount'] = el.find('distCount').text + value['stripeCount'] = el.find('stripeCount').text + value['replicaCount'] = el.find('replicaCount').text + transportType = el.find('transport').text + if transportType == '0': + value['transportType'] = [TransportType.TCP] + elif transportType == '1': + value['transportType'] = [TransportType.RDMA] + else: + value['transportType'] = [TransportType.TCP, TransportType.RDMA] + value['bricks'] = [] + value['options'] = {} + for b in el.findall('bricks/brick'): + value['bricks'].append(b.text) + for o in el.findall('options/option'): + value['options'][o.find('name').text] = o.find('value').text + volumes[value['volumeName']] = value + return volumes @exportToSuperVdsm @@ -442,11 +421,14 @@ command = _getGlusterVolCmd() + ["info"] if volumeName: command.append(volumeName) - rc, out, err = _execGluster(command) - if rc: - raise ge.GlusterVolumesListFailedException(rc, out, err) - else: - return _parseVolumeInfo(out) + try: + xmltree = _execGlusterXml(command) + except ge.GlusterCmdFailedException, e: + raise ge.GlusterVolumesListFailedException(rc=e.rc, err=e.err) + try: + return _parseVolumeInfo(xmltree) + except (etree.ParseError, AttributeError, ValueError): + raise ge.GlusterXmlErrorException(err=[etree.tostring(xmltree)]) @exportToSuperVdsm @@ -460,12 +442,14 @@ if transportList: command += ["transport", ','.join(transportList)] command += brickList - - rc, out, err = _execGluster(command) - if rc: - raise ge.GlusterVolumeCreateFailedException(rc, out, err) - else: - return True + try: + xmltree = _execGlusterXml(command) + except ge.GlusterCmdFailedException, e: + raise ge.GlusterVolumeCreateFailedException(rc=e.rc, err=e.err) + try: + return {'uuid': xmltree.find('id').text} + except (etree.ParseError, AttributeError, ValueError): + raise ge.GlusterXmlErrorException(err=[etree.tostring(xmltree)]) @exportToSuperVdsm @@ -473,11 +457,11 @@ command = _getGlusterVolCmd() + ["start", volumeName] if force: command.append('force') - rc, out, err = _execGluster(command) - if rc: - raise ge.GlusterVolumeStartFailedException(rc, out, err) - else: + try: + _execGlusterXml(command) return True + except ge.GlusterCmdFailedException, e: + raise ge.GlusterVolumeStartFailedException(rc=e.rc, err=e.err) @exportToSuperVdsm @@ -485,30 +469,31 @@ command = _getGlusterVolCmd() + ["stop", volumeName] if force: command.append('force') - rc, out, err = _execGluster(command) - if rc: - raise ge.GlusterVolumeStopFailedException(rc, out, err) - else: + try: + _execGlusterXml(command) return True + except ge.GlusterCmdFailedException, e: + raise ge.GlusterVolumeStopFailedException(rc=e.rc, err=e.err) @exportToSuperVdsm def volumeDelete(volumeName): - rc, out, err = _execGluster(_getGlusterVolCmd() + ["delete", volumeName]) - if rc: - raise ge.GlusterVolumeDeleteFailedException(rc, out, err) - else: + command = _getGlusterVolCmd() + ["delete", volumeName] + try: + _execGlusterXml(command) return True + except ge.GlusterCmdFailedException, e: + raise ge.GlusterVolumeDeleteFailedException(rc=e.rc, err=e.err) @exportToSuperVdsm def volumeSet(volumeName, option, value): - rc, out, err = _execGluster(_getGlusterVolCmd() + ["set", volumeName, - option, value]) - if rc: - raise ge.GlusterVolumeSetFailedException(rc, out, err) - else: + command = _getGlusterVolCmd() + ["set", volumeName, option, value] + try: + _execGlusterXml(command) return True + except ge.GlusterCmdFailedException, e: + raise ge.GlusterVolumeSetFailedException(rc=e.rc, err=e.err) def _parseVolumeSetHelpXml(out): @@ -538,11 +523,11 @@ command.append(option) if force: command.append('force') - rc, out, err = _execGluster(command) - if rc: - raise ge.GlusterVolumeResetFailedException(rc, out, err) - else: + try: + _execGlusterXml(command) return True + except ge.GlusterCmdFailedException, e: + raise ge.GlusterVolumeResetFailedException(rc=e.rc, err=e.err) @exportToSuperVdsm @@ -554,12 +539,11 @@ if replicaCount: command += ["replica", "%s" % replicaCount] command += brickList - - rc, out, err = _execGluster(command) - if rc: - raise ge.GlusterVolumeBrickAddFailedException(rc, out, err) - else: + try: + _execGlusterXml(command) return True + except ge.GlusterCmdFailedException, e: + raise ge.GlusterVolumeBrickAddFailedException(rc=e.rc, err=e.err) @exportToSuperVdsm @@ -820,11 +804,12 @@ @exportToSuperVdsm def peerProbe(hostName): - rc, out, err = _execGluster(_getGlusterPeerCmd() + ["probe", hostName]) - if rc: - raise ge.GlusterHostAddFailedException(rc, out, err) - else: + command = _getGlusterPeerCmd() + ["probe", hostName] + try: + _execGlusterXml(command) return True + except ge.GlusterCmdFailedException, e: + raise ge.GlusterHostAddFailedException(rc=e.rc, err=e.err) @exportToSuperVdsm @@ -832,11 +817,11 @@ command = _getGlusterPeerCmd() + ["detach", hostName] if force: command.append('force') - rc, out, err = _execGluster(command) - if rc: - raise ge.GlusterHostRemoveFailedException(rc, out, err) - else: + try: + _execGlusterXml(command) return True + except ge.GlusterCmdFailedException, e: + raise ge.GlusterHostRemoveFailedException(rc=e.rc, err=e.err) def _parsePeerStatus(tree, gHostName, gUuid, gStatus): -- To view, visit http://gerrit.ovirt.org/7951 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I364930675dcc433dfc0432a343fc66e91721e801 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Bala.FA <barumuga(a)redhat.com> Gerrit-Reviewer: Ayal Baron <abaron(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Federico Simoncelli <fsimonce(a)redhat.com> Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com> Gerrit-Reviewer: Timothy Asir <tjeyasin(a)redhat.com>

4 15

Change in vdsm[master]: lvm: make len(LV_ATTR_BITS) == number of lv attr bits
by Dan Kenigsberg 19 Nov '12

19 Nov '12

Dan Kenigsberg has uploaded a new change for review. Change subject: lvm: make len(LV_ATTR_BITS) == number of lv attr bits ...................................................................... lvm: make len(LV_ATTR_BITS) == number of lv attr bits Change-Id: Ic3605a116783663c001b0a10e616c0f5ea939b1b Bug-Url: https://bugzilla.redhat.com/876958 Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M vdsm/storage/lvm.py 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/17/9317/1 diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py index 173d0e8..65badf7 100644 --- a/vdsm/storage/lvm.py +++ b/vdsm/storage/lvm.py @@ -56,8 +56,8 @@ VG_ATTR_BITS = ("permission", "resizeable", "exported", "partial", "allocation", "clustered") -LV_ATTR_BITS = ("voltype, permission, allocations, fixedminor, state, " - "devopen, target, zero") +LV_ATTR_BITS = ("voltype", "permission", "allocations", "fixedminor", "state", + "devopen", "target", "zero") PV = namedtuple("PV", PV_FIELDS + ",guid") VG = namedtuple("VG", VG_FIELDS + ",writeable,partial") -- To view, visit http://gerrit.ovirt.org/9317 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ic3605a116783663c001b0a10e616c0f5ea939b1b Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

4 8

Change in vdsm[master]: drop type() call
by Dan Kenigsberg 19 Nov '12

19 Nov '12

Dan Kenigsberg has uploaded a new change for review. Change subject: drop type() call ...................................................................... drop type() call Python method resolution knows to find class methods as it is. Change-Id: Ifd37f580c392412159cc7641336074d8894bd501 Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M vdsm/storage/imageRepository/formatConverter.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/16/8816/1 diff --git a/vdsm/storage/imageRepository/formatConverter.py b/vdsm/storage/imageRepository/formatConverter.py index d118e87..422cb39 100644 --- a/vdsm/storage/imageRepository/formatConverter.py +++ b/vdsm/storage/imageRepository/formatConverter.py @@ -149,7 +149,7 @@ log.debug("Creating the volume lease for %s", volUUID) metaId = vol.getMetadataId() - type(vol).newVolumeLease(metaId, domain.sdUUID, volUUID) + vol.newVolumeLease(metaId, domain.sdUUID, volUUID) # If this volume is used as a template let's update the other # volume's permissions and share the volume lease (at the moment -- To view, visit http://gerrit.ovirt.org/8816 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ifd37f580c392412159cc7641336074d8894bd501 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

4 9

Change in vdsm[master]: trivial: Fix a typo of mom package name on Debian
by wudxw＠linux.vnet.ibm.com 19 Nov '12

19 Nov '12

Mark Wu has uploaded a new change for review. Change subject: trivial: Fix a typo of mom package name on Debian ...................................................................... trivial: Fix a typo of mom package name on Debian Change-Id: Iaf2631a0ac26fc72bdb7f2b79d7ac35b2e8ab8c5 Signed-off-by: Mark Wu <wudxw(a)linux.vnet.ibm.com> --- M vdsm/caps.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/20/9320/1 diff --git a/vdsm/caps.py b/vdsm/caps.py index 9fe360f..fdb06b1 100644 --- a/vdsm/caps.py +++ b/vdsm/caps.py @@ -353,7 +353,7 @@ elif getos() == OSName.DEBIAN and python_apt: KEY_PACKAGES = {'qemu-kvm': 'qemu-kvm', 'qemu-img': 'qemu-utils', 'vdsm': 'vdsmd', 'spice-server': 'libspice-server1', - 'libvirt': 'libvirt0', 'mom': 'nom'} + 'libvirt': 'libvirt0', 'mom': 'mom'} cache = apt.Cache() -- To view, visit http://gerrit.ovirt.org/9320 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Iaf2631a0ac26fc72bdb7f2b79d7ac35b2e8ab8c5 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Mark Wu <wudxw(a)linux.vnet.ibm.com>

3 7

Change in vdsm[master]: add VM creation test over localfs storage layout
by zhshzhou＠linux.vnet.ibm.com 19 Nov '12

19 Nov '12

Zhou Zheng Sheng has uploaded a new change for review. Change subject: add VM creation test over localfs storage layout ...................................................................... add VM creation test over localfs storage layout It can be easily extended to use nfs or iscsi storage layout Change-Id: Ie176b96d56803e46b46f50b9fbee5255ad019578 Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- M tests/functional/xmlrpcTests.py 1 file changed, 46 insertions(+), 17 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/83/8183/1 diff --git a/tests/functional/xmlrpcTests.py b/tests/functional/xmlrpcTests.py index 451a4cc..31ffcd0 100644 --- a/tests/functional/xmlrpcTests.py +++ b/tests/functional/xmlrpcTests.py @@ -152,6 +152,16 @@ def testStartSmallVM(self): self.skipNoKVM() + conf = {'display': 'vnc', + 'kvmEnable': 'true', + 'memSize': '256', + 'vmId': '77777777-ffff-3333-bbbb-222222222222', + 'vmName': 'vdsm_testSmallVM', + 'vmType': 'kvm'} + + self._runVMKernelBoot(conf) + + def _runVMKernelBoot(self, vmDef): kernelVer = os.uname()[2] kernelPath = "/boot/vmlinuz-" + kernelVer initramfsPath = "/boot/initramfs-%s.img" % kernelVer @@ -164,30 +174,24 @@ raise SkipTest("Can not locate initramfs image for release %s" % kernelVer) - VMID = '77777777-ffff-3333-bbbb-222222222222' - conf = {'display': 'vnc', - 'kernel': kernelPath, - 'initrd': initramfsPath, - # The initramfs is generated by dracut. The following - # arguments will be interpreted by init scripts created by - # dracut. - 'kernelArgs': 'rd.break=cmdline rd.shell rd.skipfsck', - 'kvmEnable': 'true', - 'memSize': '256', - 'vmId': VMID, - 'vmName': 'vdsm_testSmallVM', - 'vmType': 'kvm'} + vmDef.update({'kernel': kernelPath, + 'initrd': initramfsPath, + # The initramfs is generated by dracut. The following + # arguments will be interpreted by init scripts created + # by dracut. + 'kernelArgs': 'rd.break=cmdline rd.shell rd.skipfsck'}) + vmid = vmDef['vmId'] def assertVMAndGuestUp(): - self.assertVmUp(VMID) - self.assertGuestUp(VMID) + self.assertVmUp(vmid) + self.assertGuestUp(vmid) try: - self.assertVdsOK(self.s.create(conf)) + self.assertVdsOK(self.s.create(vmDef)) # wait 65 seconds for VM to come up until timeout self.retryAssert(assertVMAndGuestUp, 65, 1) finally: - destroyResult = self.s.destroy(VMID) + destroyResult = self.s.destroy(vmid) self.assertVdsOK(destroyResult) @@ -196,6 +200,31 @@ with _localfsStore(conf['conn']), self._vdsmStorageLayout(conf): pass + def testSimpleVMoLocalfs(self): + self.skipNoKVM() + + localfs = storageLayouts['localfs'] + drives = [] + for poolid, domains in localfs['layout'].iteritems(): + for sdid, imageList in domains.iteritems(): + for imgid in imageList: + volume = localfs['img'][imgid] + drives.append({'poolID': poolid, + 'domainID': sdid, + 'imageID': imgid, + 'volumeID': volume['volid'], + 'format': volume['format']}) + vmDef = {'display': 'vnc', + 'kvmEnable': 'true', + 'memSize': '256', + 'vmId': '88888888-eeee-ffff-aaaa-111111111111', + 'vmName': 'vdsm_testSmallVM_localfs', + 'vmType': 'kvm', + 'drives': drives} + + with _localfsStore(localfs['conn']), self._vdsmStorageLayout(localfs): + self._runVMKernelBoot(vmDef) + @rollbackManager def _vdsmStorageLayout(self, conf, rollback): connections = conf['conn'] -- To view, visit http://gerrit.ovirt.org/8183 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie176b96d56803e46b46f50b9fbee5255ad019578 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com>

4 14

Change in vdsm[master]: extract a method for booting a test VM through kernel boot
by zhshzhou＠linux.vnet.ibm.com 19 Nov '12

19 Nov '12

Zhou Zheng Sheng has uploaded a new change for review. Change subject: extract a method for booting a test VM through kernel boot ...................................................................... extract a method for booting a test VM through kernel boot Hide the kernel boot and initramfs details into a method, then just call the metthod with ordinary VM configurations Change-Id: I6f2e94651d0279b19b1ce6849fb6f0c1b530a10c Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- M tests/functional/xmlrpcTests.py 1 file changed, 23 insertions(+), 19 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/14/8414/1 diff --git a/tests/functional/xmlrpcTests.py b/tests/functional/xmlrpcTests.py index 2968061..91376ed 100644 --- a/tests/functional/xmlrpcTests.py +++ b/tests/functional/xmlrpcTests.py @@ -190,32 +190,36 @@ def testStartSmallVM(self): self.skipNoKVM() - def assertVMAndGuestUp(): - self.assertVmUp(VMID) - self.assertGuestUp(VMID) + conf = {'display': 'vnc', + 'kvmEnable': 'true', + 'memSize': '256', + 'vmId': '77777777-ffff-3333-bbbb-222222222222', + 'vmName': 'vdsm_testSmallVM', + 'vmType': 'kvm'} - VMID = '77777777-ffff-3333-bbbb-222222222222' + self._runVMKernelBoot(conf) + + def _runVMKernelBoot(self, vmDef): + vmid = vmDef['vmId'] + + def assertVMAndGuestUp(): + self.assertVmUp(vmid) + self.assertGuestUp(vmid) with kernelBootImages() as (kernelPath, initramfsPath): - conf = {'display': 'vnc', - 'kernel': kernelPath, - 'initrd': initramfsPath, - # The initramfs is generated by dracut. The following - # arguments will be interpreted by init scripts created by - # dracut. - 'kernelArgs': 'rd.break=cmdline rd.shell rd.skipfsck', - 'kvmEnable': 'true', - 'memSize': '256', - 'vmId': VMID, - 'vmName': 'vdsm_testSmallVM', - 'vmType': 'kvm'} - + vmDef.update( + {'kernel': kernelPath, + 'initrd': initramfsPath, + # The initramfs is generated by dracut. The following + # arguments will be interpreted by init scripts created + # by dracut. + 'kernelArgs': 'rd.break=cmdline rd.shell rd.skipfsck'}) try: - self.assertVdsOK(self.s.create(conf)) + self.assertVdsOK(self.s.create(vmDef)) # wait 65 seconds for VM to come up until timeout self.retryAssert(assertVMAndGuestUp, 65, 1) finally: - destroyResult = self.s.destroy(VMID) + destroyResult = self.s.destroy(vmid) self.assertVdsOK(destroyResult) -- To view, visit http://gerrit.ovirt.org/8414 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I6f2e94651d0279b19b1ce6849fb6f0c1b530a10c Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com>

7 26

Change in vdsm[master]: storage functional test with multiple storage domains and im...
by zhshzhou＠linux.vnet.ibm.com 19 Nov '12

19 Nov '12

Hello Adam Litke, Royce Lv, I'd like you to do a code review. Please visit http://gerrit.ovirt.org/8182 to review the following change. Change subject: storage functional test with multiple storage domains and images ...................................................................... storage functional test with multiple storage domains and images add xmlrpc functional test for local storage, it can be extend to nfs and iscsi backends Change-Id: I8287046046460f399f180d19e0717a91419297f8 Signed-off-by: Royce Lv <lvroyce(a)linux.vnet.ibm.com> Signed-off-by: Adam Litke <agl(a)us.ibm.com> Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- M tests/functional/xmlrpcTests.py 1 file changed, 213 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/82/8182/1 diff --git a/tests/functional/xmlrpcTests.py b/tests/functional/xmlrpcTests.py index 661b347..451a4cc 100644 --- a/tests/functional/xmlrpcTests.py +++ b/tests/functional/xmlrpcTests.py @@ -19,16 +19,58 @@ # import os +import sys import time +import pwd +import grp +import shutil +from contextlib import contextmanager from testrunner import VdsmTestCase as TestCaseBase from nose.plugins.skip import SkipTest from vdsm.config import config +from vdsm.constants import VDSM_USER, VDSM_GROUP +from storage.sd import BLANK_UUID +from storage.sd import name2class +from storage.sd import name2type as sdname2type +from storage.volume import name2type as volname2type from vdsm import vdscli if not config.getboolean('vars', 'xmlrpc_enable'): raise SkipTest("XML-RPC Bindings are disabled") + + +def rollbackManager(transaction): + @contextmanager + def wrapper(*args, **kwargs): + rollback = [] + exception = None + traceback = None + try: + yield transaction(rollback=rollback, *args, **kwargs) + except Exception, e: + # keep the original exception and traceback info + exception = e + traceback = sys.exc_info()[2] + finally: + rollback.reverse() + _playRollback(rollback, exception, traceback) + return wrapper + + +def _playRollback(rollback, exception=None, traceback=None): + for undo in rollback: + try: + undo() + except Exception, e: + # keep the earliest exception info + if not exception: + exception = e + # keep the original traceback info + traceback = sys.exc_info()[2] + if exception: + raise exception, None, traceback class XMLRPCTest(TestCaseBase): @@ -81,7 +123,10 @@ def assertVdsOK(self, vdsResult): # code == 0 means OK - self.assertEquals(vdsResult['status']['code'], 0) + self.assertEquals( + vdsResult['status']['code'], 0, + 'error code: %s, message: %s' % (vdsResult['status']['code'], + vdsResult['status']['message'])) def skipNoKVM(self): r = self.s.getVdsCapabilities() @@ -145,3 +190,170 @@ destroyResult = self.s.destroy(VMID) self.assertVdsOK(destroyResult) + + def testLocalfs(self): + conf = storageLayouts['localfs'] + with _localfsStore(conf['conn']), self._vdsmStorageLayout(conf): + pass + + @rollbackManager + def _vdsmStorageLayout(self, conf, rollback): + connections = conf['conn'] + storageDomains = conf['sd'] + storagePools = conf['sp'] + images = conf['img'] + layout = conf['layout'] + + # Connect storage backends + r = self.s.storageServer_ConnectionRefs_acquire(connections) + self.assertVdsOK(r) + undo = lambda: self.assertVdsOK( + self.s.storageServer_ConnectionRefs_release( + connections.keys())) + rollback.append(undo) + for _refid, status in r['results'].iteritems(): + self.assertEquals(status, 0) + + # Create storage domains + for sdid, domain in storageDomains.iteritems(): + r = self.s.createStorageDomain( + sdname2type(domain['type']), sdid, domain['name'], + domain['typeArgs'], name2class(domain['class']), 0) + self.assertVdsOK(r) + undo = lambda sdid=sdid: \ + self.assertVdsOK( + self.s.formatStorageDomain(sdid, True)) + rollback.append(undo) + + # Create storage pools + # For now we actually just support 1 pool + # So there must be only 1 pool definition in the configuration + # This code is written to create pools in case we support several pools + poolType = 0 # not used + for poolid, pool in storagePools.iteritems(): + r = self.s.createStoragePool( + poolType, poolid, pool['name'], pool['master_uuid'], + layout[poolid].keys(), pool['master_ver']) + self.assertVdsOK(r) + # Connect to pool + r = self.s.connectStoragePool( + poolid, pool['host'], 'scsikey', pool['master_uuid'], + pool['master_ver']) + self.assertVdsOK(r) + + # If spmstart fails, there is no good rollback because we need to + # be spm to tear down the pool + + # Become SPM (there is no undo operation required) + r = self.s.spmStart(storagePools.keys()[0], -1, -1, -1, 0) + self.assertVdsOK(r) + tid = r['uuid'] + self._waitTask(tid) + + for poolid in storagePools.keys(): + undo = lambda poolid=poolid: \ + self.assertVdsOK(self.s.destroyStoragePool( + poolid, storagePools[poolid]['host'], 'scsiKey')) + rollback.append(undo) + + # Activate storage domains except master ones + for poolid, domains in layout.iteritems(): + for sdid in domains.keys(): + if sdid != storagePools[poolid]['master_uuid']: + r = self.s.activateStorageDomain(sdid, poolid) + self.assertVdsOK(r) + undo = lambda sdid=sdid, poolid=poolid: \ + self.assertVdsOK( + self.s.detachStorageDomain( + sdid, poolid, BLANK_UUID, + storagePools[poolid]['master_ver'])) + rollback.append(undo) + + # Create images and volumes + for poolid, domains in layout.iteritems(): + for sdid, imageList in domains.iteritems(): + for imgid in imageList: + volume = images[imgid] + r = self.s.createVolume( + sdid, poolid, imgid, volume['size'], + volname2type(volume['format']), + volname2type(volume['preallocate']), + volname2type(volume['type']), volume['volid'], + volume['description']) + self.assertVdsOK(r) + tid = r['uuid'] + self._waitTask(tid) + undo = lambda sdid=sdid, poolid=poolid, imgid=imgid: \ + self._waitTask( + self.s.deleteImage( + sdid, poolid, imgid)['uuid']) + rollback.append(undo) + undo = (lambda sdid=sdid, poolid=poolid, + imgid=imgid, volume=volume: + self._waitTask( + self.s.deleteVolume( + sdid, poolid, imgid, + [volume['volid']])['uuid'])) + rollback.append(undo) + + def _waitTask(self, taskId): + def assertTaskOK(): + r = self.s.getTaskStatus(taskId) + self.assertVdsOK(r) + state = r['taskStatus']['taskState'] + self.assertEquals(state, 'finished') + + self.retryAssert(assertTaskOK, 20) + + +storageLayouts = \ + {'localfs': + {'conn': {'53acd629-47e6-42d8-ba99-cd0b12ff0e1e': + {'type': 'localfs', + 'params': {'path': '/tmp/teststorage0'}}, + '87e618fe-587c-4704-a9f8-9fd9321fd907': + {'type': 'localfs', + 'params': {'path': '/tmp/teststorage1'}}}, + 'sd': {"def32ac7-1234-1234-8a8c-1c887333fe65": + {"name": "test domain0", "type": "localfs", + "class": "Data", "typeArgs": "/tmp/teststorage0"}, + "9af9bd7f-6167-4ae8-aac6-95a5e5f36f60": + {"name": "test domain1", "type": "localfs", + "class": "Data", "typeArgs": "/tmp/teststorage1"}}, + 'sp': {"6e4d6a96-1234-1234-8905-b5eec55c1535": + {"name": "local storage pool", + "master_uuid": "def32ac7-1234-1234-8a8c-1c887333fe65", + "master_ver": 1, "host": 1}}, + 'img': {"47bd7538-c48b-4b94-ba94-def922151d48": + {"description": "Test volume0", "type": "leaf", + "volid": "11bd7538-c48b-4b94-ba94-def922151d48", + "format": "cow", "preallocate": "sparse", + "size": 20971520}, + "bace8f68-4c5a-43f2-acb4-fa8daf58c0f9": + {"description": "test volume1", "type": "leaf", + "volid": "bb3cbda6-a711-45a6-a6f2-c32661939e93", + "format": "cow", "preallocate": "sparse", + "size": 20971520}}, + 'layout': {"6e4d6a96-1234-1234-8905-b5eec55c1535": # pool + {"def32ac7-1234-1234-8a8c-1c887333fe65": # domains + ["47bd7538-c48b-4b94-ba94-def922151d48"], # images + "9af9bd7f-6167-4ae8-aac6-95a5e5f36f60": + ["bace8f68-4c5a-43f2-acb4-fa8daf58c0f9"]}}}, + 'nfs': {'conn': 'blah', 'sd': 'blah', 'sp': 'blah', 'img': 'blah', + 'layout': 'blah'}, + 'iscsi': {'conn': 'blah', 'sd': 'blah', 'sp': 'blah', 'img': 'blah', + 'layout': 'blah'}} + + +@rollbackManager +def _localfsStore(connDef, rollback): + for uuid, args in connDef.iteritems(): + path = args['params']['path'] + os.mkdir(path) + + undo = lambda path=path: shutil.rmtree(path, ignore_errors=True) + rollback.append(undo) + + uid = pwd.getpwnam(VDSM_USER)[2] + gid = grp.getgrnam(VDSM_GROUP)[2] + os.chown(path, uid, gid) -- To view, visit http://gerrit.ovirt.org/8182 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8287046046460f399f180d19e0717a91419297f8 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> Gerrit-Reviewer: Adam Litke <agl(a)us.ibm.com> Gerrit-Reviewer: Royce Lv <lvroyce(a)linux.vnet.ibm.com>

4 20

Change in vdsm[master]: tests: add a rollback manager for easy undoing
by zhshzhou＠linux.vnet.ibm.com 19 Nov '12

19 Nov '12

Zhou Zheng Sheng has uploaded a new change for review. Change subject: tests: add a rollback manager for easy undoing ...................................................................... tests: add a rollback manager for easy undoing Sometimes we need to perform a series of operations: op[0], op[1], ... op[N] These operations may allocate files, locks, connections, and op[K] may depend on op[K-1] 's result Consider these are contexts, after constructing contexts, we want to perform some computing using these contexts. Exception may be raised in the complicated construction stage of the contexts or when we're using the contexes. So if op[K] fails, we need to: undo op[K-1], undo op[K-2], ... undo op[0] These undo operations release the resources, or if all the operations succeed, at last we need to: undo op[N], undo op[N-1], ... undo op[0] Furthermore, We want to suppress the exceptions occured in "undo op[X]", and continue the rollback, so that all the resources can be freed. At last, we want to see the first exception raised so that we can fix the root cause, so we want to have the oldest exception occured in this batch of operation reraised. This patch proposes a concise framework to do this kind of rollback. It's an upgrade version of contextlib.contextmanager . Change-Id: Ibc932637dd81c3becf92de34ea647c1cea136111 Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- M tests/Makefile.am A tests/rollbackManagerTests.py M tests/testrunner.py 3 files changed, 161 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/71/8671/1 diff --git a/tests/Makefile.am b/tests/Makefile.am index 2b61dde..02f48ab 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -45,6 +45,7 @@ persistentDictTests.py \ restTests.py \ restData.py \ + rollbackManagerTests.py \ tcTests.py \ vdsClientTests.py \ remoteFileHandlerTests.py \ diff --git a/tests/rollbackManagerTests.py b/tests/rollbackManagerTests.py new file mode 100644 index 0000000..db3a5a0 --- /dev/null +++ b/tests/rollbackManagerTests.py @@ -0,0 +1,85 @@ +# +# Copyright IBM Corp. 2012 +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# +import glob +import os +import tempfile +import uuid + +from testrunner import VdsmTestCase as TestCaseBase +from testrunner import rollbackManager + + +class ContextError(Exception): + pass + + +class ConsumerError(Exception): + pass + + +class TestRollbackManager(TestCaseBase): + def setUp(self): + self.tmpdirPrefix = 'testrollback' + str(uuid.uuid4()) + + @rollbackManager + def tempfiles(self, fileCount, excClass, rollback): + dirPath = tempfile.mkdtemp(prefix=self.tmpdirPrefix) + undo = lambda: os.rmdir(dirPath) + rollback.append(undo) + + for i in range(0, fileCount): + path = os.path.join(dirPath, str(i)) + with open(path, "wb") as f: + undo = \ + lambda path=path: os.remove(path) + rollback.append(undo) + f.write(str(i)) + + if excClass is not None: + raise excClass("context error") + + return dirPath + + def testExceptionInContext(self): + def exceptionInContext(): + with self.tempfiles(10, ContextError): + pass + + self.assertRaises(ContextError, exceptionInContext) + # Directory and files should be removed + self.assertEquals(glob.glob(self.tmpdirPrefix + "*"), []) + + def testExceptionInConsumer(self): + def exceptionInConsumer(): + with self.tempfiles(10, None): + raise ConsumerError("consumer error") + + self.assertRaises(ConsumerError, exceptionInConsumer) + # Directory and files should be removed + self.assertEquals(glob.glob(self.tmpdirPrefix + "*"), []) + + def testNormalConsumer(self): + fileCount = 10 + with self.tempfiles(fileCount, None) as dirPath: + for i in range(0, fileCount): + with open(os.path.join(dirPath, str(i)), "rb") as f: + self.assertEquals(int(f.read()), i) + # Directory and files should be removed + self.assertEquals(glob.glob(self.tmpdirPrefix + "*"), []) diff --git a/tests/testrunner.py b/tests/testrunner.py index cdbc9d3..f9cc323 100644 --- a/tests/testrunner.py +++ b/tests/testrunner.py @@ -22,6 +22,7 @@ import os import unittest from functools import wraps +from contextlib import contextmanager from nose import config from nose import core @@ -239,6 +240,80 @@ return False +def rollbackManager(transaction): + ''' + A contextmanager-like manager for easy undoing. It's an upgraded + contextlib.contextmanager and can manage a variable number of contextes. + + It is used as a decorator to a function. There must exist a parameter + named "rollback" of the decorated function. Then the function can treat + the "rollback" as a list and append undo operations(lambdas, closures, ...) + to the list. The function will be put in the "with" statement, the return + value of the function will be assigned to the "as" variable. If an + exception is raised, whether it's raised in the function or in the block + under the "with" statement, the registered undo operations will be played + in reverse order. When peforming rollback, exceptions will be swalloweded + to let rollback continue, at last, the earliest exception with original + line number and stack trace infomation will be raised. + + Simple example: + _________________________________________ + @rollbackManager + def foo(a, b, rollback): + x = allocate_X(a) + rollback.append(lambda: release_X(x)) + + # Need not to catch the exception when allocating Y + # and release x in a "try..final" block. + # The rollback Manager will do that. + y = allocate_Y_using(x) + rollback.append(lambda: release_Y(y)) + + # Need not to catch the exception when computing z + # and release x, y. Let rollback Manager do it for you. + z = do_something_with(x, y) + return z + + with foo(blah1, blah2) as z: + visit(z) + _________________________________________ + When the "with" block is exited, resource y will be released first, then + resource x. If exception is raised when constructing y, then only the + allocated x will be released, and the original exception will be re-raised. + ''' + + @contextmanager + def wrapper(*args, **kwargs): + rollback = [] + exception = None + traceback = None + try: + yield transaction(rollback=rollback, *args, **kwargs) + except Exception as e: + # keep the original exception and traceback info + exception = e + traceback = sys.exc_info()[2] + finally: + rollback.reverse() + _playRollback(rollback, exception, traceback) + return wrapper + + +def _playRollback(rollback, exception=None, traceback=None): + for undo in rollback: + try: + undo() + except Exception as e: + # keep the earliest exception info + if not exception: + exception = e + # keep the original traceback info + traceback = sys.exc_info()[2] + # re-raise the earliest exception + if exception: + raise exception, None, traceback + + if __name__ == '__main__': if "--help" in sys.argv: print("testrunner options:\n" -- To view, visit http://gerrit.ovirt.org/8671 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ibc932637dd81c3becf92de34ea647c1cea136111 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com>

6 36

Change in vdsm[master]: add and use hypervisor autodetection in bootstrap.
by Alon Bar-Lev 18 Nov '12

18 Nov '12

Alon Bar-Lev has posted comments on this change. Change subject: add and use hypervisor autodetection in bootstrap. ...................................................................... Patch Set 5: I would prefer that you didn't submit this Hello, I use nested kvm and the hardware detection works just fine, why shouldn't this work with other hypervisors? plain qemu: add the following at command-line: -cpu core2duo,+vmx And even if is not, I don't think that placing a file at host before bootstrap in these debug cases is something that worth the extra complexity. Alon -- To view, visit http://gerrit.ovirt.org/7657 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I79f4ab08b838bd75af5d4c26f98923fca0d65d8e Gerrit-PatchSet: 5 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani <fromani(a)gmail.com> Gerrit-Reviewer: Alon Bar-Lev <alonbl(a)redhat.com> Gerrit-Reviewer: Antoni Segura Puimedon <asegurap(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Douglas Schilling Landgraf <dougsland(a)redhat.com> Gerrit-Reviewer: Francesco Romani <fromani(a)gmail.com> Gerrit-Reviewer: Ryan Harper <ryanh(a)us.ibm.com> Gerrit-Reviewer: Shu Ming <shuming(a)linux.vnet.ibm.com> Gerrit-Reviewer: oVirt Jenkins CI Server

1 0

Change in vdsm[master]: [wip] pool: refresh multipath on connectStoragePool
by Federico Simoncelli 18 Nov '12

18 Nov '12

Federico Simoncelli has uploaded a new change for review. Change subject: [wip] pool: refresh multipath on connectStoragePool ...................................................................... [wip] pool: refresh multipath on connectStoragePool On connectStoragePool we should rescan the iscsi connections to reactivate them in case they were previously interrupted. Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=870768 Change-Id: Ie26a7a2577b65d3fb70586a849e0245e64344e3b Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M vdsm/storage/sp.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/75/9275/1 diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index fdd146c..ffe1b92 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -639,7 +639,7 @@ self.id = hostID self.scsiKey = scsiKey # Make sure SDCache doesn't have stale data (it can be in case of FC) - sdCache.refresh() + sdCache.invalidateStorage() # Rebuild whole Pool self.__rebuild(msdUUID=msdUUID, masterVersion=masterVersion) self.__createMailboxMonitor() -- To view, visit http://gerrit.ovirt.org/9275 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie26a7a2577b65d3fb70586a849e0245e64344e3b Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

5 10

Change in vdsm[master]: mom: Add mom package version to vdsCapabilities
by agl＠us.ibm.com 18 Nov '12

18 Nov '12

Adam Litke has uploaded a new change for review. Change subject: mom: Add mom package version to vdsCapabilities ...................................................................... mom: Add mom package version to vdsCapabilities Change-Id: I764e48f6df33d6e46f89ca8b7324b0e5e8d146f6 Signed-off-by: Adam Litke <agl(a)us.ibm.com> --- M vdsm/caps.py M vdsm_api/vdsmapi-schema.json 2 files changed, 6 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/86/9286/1 diff --git a/vdsm/caps.py b/vdsm/caps.py index cfe4caf..9fe360f 100644 --- a/vdsm/caps.py +++ b/vdsm/caps.py @@ -331,7 +331,7 @@ if getos() in (OSName.RHEVH, OSName.OVIRT, OSName.FEDORA, OSName.RHEL): KEY_PACKAGES = ['qemu-kvm', 'qemu-img', - 'vdsm', 'spice-server', 'libvirt'] + 'vdsm', 'spice-server', 'libvirt', 'mom'] try: ts = rpm.TransactionSet() @@ -353,7 +353,7 @@ elif getos() == OSName.DEBIAN and python_apt: KEY_PACKAGES = {'qemu-kvm': 'qemu-kvm', 'qemu-img': 'qemu-utils', 'vdsm': 'vdsmd', 'spice-server': 'libspice-server1', - 'libvirt': 'libvirt0'} + 'libvirt': 'libvirt0', 'mom': 'nom'} cache = apt.Cache() diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index cf6ca1f..6591410 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -706,10 +706,13 @@ # # @libvirt: Low level virtualization API # +# @mom: Dynamically manage system resources on virtualization hosts +# # Since: 4.10.0 ## {'enum': 'SoftwarePackage', - 'data': ['kernel', 'qemu-kvm', 'qemu-img', 'vdsm', 'spice-server', 'libvirt']} + 'data': ['kernel', 'qemu-kvm', 'qemu-img', 'vdsm', 'spice-server', 'libvirt', + 'mom']} ## # @SoftwarePackageInfo: -- To view, visit http://gerrit.ovirt.org/9286 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I764e48f6df33d6e46f89ca8b7324b0e5e8d146f6 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <agl(a)us.ibm.com>

3 6

Change in vdsm[master]: GERRIT JENKINS TEST, DON'T REVIEW
by smizrahi＠redhat.com 16 Nov '12

16 Nov '12

Saggi Mizrahi has uploaded a new change for review. Change subject: GERRIT JENKINS TEST, DON'T REVIEW ...................................................................... GERRIT JENKINS TEST, DON'T REVIEW Change-Id: I3a0043dd14ed35dcb4c5edcf1c22425e521bee39 Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M vdsm/mkimage.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/47/9247/1 diff --git a/vdsm/mkimage.py b/vdsm/mkimage.py index 4eb46e5..0ae4fee 100644 --- a/vdsm/mkimage.py +++ b/vdsm/mkimage.py @@ -46,7 +46,7 @@ for name, content in files.iteritems(): filename = os.path.join(dirname, name) - with file(filename, 'w') as f: f.write(base64.b64decode(content)) + with file(filename, 'wb') as f: f.write(base64.b64decode(content)) def _commonCleanFs(dirname, media): if media is not None: -- To view, visit http://gerrit.ovirt.org/9247 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3a0043dd14ed35dcb4c5edcf1c22425e521bee39 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

2 3

Change in vdsm[master]: Adding support for force extend block domains
by derez＠redhat.com 16 Nov '12

16 Nov '12

Daniel Erez has uploaded a new change for review. Change subject: Adding support for force extend block domains ...................................................................... Adding support for force extend block domains Passing 'force' flag through extendStorageDomain to enable block storage domain extend enforcement (_initipvs already accepts the 'force' flag). Change-Id: I33ad4ee65d7303705d2cc8370502553125000ca0 Signed-off-by: Daniel Erez <derez(a)redhat.com> --- M vdsm/API.py M vdsm/BindingXMLRPC.py M vdsm/storage/blockSD.py M vdsm/storage/hsm.py M vdsm/storage/lvm.py M vdsm/storage/sd.py M vdsm/storage/sp.py 7 files changed, 13 insertions(+), 13 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/92/9192/1 diff --git a/vdsm/API.py b/vdsm/API.py index e3909e9..202afc8 100644 --- a/vdsm/API.py +++ b/vdsm/API.py @@ -847,9 +847,9 @@ return self._irs.detachStorageDomain(self._UUID, self._spUUID, masterSdUUID, masterVersion) - def extend(self, devlist): + def extend(self, devlist, force=False): return self._irs.extendStorageDomain(self._UUID, - self._spUUID, devlist) + self._spUUID, devlist, force) def format(self, autoDetach): return self._irs.formatStorageDomain(self._UUID, autoDetach) diff --git a/vdsm/BindingXMLRPC.py b/vdsm/BindingXMLRPC.py index b833cf3..d460910 100644 --- a/vdsm/BindingXMLRPC.py +++ b/vdsm/BindingXMLRPC.py @@ -409,9 +409,9 @@ domain = API.StorageDomain(sdUUID, spUUID) return domain.detach(None, None, force=True) - def domainExtend(self, sdUUID, spUUID, devlist, options=None): + def domainExtend(self, sdUUID, spUUID, devlist, force=False, options=None): domain = API.StorageDomain(sdUUID, spUUID) - return domain.extend(devlist) + return domain.extend(devlist, force) def domainFormat(self, sdUUID, autoDetach=False, options=None): diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index 26d4380..e042d4d 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -676,7 +676,7 @@ else: del self._metadata[key] - def extend(self, devlist): + def extend(self, devlist, force=False): mapping = self.readMetadataMapping().values() if self.getVersion() in VERS_METADATA_LV: if len(mapping) + len(devlist) > MAX_PVS: @@ -694,7 +694,7 @@ else: raise se.InvalidPhysDev(dev) - lvm.extendVG(self.sdUUID, devices) + lvm.extendVG(self.sdUUID, devices, force) self.updateMapping() newsize = self.metaSize(self.sdUUID) lvm.extendLV(self.sdUUID, sd.METADATA, newsize) diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 4d696bf..9052e68 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -620,7 +620,7 @@ pool.extendVolume(sdUUID, volumeUUID, size, isShuttingDown) @public - def extendStorageDomain(self, sdUUID, spUUID, devlist, options = None): + def extendStorageDomain(self, sdUUID, spUUID, devlist, force=False, options = None): """ Extends a VG. ? @@ -641,7 +641,7 @@ vars.task.getExclusiveLock(STORAGE, sdUUID) # We need to let the domain to extend itself pool = self.getPool(spUUID) - pool.extendSD(sdUUID, devlist) + pool.extendSD(sdUUID, devlist, force) @public def forcedDetachStorageDomain(self, sdUUID, spUUID, options = None): diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py index b3abc2c..49ded26 100644 --- a/vdsm/storage/lvm.py +++ b/vdsm/storage/lvm.py @@ -901,12 +901,12 @@ removeVG(vg.name) -def extendVG(vgName, devices): +def extendVG(vgName, devices, force=False): pvs = [_fqpvname(pdev) for pdev in _normalizeargs(devices)] _checkpvsblksize(pvs, getVGBlockSizes(vgName)) vg = _lvminfo.getVg(vgName) #Format extension PVs as all the other already in the VG - _initpvs(pvs, int(vg.vg_mda_size) / 2 ** 20) + _initpvs(pvs, int(vg.vg_mda_size) / 2 ** 20, force) cmd = ["vgextend", vgName] + pvs rc, out, err = _lvminfo.cmd(cmd) diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index 17b6ada..cbd2d30 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -742,7 +742,7 @@ def refresh(self): pass - def extend(self, devlist): + def extend(self, devlist, force=False): pass def isMaster(self): diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index c9489b8..93fbe10 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -2078,6 +2078,6 @@ repoPath = os.path.join(self.storage_repository, self.spUUID) image.Image(repoPath).validateVolumeChain(sdUUID, imgUUID) - def extendSD(self, sdUUID, devlist): - sdCache.produce(sdUUID).extend(devlist) + def extendSD(self, sdUUID, devlist, force=False): + sdCache.produce(sdUUID).extend(devlist, force) -- To view, visit http://gerrit.ovirt.org/9192 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I33ad4ee65d7303705d2cc8370502553125000ca0 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Daniel Erez <derez(a)redhat.com>

5 30

Change in vdsm[master]: schema: nit schema fix
by lvroyce＠linux.vnet.ibm.com 16 Nov '12

16 Nov '12

Royce Lv has uploaded a new change for review. Change subject: schema: nit schema fix ...................................................................... schema: nit schema fix Change-Id: Ic24083ece5facab57a6f2f71aa150a8b02e82b0f Signed-off-by: Royce Lv<lvroyce(a)linux.vnet.ibm.com> --- M vdsm_api/vdsmapi-schema.json 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/50/9150/1 diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index d210c7d..4c5997d 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -176,7 +176,7 @@ {'type': 'SetupNetworkNetAttributes', 'data': {'*vlan': 'str', '*bonding': 'str', '*nic': ['str'], '*ipaddr': 'str', '*netmask': 'str', '*gateway': 'str', '*bootproto': 'str', - '*delay': 'uint', '*onboot': 'bool', 'remove': 'bool'}} + '*delay': 'uint', '*onboot': 'bool', '*remove': 'bool'}} ## # @SetupNetworkBondAttributes: -- To view, visit http://gerrit.ovirt.org/9150 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ic24083ece5facab57a6f2f71aa150a8b02e82b0f Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Royce Lv <lvroyce(a)linux.vnet.ibm.com>

3 3

Change in vdsm[master]: mom: Redirect mom init stacktrace to debug log
by agl＠us.ibm.com 15 Nov '12

15 Nov '12

Adam Litke has uploaded a new change for review. Change subject: mom: Redirect mom init stacktrace to debug log ...................................................................... mom: Redirect mom init stacktrace to debug log When MOM initialization fails we are currently falling back to the old KsmMonitor. When this happens we print a warning which is appropriate. Since the error is handled, we should not print the stack trace at warning level. This information is useful in debug mode however. Split the stack trace printout into a separate debug logging message. Change-Id: I8e85fafadd3e9180b6c5911baa6552b380476e3c Signed-off-by: Adam Litke <agl(a)us.ibm.com> --- M vdsm/clientIF.py 1 file changed, 2 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/71/9271/1 diff --git a/vdsm/clientIF.py b/vdsm/clientIF.py index 38aa0d7..ed3a0f0 100644 --- a/vdsm/clientIF.py +++ b/vdsm/clientIF.py @@ -194,7 +194,8 @@ self.mom = MomThread(momconf) except: self.log.warn("MOM initialization failed and fall " - "back to KsmMonitor", exc_info=True) + "back to KsmMonitor") + self.log.debug("Details:", exc_info=True) self.ksmMonitor = ksm.KsmMonitorThread(self) def _syncLibvirtNetworks(self): -- To view, visit http://gerrit.ovirt.org/9271 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8e85fafadd3e9180b6c5911baa6552b380476e3c Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <agl(a)us.ibm.com>

3 10

Change in vdsm[master]: sslTests: fix parallel build in Jenkins
by zhshzhou＠linux.vnet.ibm.com 15 Nov '12

15 Nov '12

Zhou Zheng Sheng has uploaded a new change for review. Change subject: sslTests: fix parallel build in Jenkins ...................................................................... sslTests: fix parallel build in Jenkins sslTests uses a fixed port for binding socket. When it is run in parallel Jenkins builds, it will fail with port conflicts. This patch tries to find new available port if the default port is occupied. Change-Id: Ia8510eb418de1b045d362231dee24a994ca936ac Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- M tests/sslTests.py 1 file changed, 19 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/54/9254/1 diff --git a/tests/sslTests.py b/tests/sslTests.py index 1216acc..a9dc396 100644 --- a/tests/sslTests.py +++ b/tests/sslTests.py @@ -24,6 +24,7 @@ import tempfile import threading import subprocess +import errno import testrunner import SecureXMLRPCServer @@ -104,12 +105,28 @@ keyfile=self.keyfile, certfile=self.certfile, ca_certs=self.certfile) - self.server.bind(ADDRESS) + self.address = self.tryBind(ADDRESS) self.server.listen(5) # Start the server thread: self.thread = SSLServerThread(self.server) self.thread.start() + + def tryBind(self, address): + ipadd, port = address + while True: + try: + self.server.bind((ipadd, port)) + return (ipadd, port) + except socket.error as ex: + if ex.errno == errno.EADDRINUSE: + port += 1 + if port > 65535: + raise socket.error( + errno.EADDRINUSE, + "Can not find available port to bind") + else: + raise def tearDown(self): """Release the resources used by the tests. @@ -152,7 +169,7 @@ command = [ "openssl", "s_client", - "-connect", "%s:%d" % ADDRESS, + "-connect", "%s:%d" % self.address, ] if args: command += args -- To view, visit http://gerrit.ovirt.org/9254 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia8510eb418de1b045d362231dee24a994ca936ac Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com>

4 7

Change in vdsm[master]: build: Add BuildDepends for m2crypto
by agl＠us.ibm.com 15 Nov '12

15 Nov '12

Adam Litke has uploaded a new change for review. Change subject: build: Add BuildDepends for m2crypto ...................................................................... build: Add BuildDepends for m2crypto During the build, I get the following traceback: Traceback (most recent call last): File "../tests/testrunner.py", line 274, in <module> hackVdsmModule() File "../tests/testrunner.py", line 252, in hackVdsmModule sub = __import__(name, globals(), locals(), [], -1) File "/home/aglitke/rpmbuild/BUILD/vdsm-4.10.2/vdsm/SecureXMLRPCServer.py", line 39, in <module> from M2Crypto import SSL, X509 ImportError: No module named M2Crypto We need the m2crypto package at build time now as well (in order to run tests). Change-Id: I4bfe0c6ca979f868be665d48bce395c896bec1f7 Signed-off-by: Adam Litke <agl(a)us.ibm.com> --- M vdsm.spec.in 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/53/9253/1 diff --git a/vdsm.spec.in b/vdsm.spec.in index 20ad1dc..9ba254f 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -42,6 +42,7 @@ BuildRequires: libvirt-python BuildRequires: genisoimage BuildRequires: openssl +BuildRequires: m2crypto %if 0%{?rhel} BuildRequires: python-ordereddict %endif -- To view, visit http://gerrit.ovirt.org/9253 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I4bfe0c6ca979f868be665d48bce395c896bec1f7 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <agl(a)us.ibm.com>

6 9

Change in vdsm[master]: fix two command names of Image class in vdsmapi schema
by shaohef＠linux.vnet.ibm.com 14 Nov '12

14 Nov '12

ShaoHe Feng has uploaded a new change for review. Change subject: fix two command names of Image class in vdsmapi schema ...................................................................... fix two command names of Image class in vdsmapi schema these two command names do not match the their comments Change-Id: Ia5f75ea62f66771c1e3001a565d24eacf791c473 Signed-off-by: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> --- M vdsm_api/vdsmapi-schema.json 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/13/9113/1 diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index d210c7d..dc62be2 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -2930,7 +2930,7 @@ # # Since: 4.11.0 ## -{'command': {'class': 'Image', 'name': 'sync'}, +{'command': {'class': 'Image', 'name': 'cloneStructure'}, 'data': {'dstSdUUID': 'UUID'}, 'returns': 'UUID'} @@ -2948,7 +2948,7 @@ # # Since: 4.11.0 ## -{'command': {'class': 'Image', 'name': 'sync'}, +{'command': {'class': 'Image', 'name': 'syncData'}, 'data': {'dstSdUUID': 'UUID', 'syncType': 'syncType'}, 'returns': 'UUID'} -- To view, visit http://gerrit.ovirt.org/9113 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia5f75ea62f66771c1e3001a565d24eacf791c473 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com>

5 9

Change in vdsm[master]: Not remove volumeless or inexistent images.
by ewarszaw＠redhat.com 14 Nov '12

14 Nov '12

Eduardo has uploaded a new change for review. Change subject: Not remove volumeless or inexistent images. ...................................................................... Not remove volumeless or inexistent images. Change-Id: I04af8d9ece52a1973180dc77991364785516a02a Bug-Id: https://bugzilla.redhat.com/show_bug.cgi?id=876115 Signed-off-by: Eduardo <ewarszaw(a)redhat.com> --- M vdsm/storage/image.py 1 file changed, 6 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/44/9244/1 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 73274fe..0265dcc 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -36,6 +36,8 @@ import resourceFactories import resourceManager as rm import outOfProcess as oop + +log = logging.getLogger('Storage.Image') rmanager = rm.ResourceManager.getInstance() # Disk type @@ -74,7 +76,10 @@ """ allVols = dom.getAllVolumes() imgVols = sd.getVolsOfImage(allVols, imgUUID) - if postZero: + if not imgVols: + log.warning("No volumes found for image %s. %s", imgUUID, allVols) + return + elif postZero: dom.zeroImage(dom.sdUUID, imgUUID, imgVols) else: dom.deleteImage(dom.sdUUID, imgUUID, imgVols) -- To view, visit http://gerrit.ovirt.org/9244 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I04af8d9ece52a1973180dc77991364785516a02a Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>

4 5

Change in vdsm[master]: join supervdsm subprocess after terminated to avoid defunct ...
by lvroyce＠linux.vnet.ibm.com 14 Nov '12

14 Nov '12

Royce Lv has uploaded a new change for review. Change subject: join supervdsm subprocess after terminated to avoid defunct status ...................................................................... join supervdsm subprocess after terminated to avoid defunct status After validateAccess there leaves a defunct process clear it by join after termination Change-Id: I0cdcfb3b467e9226f14a4ebc2845ff2db0f19b5f Signed-off-by: Royce Lv<lvroyce(a)linux.vnet.ibm.com> --- M vdsm/supervdsmServer.py 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/03/6503/1 -- To view, visit http://gerrit.ovirt.org/6503 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0cdcfb3b467e9226f14a4ebc2845ff2db0f19b5f Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Royce Lv <lvroyce(a)linux.vnet.ibm.com>

5 5

Change in vdsm[master]: xmlrpcTests: narrow the expected exception when using retry
by zhshzhou＠linux.vnet.ibm.com 13 Nov '12

13 Nov '12

Zhou Zheng Sheng has uploaded a new change for review. Change subject: xmlrpcTests: narrow the expected exception when using retry ...................................................................... xmlrpcTests: narrow the expected exception when using retry Change the expected exception from the default Exception to AssertionError Change-Id: I0deded14d02a21bcfbca49dbdfadfe8652added3 Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- M tests/functional/xmlrpcTests.py 1 file changed, 4 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/44/8444/1 diff --git a/tests/functional/xmlrpcTests.py b/tests/functional/xmlrpcTests.py index 6ccaf66..1d44059 100644 --- a/tests/functional/xmlrpcTests.py +++ b/tests/functional/xmlrpcTests.py @@ -122,7 +122,8 @@ 'vmName': 'foo'}) self.assertVdsOK(r) try: - retry(lambda: self.assertVmUp(VMID), timeout=20) + retry(lambda: self.assertVmUp(VMID), + expectedException=AssertionError, timeout=20) finally: # FIXME: if the server dies now, we end up with a leaked VM. r = self.s.destroy(VMID) @@ -154,7 +155,8 @@ try: self.assertVdsOK(self.s.create(conf)) # wait 65 seconds for VM to come up until timeout - retry(assertVMAndGuestUp, timeout=65) + retry(assertVMAndGuestUp, expectedException=AssertionError, + timeout=65) finally: destroyResult = self.s.destroy(VMID) -- To view, visit http://gerrit.ovirt.org/8444 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0deded14d02a21bcfbca49dbdfadfe8652added3 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com>

3 11

Change in vdsm[master]: Implement SSL session cache
by juan.hernandez＠redhat.com 13 Nov '12

13 Nov '12

Juan Hernandez has uploaded a new change for review. Change subject: Implement SSL session cache ...................................................................... Implement SSL session cache We are currently implementing SSL for the XML-RPC communications with the ssl module. Unfortunately this module uses OpenSSL in such a way that a new context is created for each TCP connection. This means that the heavy part of the SSL handshake (asymmetric encryption) is performed for each connection. In addition as we use HTTP 1.0 this also means that we use one connection per request, and thus one handshake per request. This is a potential performance problem in the VDSM side and a huge performance problem in the engine side: when the number of hosts managed by the engine the use of CPU grows to a point where almost all the CPU is used for SSL handshares. This patch uses the OpenSSL.SSL module (pyOpenSSL) in a way such that SSL sessions are cached, thus greatly reducing the amount of CPU needed for handshakes, specially in the engine. Bug-Id: https://bugzilla.redhat.com/857035 Change-Id: Ic75adee4070b415b8855af1f2ea289825496fbc1 Signed-off-by: Juan Hernandez <juan.hernandez(a)redhat.com> --- M vdsm/SecureXMLRPCServer.py 1 file changed, 125 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/23/8123/1 diff --git a/vdsm/SecureXMLRPCServer.py b/vdsm/SecureXMLRPCServer.py index 9396a28..744e594 100644 --- a/vdsm/SecureXMLRPCServer.py +++ b/vdsm/SecureXMLRPCServer.py @@ -35,6 +35,129 @@ import socket import SocketServer +from OpenSSL import SSL + + +class SSLServerSocket(object): + """SSL decorator for server sockets. + + This class wraps a normal socket so that when the accept method + is called the client socket as also decorated with SSL functionality. + The rest of the methods are just delegated to the raw socket. + """ + + def __init__(self, raw, certfile=None, keyfile=None, ca_certs=None): + # Save the reference to the raw socket so that we can delegate + # calls to it later: + self.raw = raw + + # Create the OpenSSL context: + self.context = SSL.Context(SSL.SSLv3_METHOD) + self.context.set_session_id("vdsm") + + # Load the crendentials: + if certfile and keyfile: + self.context.use_certificate_file(certfile) + self.context.use_privatekey_file(keyfile) + + # Load the CA certificates used to verify certifictes presented + # by the clients: + if ca_certs: + self.context.load_verify_locations(ca_certs) + self.context.set_verify( + SSL.VERIFY_PEER | + SSL.VERIFY_FAIL_IF_NO_PEER_CERT | + SSL.VERIFY_CLIENT_ONCE, + SSLServerSocket.verify) + + @staticmethod + def verify(connection, certificate, x, y, z): + # No need for additional verifications: + return True + + def accept(self): + # Get the client socket and address calling the accept method of + # the raw socket and replace the client socket with a wrapper: + client, address = self.raw.accept() + client = SSLClientSocket(client, self.context) + return client, address + + def bind(self, address): + return self.raw.bind(address) + + def fileno(self): + return self.raw.fileno() + + def getsockname(self): + return self.raw.getsockname() + + def gettimeout(self): + return self.raw.gettimeout() + + def listen(self, backlog=5): + return self.raw.listen(backlog) + + def setsockopt(self, level, optname, value): + return self.raw.setsockopt(level, optname, value) + + +class SSLClientSocket(object): + """SSL decorator for client sockets. + + This class wraps a client socket returned by the accept + method of a server socket providing the SSL functionality. + + Note that most methods are delegated to the wrapped raw + and many are just not implemented because they are not used + in our use case. + """ + + def __init__(self, raw, context): + # Save the reference to the raw client socket so that we can + # delegate calls to it later: + self.raw = raw + self.context = context + + # Create a new SSL connection and a file like object on top of it + # in order to be able to implement the makefile method used by + # the XML RPC server: + self.connection = SSL.Connection(self.context, self.raw) + + # Configure the connection to that it will automatically do the + # hand shake before the first read or write operation: + self.connection.set_accept_state() + + def close(self): + return self.connection.close() + + def do_handshake(self): + return self.connection.do_handshake() + + def makefile(self, mode="r", bufsize=-1): + # In order to implement this method we borrow the _fileobject + # function from the socket module, and that forces us to + # implement correcty the recv and send family of methods: + return socket._fileobject(self, mode, bufsize) + + def recv(self, bufsize, flags=0): + return self.connection.recv(bufsize) + + def send(self, string, flags=0): + return self.connection.send(string) + + def sendall(self, string, flags=0): + return self.connection.sendall(string) + + def setsockopt(self, level, optname, value): + self.connection.setsockopt(level, optname, value) + + def shutdown(self, how): + # Not completly sure, but I think we should first shutdown the + # SSL connection and then the raw socket: + self.connection.shutdown() + self.raw.shutdown(how) + + SecureXMLRPCRequestHandler = SimpleXMLRPCServer.SimpleXMLRPCRequestHandler @@ -53,11 +176,9 @@ requestHandler, logRequests, allow_none, encoding, bind_and_activate=False) - self.socket = ssl.wrap_socket(self.socket, + self.socket = SSLServerSocket(self.socket, keyfile=keyfile, certfile=certfile, - ca_certs=ca_certs, server_side=True, - cert_reqs=ssl.CERT_REQUIRED, - do_handshake_on_connect=False) + ca_certs=ca_certs) if timeout is not None: self.socket.settimeout = timeout if bind_and_activate: -- To view, visit http://gerrit.ovirt.org/8123 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ic75adee4070b415b8855af1f2ea289825496fbc1 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Juan Hernandez <juan.hernandez(a)redhat.com>

6 43

Change in vdsm[master]: skip supervdsmTests until they work in-tree
by Dan Kenigsberg 13 Nov '12

13 Nov '12

Dan Kenigsberg has uploaded a new change for review. Change subject: skip supervdsmTests until they work in-tree ...................................................................... skip supervdsmTests until they work in-tree Change-Id: I76eda885a8d66ca9c973dbc1a3a1b72d9e2035cd Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M tests/Makefile.am 1 file changed, 0 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/19/9219/1 diff --git a/tests/Makefile.am b/tests/Makefile.am index 9683e7b..b81820f 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -48,7 +48,6 @@ restData.py \ restTests.py \ storageMailboxTests.py \ - superVdsmTests.py \ tcTests.py \ vdsClientTests.py \ volumeTests.py \ -- To view, visit http://gerrit.ovirt.org/9219 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I76eda885a8d66ca9c973dbc1a3a1b72d9e2035cd Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

2 4

Change in vdsm[master]: tests: sort the tests and remove duplicates
by Federico Simoncelli 13 Nov '12

13 Nov '12

Federico Simoncelli has uploaded a new change for review. Change subject: tests: sort the tests and remove duplicates ...................................................................... tests: sort the tests and remove duplicates Change-Id: I1ef4d7317b5e00512f078792aecd4a1c55eac302 Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M tests/Makefile.am 1 file changed, 1 insertion(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/18/9218/1 diff --git a/tests/Makefile.am b/tests/Makefile.am index dcb0b2e..9683e7b 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -47,13 +47,11 @@ resourceManagerTests.py \ restData.py \ restTests.py \ - superVdsmTests.py \ storageMailboxTests.py \ + superVdsmTests.py \ tcTests.py \ vdsClientTests.py \ volumeTests.py \ - remoteFileHandlerTests.py \ - resourceManagerTests.py \ zombieReaperTests.py \ $(NULL) -- To view, visit http://gerrit.ovirt.org/9218 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I1ef4d7317b5e00512f078792aecd4a1c55eac302 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

3 5

Change in vdsm[master]: tests: add FileVolumeGetVSizeTest
by Federico Simoncelli 13 Nov '12

13 Nov '12

Federico Simoncelli has uploaded a new change for review. Change subject: tests: add FileVolumeGetVSizeTest ...................................................................... tests: add FileVolumeGetVSizeTest This patch adds a test for the getVSize method for file domains. Change-Id: Iac8b28f9fa1268f83c658bde1d0302e013584826 Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M tests/Makefile.am A tests/volumeTests.py 2 files changed, 70 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/03/9203/1 diff --git a/tests/Makefile.am b/tests/Makefile.am index f53470f..1753c75 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -49,6 +49,7 @@ vdsClientTests.py \ remoteFileHandlerTests.py \ resourceManagerTests.py \ + volumeTests.py \ $(NULL) dist_noinst_DATA = \ diff --git a/tests/volumeTests.py b/tests/volumeTests.py new file mode 100644 index 0000000..5dfd8e0 --- /dev/null +++ b/tests/volumeTests.py @@ -0,0 +1,69 @@ +# Copyright 2012 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +import os +import shutil +import tempfile +import uuid + +from testrunner import VdsmTestCase as TestCaseBase + +from storage import outOfProcess +from storage.fileVolume import FileVolume + + +class FileDomainMockObject(object): + def __init__(self, repoPath, sdUUID): + self.repoPath = repoPath + self.sdUUID = sdUUID + + def _getRepoPath(self): + return self.repoPath + + @property + def oop(self): + return outOfProcess.getProcessPool(self.sdUUID) + + +class FileVolumeGetVSizeTest(TestCaseBase): + VOLSIZE = 1024 + SDBLKSZ = 512 + + def setUp(self): + self.repoPath = tempfile.mkdtemp() + + self.sdUUID = str(uuid.uuid4()) + self.imgUUID = str(uuid.uuid4()) + self.volUUID = str(uuid.uuid4()) + + imgPath = os.path.join(self.repoPath, self.sdUUID, "images", + self.imgUUID) + volPath = os.path.join(imgPath, self.volUUID) + + os.makedirs(imgPath) + open(volPath, "w").truncate(self.VOLSIZE * self.SDBLKSZ) + self.sdobj = FileDomainMockObject(self.repoPath, self.sdUUID) + + def tearDown(self): + shutil.rmtree(self.repoPath) + + def test(self): + volSize = FileVolume.getVSize(self.sdobj, self.imgUUID, + self.volUUID) + assert volSize == self.VOLSIZE -- To view, visit http://gerrit.ovirt.org/9203 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Iac8b28f9fa1268f83c658bde1d0302e013584826 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

3 3

Change in vdsm[master]: tests: drop the sanlock-python dependency
by Federico Simoncelli 13 Nov '12

13 Nov '12

Federico Simoncelli has uploaded a new change for review. Change subject: tests: drop the sanlock-python dependency ...................................................................... tests: drop the sanlock-python dependency Testing vdsm shouldn't require sanlock to be installed and running. Change-Id: I88fac9a1f538c92f696b8f097b56bf231dae9b9f Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M tests/testrunner.py M vdsm.spec.in 2 files changed, 3 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/02/9202/1 diff --git a/tests/testrunner.py b/tests/testrunner.py index 9851f3e..e068426 100644 --- a/tests/testrunner.py +++ b/tests/testrunner.py @@ -39,6 +39,9 @@ PERMUTATION_ATTR = "_permutations_" +# At the moment of this writing the tests don't need a mock object for sanlock +sys.modules.update({'sanlock': object()}) + def _getPermutation(f, args): @wraps(f) diff --git a/vdsm.spec.in b/vdsm.spec.in index a7614dc..782d91c 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -40,7 +40,6 @@ # BuildRequires needed by the tests during the build BuildRequires: python-ethtool BuildRequires: libvirt-python -BuildRequires: sanlock-python >= 2.3 BuildRequires: genisoimage # Autotools BuildRequires -- To view, visit http://gerrit.ovirt.org/9202 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I88fac9a1f538c92f696b8f097b56bf231dae9b9f Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

2 2

Change in vdsm[master]: build on el6: import OrderedDict from elsewhere
by Dan Kenigsberg 13 Nov '12

13 Nov '12

Dan Kenigsberg has uploaded a new change for review. Change subject: build on el6: import OrderedDict from elsewhere ...................................................................... build on el6: import OrderedDict from elsewhere Commit 1db772c71e has introduced usage of collections.OrderedDict which is not part of el6's Python 2.6. This patch imports Raymond Hettinger's implementation if OrderedDict is missing in the default location. Change-Id: I92079c2639d892f283c771576ca25e333d0b1936 Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M vdsm.spec.in M vdsm_api/vdsmapi.py 2 files changed, 11 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/12/9212/1 diff --git a/vdsm.spec.in b/vdsm.spec.in index d268948..88e3a1f 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -42,6 +42,9 @@ BuildRequires: libvirt-python BuildRequires: sanlock-python >= 2.3 BuildRequires: genisoimage +%if 0%{?rhel} +BuildRequires: python-ordereddict +%endif # Autotools BuildRequires %if 0%{?enable_autotools} @@ -162,6 +165,9 @@ BuildArch: noarch Requires: %{name}-python = %{version}-%{release} +%if 0%{?rhel} +Requires: python-ordereddict +%endif %description jsonrpc A Json-based RPC interface that serves as the protocol for libvdsm. diff --git a/vdsm_api/vdsmapi.py b/vdsm_api/vdsmapi.py index a91b233..c68daa1 100644 --- a/vdsm_api/vdsmapi.py +++ b/vdsm_api/vdsmapi.py @@ -24,7 +24,11 @@ # Refer to the README and COPYING files for full details of the license # -from collections import OrderedDict +try: + from collections import OrderedDict + OrderedDict # make pyflakes happy +except ImportError: + from ordereddict import OrderedDict def tokenize(data): -- To view, visit http://gerrit.ovirt.org/9212 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I92079c2639d892f283c771576ca25e333d0b1936 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

2 3

Change in vdsm[master]: tests: override the P_VDSM definition in local tests
by Federico Simoncelli 13 Nov '12

13 Nov '12

Federico Simoncelli has uploaded a new change for review. Change subject: tests: override the P_VDSM definition in local tests ...................................................................... tests: override the P_VDSM definition in local tests The P_VDSM constant is used to configure the PYTHONPATH variable for the out of process handlers and it should be overridden during the local tests (vdsm is not installed). Change-Id: I06a11ab32b0adc635fe236c6033b6b3d2257b235 Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M tests/testrunner.py 1 file changed, 2 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/17/9217/1 diff --git a/tests/testrunner.py b/tests/testrunner.py index ff09862..36d76c6 100644 --- a/tests/testrunner.py +++ b/tests/testrunner.py @@ -240,6 +240,8 @@ setattr(mod, name, sub) sys.modules['vdsm.%s' % name] = getattr(mod, name) + sys.modules['vdsm.constants'].P_VDSM = "../" + def findRemove(listR, value): """used to test if a value exist, if it is, return true and remove it.""" -- To view, visit http://gerrit.ovirt.org/9217 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I06a11ab32b0adc635fe236c6033b6b3d2257b235 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

2 2

Change in vdsm[master]: pep8: fix hsm.py
by Dan Kenigsberg 13 Nov '12

13 Nov '12

Dan Kenigsberg has uploaded a new change for review. Change subject: pep8: fix hsm.py ...................................................................... pep8: fix hsm.py Errors introduced by commit b83a0c157d5c and overlooked by my rebase of commit af99446a7cd35. Change-Id: Ib31cea93e3488ea04b3fac9ddc2cc31b4454f401 Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M vdsm/storage/hsm.py 1 file changed, 5 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/13/9213/1 diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 8b269c3..46d1605 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -1438,7 +1438,8 @@ needFake = True else: raise se.CannotDeleteSharedVolume("Cannot delete shared image %s. " - "volImgs: %s" % (imgUUID, volsByImg)) + "volImgs: %s" % (imgUUID, + volsByImg)) # zeroImage will delete zeroed volumes at the end. if misc.parseBool(postZero): @@ -1456,12 +1457,12 @@ # we should use the new resource system to synchronize the process # an eliminate all race conditions if needFake: - img = image.Image(os.path.join(self.storage_repository, spUUID)) + img = image.Image(os.path.join(self.storage_repository, + spUUID)) tName = volsByImg.iterkeys()[0] tParams = dom.produceVolume(imgUUID, tName).getVolumeParams() img.createFakeTemplate(sdUUID=sdUUID, volParams=tParams) - self._spmSchedule(spUUID, "deleteImage_%s" % imgUUID, lambda : True) - + self._spmSchedule(spUUID, "deleteImage_%s" % imgUUID, lambda: True) def validateImageMove(self, srcDom, dstDom, imgUUID): """ -- To view, visit http://gerrit.ovirt.org/9213 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib31cea93e3488ea04b3fac9ddc2cc31b4454f401 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

2 4

Change in vdsm[master]: fix in import error
by ybronhei＠redhat.com 13 Nov '12

13 Nov '12

Yaniv Bronhaim has uploaded a new change for review. Change subject: fix in import error ...................................................................... fix in import error Signed-off-by: Yaniv Bronhaim <ybronhei(a)redhat.com> Change-Id: I6bb9823316dd051bbc4af80a4f30b5b6c60c45d3 --- M vdsm/supervdsmServer.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/14/9214/1 diff --git a/vdsm/supervdsmServer.py b/vdsm/supervdsmServer.py index 71309ff..34fefdb 100755 --- a/vdsm/supervdsmServer.py +++ b/vdsm/supervdsmServer.py @@ -29,7 +29,7 @@ import signal from multiprocessing import Pipe, Process import storage.misc as misc -import utils +from vdsm import utils try: from gluster import cli as gcli -- To view, visit http://gerrit.ovirt.org/9214 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I6bb9823316dd051bbc4af80a4f30b5b6c60c45d3 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim <ybronhei(a)redhat.com>

2 3

Change in vdsm[master]: modify hsm.py for PEP8 compliance
by zhshzhou＠linux.vnet.ibm.com 13 Nov '12

13 Nov '12

Zhou Zheng Sheng has uploaded a new change for review. Change subject: modify hsm.py for PEP8 compliance ...................................................................... modify hsm.py for PEP8 compliance clean hsm.py code style problems, and add it to PEP 8 whitelist in Makefile.am Change-Id: Ia9b2f8b9e83db6f52678b4e0865668a79bcf53f5 Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- M Makefile.am M vdsm/storage/hsm.py 2 files changed, 546 insertions(+), 366 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/30/4530/1 -- To view, visit http://gerrit.ovirt.org/4530 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia9b2f8b9e83db6f52678b4e0865668a79bcf53f5 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com>

7 26

Change in vdsm[master]: BZ#836161 - Changed code obsoleted by deleteImage() rewrite.
by ewarszaw＠redhat.com 13 Nov '12

13 Nov '12

Hello Ayal Baron, I'd like you to do a code review. Please visit http://gerrit.ovirt.org/9209 to review the following change. Change subject: BZ#836161 - Changed code obsoleted by deleteImage() rewrite. ...................................................................... BZ#836161 - Changed code obsoleted by deleteImage() rewrite. Change image.move and image.multipleMove to call the new deleteImage implementation instead of image.delete Change-Id: Icee221a3e8282194971afefd21540dc759a02aab Signed-off-by: Eduardo <ewarszaw(a)redhat.com> Reviewed-on: https://gerrit.eng.lab.tlv.redhat.com/3141 Reviewed-by: Ayal Baron <abaron(a)redhat.com> Tested-by: Daniel Paikov <dpaikov(a)redhat.com> --- M vdsm/storage/image.py M vdsm/storage/sp.py 2 files changed, 30 insertions(+), 83 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/09/9209/1 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 4cd9785..73274fe 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -67,6 +67,19 @@ TEMPORARY_VOLUME_SIZE = 20480 # in sectors (10M) +def _deleteImage(dom, imgUUID, postZero): + """This ancillary function will be removed. + + Replaces Image.delete() in Image.[copy(), move(),multimove()]. + """ + allVols = dom.getAllVolumes() + imgVols = sd.getVolsOfImage(allVols, imgUUID) + if postZero: + dom.zeroImage(dom.sdUUID, imgUUID, imgVols) + else: + dom.deleteImage(dom.sdUUID, imgUUID, imgVols) + + class Image: """ Actually represents a whole virtual disk. Consist from chain of volumes. @@ -612,7 +625,7 @@ # in destination domain, if we got the overwrite command if force: self.log.info("delete image %s on domain %s before overwriting", imgUUID, destDom.sdUUID) - self.delete(destDom.sdUUID, imgUUID, postZero, force=True) + _deleteImage(destDom, imgUUID, postZero) chains = self._createTargetImage(destDom, srcSdUUID, imgUUID) self._interImagesCopy(destDom, srcSdUUID, imgUUID, chains) @@ -627,7 +640,12 @@ vars.task.clearRecoveries() # If it's 'move' operation, we should delete src image after copying if op == MOVE_OP: - self.delete(srcSdUUID, imgUUID, postZero, force=True) + try: + dom = sdCache.produce(srcSdUUID) + _deleteImage(dom, imgUUID, postZero) + except se.StorageException: + self.log.warning("Failed to remove img: %s from srcDom %s: " + "after it was copied to: %s", imgUUID, srcSdUUID, dstSdUUID) self.log.info("%s task on image %s was successfully finished", OP_TYPES[op], imgUUID) return True @@ -676,9 +694,11 @@ """ for imgUUID in imgList: try: - self.delete(sdUUID, imgUUID, postZero, force=True) - except Exception: - pass + dom = sdCache.produce(sdUUID) + _deleteImage(dom, imgUUID, postZero) + except se.StorageException: + self.log.warning("Delete image failed for image: %s in SD: %s", + imgUUID, sdUUID, exc_info=True) def multiMove(self, srcSdUUID, dstSdUUID, imgDict, vmUUID, force): """ @@ -706,9 +726,11 @@ # Remove images from source domain only after successfull copying of all images to the destination domain for (imgUUID, postZero) in imgDict.iteritems(): try: - self.delete(srcSdUUID, imgUUID, postZero, force=True) - except Exception: - pass + dom = sdCache.produce(srcSdUUID) + _deleteImage(dom, imgUUID, postZero) + except se.StorageException: + self.log.warning("Delete image failed for image %s in SD: %s", + imgUUID, dom.sdUUID, exc_info=True) def __cleanupCopy(self, srcVol, dstVol): """ diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index c9489b8..fdd146c 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -1823,73 +1823,6 @@ image.Image(repoPath).multiMove(srcDomUUID, dstDomUUID, imgDict, vmUUID, force) - def deleteImage(self, sdUUID, imgUUID, postZero, force): - """ - Deletes an Image folder with all its volumes. - - This function assumes that imgUUID is locked. - In addition to removing image, this function also does the following: - If removing a template from a backup SD which has dependent images: - - creates a fake template. - If removing the last image which depends on a fake template: - - removes the fake template as well - :param sdUUID: The UUID of the storage domain that contains the images. - :type sdUUID: UUID - :param imgUUID: The UUID of the image you want to delete. - :type imgUUID: UUID - :param postZero: bool - :param force: Should the operation be forced. - :type force: bool - """ - # TODO: This function works on domains. No relation with pools. - # Therefore move this to the relevant *sd module - repoPath = os.path.join(self.storage_repository, self.spUUID) - img = image.Image(repoPath) - dom = sdCache.produce(sdUUID) - allVols = dom.getAllVolumes() - # Filter volumes related to this image - imgsByVol = sd.getVolsOfImage(allVols, imgUUID) - if all(len(v.imgs) == 1 for k, v in imgsByVol.iteritems()): - # This is a self contained regular image, i.e. it's either an image - # which is not based on a template or a template which has no - # derived images, e.g. not derived from a template - img.delete(sdUUID=sdUUID, imgUUID=imgUUID, postZero=postZero, force=force) - else: - # This is either a template with derived images or a derived image - # so needs further scrutiny - ts = tuple((volName, vol.imgs) for volName, vol in - imgsByVol.iteritems() if len(vol.imgs) > 1) - if len(ts) != 1: - raise se.ImageValidationError("Image points to multiple" - "templates %s in %s from %s" % - (ts, imgsByVol, allVols)) - # TODO: Lock the template, reload allVols. - # template = ts[0] = [(tName, tImgs)] - tName, tImgs = ts[0] - # getAllVolumes makes the template self img the 1st one in tImgs - templateImage = tImgs[0] - numOfDependentImgs = len(tImgs) - 1 - if templateImage != imgUUID: - # Removing an image based on a template - img.delete(sdUUID=sdUUID, imgUUID=imgUUID, postZero=postZero, force=force) - if numOfDependentImgs == 1 and dom.produceVolume(templateImage, tName).isFake(): - # Remove the fake template since last consumer was removed - img.delete(sdUUID=sdUUID, imgUUID=templateImage, postZero=False, force=True) - - # Removing a template with dependencies - elif force: - img.delete(sdUUID=sdUUID, imgUUID=templateImage, postZero=postZero, - force=force) - elif not dom.isBackup(): - raise se.ImagesActionError("Can't remove template with children %s" % - allVols) - else: - # Removing a template with dependencies in backup domain - # A fake template will be created - img.delete(sdUUID=sdUUID, imgUUID=imgUUID, postZero=postZero, force=True) - tParams = dom.produceVolume(imgUUID, tName).getVolumeParams() - img.createFakeTemplate(sdUUID=sdUUID, volParams=tParams) - def mergeSnapshots(self, sdUUID, vmUUID, imgUUID, ancestor, successor, postZero): """ Merges the source volume to the destination volume. @@ -2065,14 +1998,6 @@ vol.teardown(sdUUID, volUUID) except: self.log.warning("SP %s SD %s img %s Vol %s - teardown failed") - - def preDeleteRename(self, sdUUID, imgUUID): - repoPath = os.path.join(self.storage_repository, self.spUUID) - return image.Image(repoPath).preDeleteRename(sdUUID, imgUUID) - - def validateDelete(self, sdUUID, imgUUID): - repoPath = os.path.join(self.storage_repository, self.spUUID) - image.Image(repoPath).validateDelete(sdUUID, imgUUID) def validateVolumeChain(self, sdUUID, imgUUID): repoPath = os.path.join(self.storage_repository, self.spUUID) -- To view, visit http://gerrit.ovirt.org/9209 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Icee221a3e8282194971afefd21540dc759a02aab Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com> Gerrit-Reviewer: Ayal Baron <abaron(a)redhat.com>

2 3

Change in vdsm[master]: BZ#836161 - Rewrite of deleteImage() complement.
by ewarszaw＠redhat.com 13 Nov '12

13 Nov '12

Hello Ayal Baron, Igor Lvovsky, I'd like you to do a code review. Please visit http://gerrit.ovirt.org/9208 to review the following change. Change subject: BZ#836161 - Rewrite of deleteImage() complement. ...................................................................... BZ#836161 - Rewrite of deleteImage() complement. Volume operations should be done at the SD level to avoid retrieving static data multiple times from disk. Added lvm.lvDmDev() returning the dm-X for active LVs. Use this to get active LV size without issue a lvm command. Change-Id: I6c8c9baecee1ad7aa7789026e533aa254a738097 Complements: I304ff5cd70186ffc9789cd1ac9337efa6c5ff695 Signed-off-by: Eduardo <ewarszaw(a)redhat.com> Reviewed-on: https://gerrit.eng.lab.tlv.redhat.com/3140 Reviewed-by: Ayal Baron <abaron(a)redhat.com> Tested-by: Igor Lvovsky <ilvovsky(a)redhat.com> --- M vdsm/storage/blockSD.py M vdsm/storage/fileSD.py M vdsm/storage/hsm.py 3 files changed, 63 insertions(+), 11 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/08/9208/1 diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index 26d4380..506059c 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -207,8 +207,10 @@ lvm.changelv(sdUUID, volUUIDs, (("-a", "y"), ("--deltag", imgUUID), ("--addtag", sd.REMOVED_IMAGE_PREFIX + imgUUID))) except se.StorageException as e: - log.debug("SD %s, Image %s pre zeroing ops failed", sdUUID, imgUUID, - volUUIDs) + log.error("Can't activate or change LV tags in SD %s. " + "failing Image %s pre zeroing operation for vols: %s. %s", + sdUUID, imgUUID, volUUIDs, e) + raise # Following call to changelv is separate since setting rw permission on an # LV fails if the LV is already set to the same value, hence we would not # be able to differentiate between a real failure of deltag/addtag and one @@ -915,11 +917,39 @@ if i.startswith(blockVolume.TAG_PREFIX_IMAGE) ] return images + def rmDCVolLinks(self, imgPath, volsImgs): + for vol in volsImgs: + lPath = os.path.join(imgPath, vol) + removedPaths = [] + try: + os.unlink(lPath) + except OSError as e: + self.log.warning("Can't unlink %s. %s", lPath, e) + else: + removedPaths.append(lPath) + self.log.debug("removed: %s", removedPaths) + return tuple(removedPaths) + + def rmDCImgDir(self, imgUUID, volsImgs): + imgPath = os.path.join(self.domaindir, sd.DOMAIN_IMAGES, imgUUID) + self.rmDCVolLinks(imgPath, volsImgs) + try: + os.rmdir(imgPath) + except OSError: + self.log.warning("Can't rmdir %s. %s", imgPath, exc_info = True) + else: + self.log.debug("removed image dir: %s", imgPath) + return imgPath + def deleteImage(self, sdUUID, imgUUID, volsImgs): - return deleteVolumes(sdUUID, volsImgs) + toDel = tuple(vName for vName, v in volsImgs.iteritems() + if v.imgs[0] == imgUUID) + deleteVolumes(sdUUID, toDel) + self.rmDCImgDir(imgUUID, volsImgs) def zeroImage(self, sdUUID, imgUUID, volsImgs): - return zeroImgVolumes(sdUUID, imgUUID, volsImgs) + zeroImgVolumes(sdUUID, imgUUID, volsImgs) + self.rmDCImgDir(imgUUID, volsImgs) def getAllVolumes(self): """ diff --git a/vdsm/storage/fileSD.py b/vdsm/storage/fileSD.py index 90ffecd..c4415da 100644 --- a/vdsm/storage/fileSD.py +++ b/vdsm/storage/fileSD.py @@ -496,9 +496,8 @@ remove the remnants of the removed images (they could be left sometimes (on NFS mostly) due to lazy file removal """ - removedPattern = os.path.join( - self.domaindir, sd.DOMAIN_IMAGES, - sd.REMOVED_IMAGE_PREFIX + '*') + removedPattern = os.path.join(self.domaindir, sd.DOMAIN_IMAGES, + sd.REMOVED_IMAGE_PREFIX + '*') removedImages = self.oop.glob.glob(removedPattern) self.log.debug("Removing remnants of deleted images %s" % removedImages) diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 4d696bf..e712102 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -1308,6 +1308,8 @@ def deleteImage(self, sdUUID, spUUID, imgUUID, postZero=False, force=False): """ Delete Image folder with all volumes + + force parameter is deprecated and not evaluated. """ #vars.task.setDefaultException(se.ChangeMeError("%s" % args)) self.getPool(spUUID) #Validates that the pool is connected. WHY? @@ -1317,15 +1319,31 @@ vars.task.getSharedLock(STORAGE, sdUUID) allVols = dom.getAllVolumes() volsByImg = sd.getVolsOfImage(allVols, imgUUID) + if not volsByImg: + self.log.error("Empty or not found image %s in SD %s. %s", + imgUUID, sdUUID, allVols) + raise se.ImageDoesNotExistInSD(imgUUID, sdUUID) + # on data domains, images should not be deleted if they are templates # being used by other images. - if not misc.parseBool(force) and not dom.isBackup() \ - and not all(len(v.imgs) == 1 for v in volsByImg.itervalues()): - raise se.CannotDeleteSharedVolume("Cannot delete shared image" - "%s. volImgs: %s" % (imgUUID, volsByImg)) + isTemplateWithChildren = False + for v in volsByImg.itervalues(): + if len(v.imgs) > 1 and v.imgs[0] == imgUUID: + isTemplateWithChildren = True + break + + if not isTemplateWithChildren: + needFake = False + elif dom.isBackup(): + needFake = True + else: + raise se.CannotDeleteSharedVolume("Cannot delete shared image %s. " + "volImgs: %s" % (imgUUID, volsByImg)) # zeroImage will delete zeroed volumes at the end. if misc.parseBool(postZero): + # postZero implies block domain. Backup domains are always NFS + # hence no need to create fake template if postZero is true. self._spmSchedule(spUUID, "zeroImage_%s" % imgUUID, dom.zeroImage, sdUUID, imgUUID, volsByImg.keys()) else: @@ -1337,6 +1355,11 @@ # intended to quickly fix the integration issue with rhev-m. In 2.3 # we should use the new resource system to synchronize the process # an eliminate all race conditions + if needFake: + img = image.Image(os.path.join(self.storage_repository, spUUID)) + tName = volsByImg.iterkeys()[0] + tParams = dom.produceVolume(imgUUID, tName).getVolumeParams() + img.createFakeTemplate(sdUUID=sdUUID, volParams=tParams) self._spmSchedule(spUUID, "deleteImage_%s" % imgUUID, lambda : True) -- To view, visit http://gerrit.ovirt.org/9208 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I6c8c9baecee1ad7aa7789026e533aa254a738097 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com> Gerrit-Reviewer: Ayal Baron <abaron(a)redhat.com> Gerrit-Reviewer: Igor Lvovsky <ilvovsky(a)redhat.com>

2 3

Change in vdsm[master]: remoteFileHandler: improve PYTHONPATH definition
by Federico Simoncelli 13 Nov '12

13 Nov '12

Federico Simoncelli has uploaded a new change for review. Change subject: remoteFileHandler: improve PYTHONPATH definition ...................................................................... remoteFileHandler: improve PYTHONPATH definition The recently added import for logUtils in misc.py broke the out of process (remoteFileHandler) subsystem (ImportError: No module named logUtils). Defining PYTHONPATH with relative paths is both unreliable and insecure. In this patch: * Remove an old sys.path modification in misc.py * Improve the PYTHONPATH definition in remoteFileHandler removing the relative path Change-Id: I29dd748bcc9658a98ad86313921bfd94c96b83a6 Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M vdsm/storage/misc.py M vdsm/storage/remoteFileHandler.py 2 files changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/93/9193/1 diff --git a/vdsm/storage/misc.py b/vdsm/storage/misc.py index bf3ae79..7453b7f 100644 --- a/vdsm/storage/misc.py +++ b/vdsm/storage/misc.py @@ -55,7 +55,6 @@ import fcntl import inspect -sys.path.append("../") from vdsm import constants import storage_exception as se from vdsm.betterPopen import BetterPopen diff --git a/vdsm/storage/remoteFileHandler.py b/vdsm/storage/remoteFileHandler.py index edf8e68..933dc82 100644 --- a/vdsm/storage/remoteFileHandler.py +++ b/vdsm/storage/remoteFileHandler.py @@ -220,7 +220,8 @@ try: # Some imports in vdsm assume /usr/share/vdsm is in your PYTHONPATH env = os.environ.copy() - env['PYTHONPATH'] = "../:" + env.get('PYTHONPATH', "") + env['PYTHONPATH'] = "%s:%s" % ( + constants.P_VDSM, env.get("PYTHONPATH"])) self.process = BetterPopen([constants.EXT_PYTHON, __file__, str(hisRead), str(hisWrite)], close_fds=False, env=env) -- To view, visit http://gerrit.ovirt.org/9193 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I29dd748bcc9658a98ad86313921bfd94c96b83a6 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

5 7

Change in vdsm[master]: Another missing of joinAll after using ThreadPool
by ybronhei＠redhat.com 13 Nov '12

13 Nov '12

Yaniv Bronhaim has uploaded a new change for review. Change subject: Another missing of joinAll after using ThreadPool ...................................................................... Another missing of joinAll after using ThreadPool ThreadPool initiates threads during its construct. The threads are cleaned only by calling joinAll function. This patch adds this call after using HSM_MailMonitor's threadPool. Otherwise, the threads will stay alive until vdsm process will be killed, and this was a leak. Change-Id: I207d8db5863e11e89394b9144600b625282c51b6 Signed-off-by: Yaniv Bronhaim <ybronhei(a)redhat.com> --- M vdsm/storage/storage_mailbox.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/09/9009/1 diff --git a/vdsm/storage/storage_mailbox.py b/vdsm/storage/storage_mailbox.py index c537883..f6f5337 100644 --- a/vdsm/storage/storage_mailbox.py +++ b/vdsm/storage/storage_mailbox.py @@ -446,7 +446,7 @@ self.log.info("HSM_MailboxMonitor - Incoming mail monitoring thread stopped, clearing outgoing mail") self._outgoingMail = EMPTYMAILBOX self._sendMail() # Clear outgoing mailbox - + self.tp.joinAll(waitForTasks=False) class SPM_MailMonitor: -- To view, visit http://gerrit.ovirt.org/9009 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I207d8db5863e11e89394b9144600b625282c51b6 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim <ybronhei(a)redhat.com>

2 3

Change in vdsm[master]: Adding supervdsm uts file
by ybronhei＠redhat.com 13 Nov '12

13 Nov '12

Yaniv Bronhaim has uploaded a new change for review. Change subject: Adding supervdsm uts file ...................................................................... Adding supervdsm uts file Change-Id: I08880b12db8a6ca9cc57d74d2ef2a86980a5b097 Signed-off-by: Yaniv Bronhaim <ybronhei(a)redhat.com> --- M tests/Makefile.am A tests/superVdsmTests.py 2 files changed, 46 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/00/8600/1 diff --git a/tests/Makefile.am b/tests/Makefile.am index a5e4c64..f16239c 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -42,12 +42,14 @@ parted_utils_tests.py \ permutationTests.py \ persistentDictTests.py \ - restTests.py \ + remoteFileHandlerTests.py \ + resourceManagerTests.py \ restData.py \ + restTests.py \ + superVdsmTests.py \ tcTests.py \ vdsClientTests.py \ - remoteFileHandlerTests.py \ - resourceManagerTests.py + $(NULL) dist_noinst_DATA = \ run_tests_local.sh diff --git a/tests/superVdsmTests.py b/tests/superVdsmTests.py new file mode 100644 index 0000000..ecb876f --- /dev/null +++ b/tests/superVdsmTests.py @@ -0,0 +1,41 @@ +from testrunner import VdsmTestCase as TestCaseBase +import supervdsm +import testValidation +import tempfile + + +class TestSuperVdsm(TestCaseBase): + + def setUp(self): + self._proxy = supervdsm.getProxy() + + # changing default constants for test. remember to return the origin + # value + self._originpidfile = self._proxy.pidfile + self._origintimestempfile = self._proxy.timestamp + self._originsock = self._proxy.address + + # temporary values to run temporary svdsm + self._proxy.pidfile = tempfile.mkdtemp() + self._proxy.timestamp = tempfile.mkdtemp() + self._proxy.address = tempfile.mkdtemp() + + def tearDown(self): + self._proxy.kill() # cleanning old temp files + self._proxy.pidfile = self._originpidfile + self._proxy.timestamp = self._origintimestempfile + self._proxy.address = self._originsock + + def testIsSuperUp(self): + testValidation.checkSudo(['python', "supervdsmServer.py"]) + self._proxy.ping() # this call initiate svdsm + + self.assertTrue(self._proxy.isRunning()) + + def testKillSuper(self): + testValidation.checkSudo(['python', "supervdsmServer.py"]) + self._proxy.ping() + self._proxy.kill() + self.assertFail(self._proxy.isRunning()) + self._proxy.ping() # Launching vdsm after kill + self.assertTrue(self._proxy.isRunning()) -- To view, visit http://gerrit.ovirt.org/8600 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I08880b12db8a6ca9cc57d74d2ef2a86980a5b097 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim <ybronhei(a)redhat.com>

5 24

Change in vdsm[master]: Add dependency to tuned package Change tuned profile to virt...
by nslomian＠redhat.com 13 Nov '12

13 Nov '12

Noam Slomianko has uploaded a new change for review. Change subject: Add dependency to tuned package Change tuned profile to virtual-host ...................................................................... Add dependency to tuned package Change tuned profile to virtual-host Change-Id: I5c87098ddced38a3e24405325a9c011de3a86e0e Signed-off-by: Noam Slomianko <nslomian(a)redhat.com> --- M vdsm.spec.in M vdsm/vdsmd.init.in 2 files changed, 7 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/30/8930/1 diff --git a/vdsm.spec.in b/vdsm.spec.in index 37f37a3..87109f5 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -120,6 +120,8 @@ Requires(post): /usr/sbin/saslpasswd2 Requires(post): /bin/hostname +Requires: tuned + %description The VDSM service is required by a Virtualization Manager to manage the Linux hosts. VDSM manages and monitors the host's storage, memory and diff --git a/vdsm/vdsmd.init.in b/vdsm/vdsmd.init.in index 5b38d55..e8dcfa7 100755 --- a/vdsm/vdsmd.init.in +++ b/vdsm/vdsmd.init.in @@ -43,6 +43,8 @@ LDCONF=/etc/sysconfig/libvirtd QLCONF=/etc/libvirt/qemu-sanlock.conf +TUNED_PROFILE=virtual-host + is_coredump=`$GETCONFITEM $CONF_FILE vars core_dump_enable false | tr A-Z a-z` [ $is_coredump != true ] && is_coredump=false @@ -401,6 +403,9 @@ ;; esac configure_libvirt $args + + #Switch to optimised profile + tuned-adm profile $TUNED_PROFILE } stop_libvirtd_sysv() { -- To view, visit http://gerrit.ovirt.org/8930 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I5c87098ddced38a3e24405325a9c011de3a86e0e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Noam Slomianko <nslomian(a)redhat.com>

3 4

Change in vdsm[master]: remoteFileHandler: set PYTHONPATH properly for new process
by zhshzhou＠linux.vnet.ibm.com 12 Nov '12

12 Nov '12

Zhou Zheng Sheng has uploaded a new change for review. Change subject: remoteFileHandler: set PYTHONPATH properly for new process ...................................................................... remoteFileHandler: set PYTHONPATH properly for new process PoolHandler.__init__() starts new process of remoteFileHandler.py. To set the corrected PYTHONPATH for the new process, it copies existing os.environ['PYTHONPATH'] and prepends "../:" to it. It's good for unit test, but not for vdsmd service. When vdsmd.service is running as a daemon, the current working dir is "/", and PYTHONPATH is not set actually when vdsm is started. So the existing PYTHONPATH is empty and "../" refers to "/". When new process of remoteFileHandler.py is started this way, any attempt to import modules under /usr/share/vdsm will fail. remoteFileHandler imports misc, and misc imports logUtils. logUtils.py is under /usr/share/vdsm. Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=875678 Change-Id: I8d4a38d5848f74d327b1d9f4ecadc6de6a0cc091 Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- M vdsm/storage/remoteFileHandler.py 1 file changed, 7 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/83/9183/1 diff --git a/vdsm/storage/remoteFileHandler.py b/vdsm/storage/remoteFileHandler.py index 34de3ef..a2a9c41 100644 --- a/vdsm/storage/remoteFileHandler.py +++ b/vdsm/storage/remoteFileHandler.py @@ -227,7 +227,13 @@ try: # Some imports in vdsm assume /usr/share/vdsm is in your PYTHONPATH env = os.environ.copy() - env['PYTHONPATH'] = "../:" + env.get('PYTHONPATH', "") + env['PYTHONPATH'] = "../" + try: + if os.environ['PYTHONPATH'] != "": + env['PYTHONPATH'] += ":" + os.environ['PYTHONPATH'] + except KeyError: + pass + env['PYTHONPATH'] += ":" + constants.P_VDSM self.process = BetterPopen([constants.EXT_PYTHON, __file__, str(hisRead), str(hisWrite)], close_fds=False, env=env) -- To view, visit http://gerrit.ovirt.org/9183 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8d4a38d5848f74d327b1d9f4ecadc6de6a0cc091 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com>

1 2

Change in vdsm[master]: configure.ac: check python-pthreading module
by Douglas Schilling Landgraf 12 Nov '12

12 Nov '12

Douglas Schilling Landgraf has uploaded a new change for review. Change subject: configure.ac: check python-pthreading module ...................................................................... configure.ac: check python-pthreading module Adding AX_PYTHON_MODULE macro to check if python-pthreading module is installed. Change-Id: I71eb94f8081159013bfb145bf29025998ff5905c Signed-off-by: Douglas Schilling Landgraf <dougsland(a)redhat.com> --- M configure.ac 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/98/9198/1 diff --git a/configure.ac b/configure.ac index 17ec6ce..5d8b238 100644 --- a/configure.ac +++ b/configure.ac @@ -125,6 +125,7 @@ # Checking for python modules (sorted, please keep in order) AX_PYTHON_MODULE([ethtool], [fatal]) AX_PYTHON_MODULE([libvirt], [fatal]) +AX_PYTHON_MODULE([pthreading], [fatal]) # External programs (sorted, please keep in order) AC_PATH_PROG([BLKID_PATH], [blkid], [/sbin/blkid]) -- To view, visit http://gerrit.ovirt.org/9198 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I71eb94f8081159013bfb145bf29025998ff5905c Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Douglas Schilling Landgraf <dougsland(a)redhat.com>

2 3

Change in vdsm[master]: logging: report the node hostname when vdsm starts
by Federico Simoncelli 12 Nov '12

12 Nov '12

Federico Simoncelli has uploaded a new change for review. Change subject: logging: report the node hostname when vdsm starts ...................................................................... logging: report the node hostname when vdsm starts It is convinient to correlate a log report to a specific host. Change-Id: I11294d3accb8a596225166f5cc3895bb8965b99f Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M vdsm/vdsm 1 file changed, 4 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/01/9201/1 diff --git a/vdsm/vdsm b/vdsm/vdsm index f2db404..71c85cc 100755 --- a/vdsm/vdsm +++ b/vdsm/vdsm @@ -80,9 +80,10 @@ file(pidfile, 'w').write(str(os.getpid()) + "\n") os.chmod(pidfile, 0664) - log.info('I am the actual vdsm %s-%s', - dsaversion.software_version, - dsaversion.software_revision) + sysname, nodename, release, version, machine = os.uname() + log.info('I am the actual vdsm %s-%s %s (%s)', + dsaversion.software_version, dsaversion.software_revision, + nodename, release) serve_clients(log) except: log.error("Exception raised", exc_info=True) -- To view, visit http://gerrit.ovirt.org/9201 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I11294d3accb8a596225166f5cc3895bb8965b99f Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

3 3

Change in vdsm[master]: Handle a race where it takes some time fot the cmdline to di...
by smizrahi＠redhat.com 12 Nov '12

12 Nov '12

Saggi Mizrahi has uploaded a new change for review. Change subject: Handle a race where it takes some time fot the cmdline to disappear after killing a process ...................................................................... Handle a race where it takes some time fot the cmdline to disappear after killing a process Change-Id: I2d7a8190bb3b8e2251309b91f341a00f89618be5 Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M tests/miscTests.py 1 file changed, 4 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/04/9204/1 diff --git a/tests/miscTests.py b/tests/miscTests.py index 6d6b234..964fc74 100644 --- a/tests/miscTests.py +++ b/tests/miscTests.py @@ -24,6 +24,7 @@ import threading from testrunner import VdsmTestCase as TestCaseBase import inspect +from vdsm import utils import storage.outOfProcess as oop import storage.misc as misc @@ -86,7 +87,9 @@ sproc = misc.execCmd(args, sync=False, sudo=False) sproc.kill() try: - self.assertEquals(misc.getCmdArgs(sproc.pid), tuple()) + test = lambda: self.assertEquals(misc.getCmdArgs(sproc.pid), + tuple()) + utils.retry(AssertionError, test, tries=10, sleep=0.1) finally: sproc.wait() -- To view, visit http://gerrit.ovirt.org/9204 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2d7a8190bb3b8e2251309b91f341a00f89618be5 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

3 3

Change in vdsm[master]: Fix problem where a 0 error code will cause remoteFileHandle...
by smizrahi＠redhat.com 12 Nov '12

12 Nov '12

Saggi Mizrahi has uploaded a new change for review. Change subject: Fix problem where a 0 error code will cause remoteFileHandler to become a zombie ...................................................................... Fix problem where a 0 error code will cause remoteFileHandler to become a zombie Change-Id: I738d9739673d11c15b882743d412b9b267013208 Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M vdsm/storage/remoteFileHandler.py 1 file changed, 6 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/07/8907/1 diff --git a/vdsm/storage/remoteFileHandler.py b/vdsm/storage/remoteFileHandler.py index 43ee951..446087b 100644 --- a/vdsm/storage/remoteFileHandler.py +++ b/vdsm/storage/remoteFileHandler.py @@ -240,10 +240,12 @@ except: pass - # For some reason Thread might have been released if python is going - # down. This makes sure that there are no issues when this happens - if self.process.poll() and Thread: - Thread(target=self.process.wait).start() + if not self.process.wait(0): + # For some reason Thread might have been released if python is + # going down. This makes sure that there are no issues when this + # happens + if Thread: + Thread(target=self.process.wait).start() def __del__(self): self.stop() -- To view, visit http://gerrit.ovirt.org/8907 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I738d9739673d11c15b882743d412b9b267013208 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

7 14

Change in vdsm[master]: Integrate zombie reaper in test framework
by smizrahi＠redhat.com 12 Nov '12

12 Nov '12

Saggi Mizrahi has uploaded a new change for review. Change subject: Integrate zombie reaper in test framework ...................................................................... Integrate zombie reaper in test framework Some parts of VDSM count on the zombie reaper being set up. Change-Id: I74dec57129b65492d4bf41ce8fc2785d896f630e Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M tests/testrunner.py M tests/zombieReaperTests.py 2 files changed, 3 insertions(+), 6 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/38/8938/1 diff --git a/tests/testrunner.py b/tests/testrunner.py index cdbc9d3..34d09f6 100644 --- a/tests/testrunner.py +++ b/tests/testrunner.py @@ -29,6 +29,9 @@ from testValidation import SlowTestsPlugin +import zombieReaper +zombieReaper.registerSignalHandler() + PERMUTATION_ATTR = "_permutations_" diff --git a/tests/zombieReaperTests.py b/tests/zombieReaperTests.py index 6406ce3..42f3431 100644 --- a/tests/zombieReaperTests.py +++ b/tests/zombieReaperTests.py @@ -26,12 +26,6 @@ class zombieReaperTests(TestCaseBase): - def setUp(self): - zombieReaper.registerSignalHandler() - - def tearDown(self): - zombieReaper.unregisterSignalHandler() - def test(self): p = BetterPopen(["sleep", "1"]) zombieReaper.autoReapPID(p.pid) -- To view, visit http://gerrit.ovirt.org/8938 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I74dec57129b65492d4bf41ce8fc2785d896f630e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

6 13

Change in vdsm[master]: Implement a zombie reaper
by smizrahi＠redhat.com 12 Nov '12

12 Nov '12

Saggi Mizrahi has uploaded a new change for review. Change subject: Implement a zombie reaper ...................................................................... Implement a zombie reaper This is meant to solve the issues where one starts a process but doesn't care about the return value. By using zombieReaper.autoReapPID(pid) you can register the reaper to automatically reap your process when it dies. Change-Id: I0fdf96e195135284b2733b21f611918ffbf69791 Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M tests/Makefile.am A tests/zombieReaperTests.py M vdsm.spec.in M vdsm/Makefile.am M vdsm/vdsm A vdsm/zombieReaper.py 6 files changed, 81 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/37/8937/1 diff --git a/tests/Makefile.am b/tests/Makefile.am index 0e3763b..ac9804a 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -47,7 +47,9 @@ tcTests.py \ vdsClientTests.py \ remoteFileHandlerTests.py \ - resourceManagerTests.py + resourceManagerTests.py \ + zombieReaperTests.py \ + $(NULL) dist_noinst_DATA = \ run_tests_local.sh diff --git a/tests/zombieReaperTests.py b/tests/zombieReaperTests.py new file mode 100644 index 0000000..6406ce3 --- /dev/null +++ b/tests/zombieReaperTests.py @@ -0,0 +1,42 @@ +# +# Copyright 2012 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# +from time import sleep +import os + +from testrunner import VdsmTestCase as TestCaseBase +import zombieReaper +from vdsm.betterPopen import BetterPopen + + +class zombieReaperTests(TestCaseBase): + def setUp(self): + zombieReaper.registerSignalHandler() + + def tearDown(self): + zombieReaper.unregisterSignalHandler() + + def test(self): + p = BetterPopen(["sleep", "1"]) + zombieReaper.autoReapPID(p.pid) + # wait for the grim reaper to arrive + sleep(4) + + # Throws error because pid is not found or is not child + self.assertRaises(OSError, os.waitpid, p.pid, os.WNOHANG) diff --git a/vdsm.spec.in b/vdsm.spec.in index 37f37a3..649581f 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -625,6 +625,7 @@ %{_datadir}/%{vdsm_name}/vdsm-store-net-config %{_datadir}/%{vdsm_name}/vm.py* %{_datadir}/%{vdsm_name}/write-net-config +%{_datadir}/%{vdsm_name}/zombieReaper.py* %attr (755,vdsm,kvm) %{_datadir}/%{vdsm_name}/nwfilter.py* %config(noreplace) %{_sysconfdir}/%{vdsm_name}/logger.conf %config(noreplace) %{_sysconfdir}/%{vdsm_name}/mom.conf diff --git a/vdsm/Makefile.am b/vdsm/Makefile.am index 7c79036..76a81c6 100644 --- a/vdsm/Makefile.am +++ b/vdsm/Makefile.am @@ -52,6 +52,7 @@ vdsmDebugPlugin.py \ vmChannels.py \ vm.py \ + zombieReaper.py \ $(NULL) dist_vdsmpylib_PYTHON = \ diff --git a/vdsm/vdsm b/vdsm/vdsm index f2db404..00b99a0 100755 --- a/vdsm/vdsm +++ b/vdsm/vdsm @@ -20,6 +20,7 @@ from logging import config as lconfig from vdsm import constants +import zombieReaper import dsaversion if sys.version_info[0] == 2: # as long as we work with Python 2, we need to monkey-patch threading @@ -49,6 +50,7 @@ signal.signal(signal.SIGTERM, sigtermHandler) signal.signal(signal.SIGUSR1, sigusr1Handler) + zombieReaper.registerSignalHandler() from clientIF import clientIF # must import after config is read cif = clientIF.getInstance(log) diff --git a/vdsm/zombieReaper.py b/vdsm/zombieReaper.py new file mode 100644 index 0000000..74b311d --- /dev/null +++ b/vdsm/zombieReaper.py @@ -0,0 +1,32 @@ +import os +import signal + +_trackedPids = set() + + +def autoReapPID(pid): + _trackedPids.add(pid) + # SIGCHLD happend before we added the pid to the set + _tryReap(pid) + + +def _tryReap(pid): + try: + pid, rv = os.waitpid(pid, os.WNOHANG) + if pid != 0: + _trackedPids.discard(pid) + except OSError: + _trackedPids.discard(pid) + + +def _zombieReaper(signum, frame): + for pid in _trackedPids.copy(): + _tryReap(pid) + + +def registerSignalHandler(): + signal.signal(signal.SIGCHLD, _zombieReaper) + + +def unregisterSignalHandler(): + signal.signal(signal.SIGCHLD, signal.SIG_DFL) -- To view, visit http://gerrit.ovirt.org/8937 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0fdf96e195135284b2733b21f611918ffbf69791 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

7 15

Change in vdsm[master]: vdsmapi: fix a pep8 glitch
by Dan Kenigsberg 12 Nov '12

12 Nov '12

Dan Kenigsberg has uploaded a new change for review. Change subject: vdsmapi: fix a pep8 glitch ...................................................................... vdsmapi: fix a pep8 glitch I've forgot to include it into Adam's former patch. Change-Id: I05c77b507fcde6ea4072179da9ea03184635dd53 Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M vdsm_api/vdsmapi.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/95/9195/1 diff --git a/vdsm_api/vdsmapi.py b/vdsm_api/vdsmapi.py index 7a169bd..a91b233 100644 --- a/vdsm_api/vdsmapi.py +++ b/vdsm_api/vdsmapi.py @@ -197,7 +197,7 @@ return '%s *' % c_list_type(name[0]) elif is_enum(name): return name - elif name == None or len(name) == 0: + elif name is None or len(name) == 0: return 'void' elif name == name.upper(): return '%sEvent *' % camel_case(name) -- To view, visit http://gerrit.ovirt.org/9195 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I05c77b507fcde6ea4072179da9ea03184635dd53 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

1 2

Change in vdsm[master]: jsonrpc: Implement the JsonRPC server for the next-gen API
by agl＠us.ibm.com 12 Nov '12

12 Nov '12

Adam Litke has uploaded a new change for review. Change subject: jsonrpc: Implement the JsonRPC server for the next-gen API ...................................................................... jsonrpc: Implement the JsonRPC server for the next-gen API This patch implements a new Binding plugin to serve the next generation vdsm API over a JsonRPC wire protocol. The basic format of a message is: <size><json-data> <size> is an unsigned 64 bit integer in big endian format that indicates the length of <json-data> in bytes. <json-data> is either a request or a response in Javascript object notation (JSON). A request object has the following fields: id: An integer which will be repeated in the matching response methodName: The name of the API method to be called args (optional): A JSON object containing arguments to the method A response object has the following fields: id: An integer which will be the same as the matching request result: The return value of the method (defined in the API schema) error: A JSON object containing error information code: An integer error code message: Error context information Method calls are dispatched to vdsm using a MethodBridge. The DynamicBridge dispatches calls based on an API schema document and a set of schema exceptions. In this way, the API can be expanded without the need to add more code to this server infrastructure. When a request is made, the DynamicBridge attempts to resolve 'methodName' to a schema defined method. If found, the necessary arguments are collected from the request and the function call is dispatched to the internal vdsm API. Schema exceptions: Currently, vdsm does not completely conform to its own API schema. It is not possible to correct all of the discrepencies due to the need to maintain backwards compatibility with current API users. To facilitate migration to this API, the DynamicBridge implements a set of overrides to translate between the schema-defined API and what is implemented in vdsm today. There are three types of method overrides and one type of argument override: Method overrides: 1. Custom call function: If a new API does not map directly to an existing vdsm API (or multiple functions must be called to get the result), a custom 'call' function can be defined for the API. This function will be called instead of trying to find a vdsm API to call. 2. Return field name: Many vdsm functions' return values are nested in a dictionary. This override specifies the key to use when accessing the result so that it can be returned un-nested to the caller. 3. Custom result post-processing function: Some vdsm APIs return results in a non-standard format and a special function must be used to reformat the data to comply with the schema. Type override: Some data types have a different representation internally to vdsm than we have defined in the schema. A type override can be used to convert between the two formats. Currently, the only kind of translation we do is to rename fields within a given type. Change-Id: Idae0faa80ffc6a5af002a8a7151aa40dc9a6673d Signed-off-by: Adam Litke <agl(a)us.ibm.com> --- M Makefile.am M configure.ac M tests/Makefile.am A tests/apiData.py A tests/apiTests.py M tests/run_tests_local.sh.in M vdsm.spec.in M vdsm/clientIF.py M vdsm/config.py.in A vdsm_api/BindingJsonRpc.py A vdsm_api/Bridge.py A vdsm_api/Makefile.am 12 files changed, 725 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/14/8614/1 diff --git a/Makefile.am b/Makefile.am index 7548f42..b1a0418 100644 --- a/Makefile.am +++ b/Makefile.am @@ -18,7 +18,7 @@ # Refer to the README and COPYING files for full details of the license # -SUBDIRS = vdsm vdsm_cli vds_bootstrap vdsm_reg vdsm_hooks tests vdsm-tool +SUBDIRS = vdsm vdsm_cli vds_bootstrap vdsm_reg vdsm_hooks tests vdsm-tool vdsm_api include $(top_srcdir)/build-aux/Makefile.subs @@ -112,8 +112,7 @@ vds_bootstrap/miniyum.py \ $(NULL) -PEP8_BLACKLIST = \ - restData.py +PEP8_BLACKLIST = apiData.py,restData.py check-local: find . -path './.git' -prune -type f -o \ diff --git a/configure.ac b/configure.ac index 450f714..679ad04 100644 --- a/configure.ac +++ b/configure.ac @@ -79,6 +79,7 @@ AC_SUBST([vdsmpylibdir], ['${pyexecdir}/vdsm']) AC_SUBST([vdsmtooldir], ['${vdsmpylibdir}/tool']) AC_SUBST([vdsmtestsdir], ['${datarootdir}/vdsm/tests']) +AC_SUBST([vdsmapidir], ['${datarootdir}/vdsm_api']) # VDSM registration default paths AC_SUBST([vdsmregdir], ['${datarootdir}/vdsm-reg']) @@ -191,6 +192,7 @@ tests/Makefile tests/functional/Makefile vds_bootstrap/Makefile + vdsm_api/Makefile vdsm_cli/Makefile vdsm_hooks/directlun/Makefile vdsm_hooks/faqemu/Makefile diff --git a/tests/Makefile.am b/tests/Makefile.am index 2b61dde..d1fca34 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -22,6 +22,7 @@ test_modules = \ alignmentScanTests.py \ + apiTests.py \ betterPopenTests.py \ betterThreadingTests.py \ capsTests.py \ @@ -66,6 +67,7 @@ dist_vdsmtests_PYTHON = \ $(test_modules) \ + apiData.py \ monkeypatch.py \ testrunner.py \ testValidation.py diff --git a/tests/apiData.py b/tests/apiData.py new file mode 100644 index 0000000..fdb2611 --- /dev/null +++ b/tests/apiData.py @@ -0,0 +1,10 @@ +class APIData(object): + def __init__(self, obj, meth, data): + self.obj = obj + self.meth = meth + self.data = data + +testPing_apidata = [ + APIData('Global', 'ping', { + 'status': {'code': 0, 'message': 'OK'}}) +] diff --git a/tests/apiTests.py b/tests/apiTests.py new file mode 100644 index 0000000..9bfe347 --- /dev/null +++ b/tests/apiTests.py @@ -0,0 +1,228 @@ +# +# Copyright 2012 Adam Litke, IBM Corporation +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +import logging +import os +import os.path +import socket +import json +import struct +import time + +from testrunner import VdsmTestCase as TestCaseBase +from vdsm import constants +import apiData + +ip = '0.0.0.0' +port = 9824 +_fakeret = {} + +apiWhitelist = ('StorageDomain.Classes', 'StorageDomain.Types', + 'Volume.Formats', 'Volume.Types', 'Volume.Roles', 'Image.DiskTypes') + + +def getFakeAPI(): + """ + Create a Mock API module for testing. Mock API will return data from + the _fakeret global variable instead of calling into vdsm. _fakeret is + expected to have the following format: + + { + '<class1>': { + '<func1>': [ <ret1>, <ret2>, ... ], + '<func2>': [ ... ], + }, '<class2>': { + ... + } + } + """ + class FakeObj(object): + def __new__(cls, *args, **kwargs): + return object.__new__(cls) + + def default(self, *args, **kwargs): + try: + return _fakeret[self.type][self.lastFunc].pop(0) + except (KeyError, IndexError): + raise Exception("No API data avilable for %s.%s" % \ + (self.type, self.lastFunc)) + + def __getattr__(self, name): + # While we are constructing the API module, use the normal getattr + if 'API' not in sys.modules: + return object.__getattr__(name) + self.lastFunc = name + return self.default + + import sys + import imp + from new import classobj + + _API = __import__('API', globals(), locals(), {}, -1) + _newAPI = imp.new_module('API') + + for obj in ('Global', 'ConnectionRefs', 'StorageDomain', 'Image', 'Volume', + 'Task', 'StoragePool', 'VM'): + cls = classobj(obj, (FakeObj,), {'type': obj}) + setattr(_newAPI, obj, cls) + + # Apply the whitelist to our version of API + for name in apiWhitelist: + parts = name.split('.') + dstObj = _newAPI + srcObj = _API + # Walk the object hierarchy copying each component of the whitelisted + # attribute from the real API to our fake one + for obj in parts: + srcObj = getattr(srcObj, obj) + try: + dstObj = getattr(dstObj, obj) + except AttributeError: + setattr(dstObj, obj, srcObj) + + # Install our fake API into the module table for use by the whole program + sys.modules['API'] = _newAPI + + +def findSchema(): + """ + Find the API schema file whether we are running tests from the source dir + or from the tests install location + """ + scriptdir = os.path.dirname(__file__) + localpath = os.path.join(scriptdir, '../vdsm_api/vdsmapi-schema.json') + installedpath = os.path.join(constants.P_VDSM, 'vdsmapi-schema.json') + for f in localpath, installedpath: + if os.access(f, os.R_OK): + return f + raise Exception("Unable to find schema in %s or %s", + localpath, installedpath) + +def setUpModule(): + """ + Set up the environment for all tests: + 1. Override the API so we can program our own return values + 2. Start an embedded server to process our requests + """ + log = logging.getLogger('apiTests') + handler = logging.StreamHandler() + fmt_str = "%(asctime)s - %(name)s - %(levelname)s - %(message)s" + formatter = logging.Formatter(fmt_str) + handler.setFormatter(formatter) + handler.setLevel(logging.INFO) + log.addHandler(handler) + + schema = findSchema() + getFakeAPI() + import BindingJsonRpc + import Bridge + bridge = Bridge.DynamicBridge(schema) + server = BindingJsonRpc.BindingJsonRpc(bridge, ip, port) + server.start() + + +class APITest(TestCaseBase): + def expectAPI(self, obj, meth, retval): + global _fakeret + if obj not in _fakeret: + _fakeret[obj] = {} + if meth not in _fakeret[obj]: + _fakeret[obj][meth] = [] + _fakeret[obj][meth].append(retval) + + def programAPI(self, key): + key += '_apidata' + for item in getattr(apiData, key): + self.expectAPI(item.obj, item.meth, item.data) + + def clearAPI(self): + global _fakeret + _fakeret = {} + +class JsonRawTest(APITest): + _Size = struct.Struct("!Q") + + def buildMessage(self, data): + msg = json.dumps(data) + msg = msg.encode('utf-8') + msize = JsonRawTest._Size.pack(len(msg)) + resp = msize + msg + return resp + + def sendMessage(self, msg): + sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + try: + sock.connect((ip, port)) + sock.sendall(msg) + data = sock.recv(JsonRawTest._Size.size) + msgLen = JsonRawTest._Size.unpack(data)[0] + data = sock.recv(msgLen) + return json.loads(data) + finally: + sock.close() + + def testPing(self): + self.clearAPI() + self.programAPI("testPing") + msg = self.buildMessage({'id': 1, 'methodName': 'Host.ping', + 'args': {}}) + reply = self.sendMessage(msg) + self.assertEquals(0, reply['error']['code']) + self.assertEquals('Success', reply['error']['message']) + + def testNoMethod(self): + msg = self.buildMessage({'id': 1, 'methodName': 'Host.fake'}) + reply = self.sendMessage(msg) + self.assertEquals(4, reply['error']['code']) + + def testBadMethod(self): + msg = self.buildMessage({'id': 1, 'methodName': 'malformed\''}) + reply = self.sendMessage(msg) + self.assertEquals(4, reply['error']['code']) + + def testMissingSize(self): + self.assertRaises(struct.error, self.sendMessage, + "malformed message") + + def testNotJson(self): + msg = "malformed message" + msize = JsonRawTest._Size.pack(len(msg)) + msg = msize + msg + self.assertRaises(struct.error, self.sendMessage, msg) + + def testSynchronization(self): + def doPing(msg): + self.clearAPI() + self.programAPI("testPing") + return self.sendMessage(msg)['error']['code'] + + msg = self.buildMessage({'id': 1, 'methodName': 'Host.ping'}) + # Send Truncated message + self.assertRaises(struct.error, doPing, msg[:-1]) + + # Test that the server recovers + self.assertEquals(0, doPing(msg)) + + # Send too much data + self.assertEquals(0, doPing(msg + "Hello")) + + # Test that the server recovers + self.assertEquals(0, doPing(msg)) + diff --git a/tests/run_tests_local.sh.in b/tests/run_tests_local.sh.in index fc4ac37..8f668f9 100644 --- a/tests/run_tests_local.sh.in +++ b/tests/run_tests_local.sh.in @@ -1,2 +1,2 @@ #!/bin/sh -PYTHONDONTWRITEBYTECODE=1 LC_ALL=C PYTHONPATH="@builddir@/vdsm:@top_srcdir@/vdsm:@top_srcdir@/vdsm_cli:$PYTHONPATH" @PYTHON@ @top_srcdir@/tests/testrunner.py --local-modules $@ +PYTHONDONTWRITEBYTECODE=1 LC_ALL=C PYTHONPATH="@builddir@/vdsm:@top_srcdir@/vdsm:@top_srcdir@/vdsm_cli:@top_srcdir@/vdsm_api:$PYTHONPATH" @PYTHON@ @top_srcdir@/tests/testrunner.py --local-modules $@ diff --git a/vdsm.spec.in b/vdsm.spec.in index dce14a9..8e7fa6c 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -154,6 +154,15 @@ %description rest A REST interface for interacting with vdsmd. +%package jsonrpc +Summary: VDSM API Server +BuildArch: noarch + +Requires: %{name}-python = %{version}-%{release} + +%description jsonrpc +A Json-based RPC interface that serves as the protocol for libvdsm. + %package bootstrap Summary: VDSM bootstrapping package BuildArch: noarch @@ -912,6 +921,12 @@ %{_datadir}/%{vdsm_name}/rest/templates/api.xsd %{_datadir}/%{vdsm_name}/rest/templates/rsdl.xml +%files jsonrpc +%{_datadir}/%{vdsm_name}/BindingJsonRpc.py* +%{_datadir}/%{vdsm_name}/Bridge.py* +%{_datadir}/%{vdsm_name}/vdsmapi.py* +%{_datadir}/%{vdsm_name}/vdsmapi-schema.json + %files bootstrap %defattr(-, root, root, -) %doc COPYING diff --git a/vdsm/clientIF.py b/vdsm/clientIF.py index e9ca7c9..cf8468c 100644 --- a/vdsm/clientIF.py +++ b/vdsm/clientIF.py @@ -158,6 +158,15 @@ self.bindings['rest'] = BindingREST(self, self.log, ip, rest_port, templatePath) + def _loadBindingJsonRpc(self): + from BindingJsonRpc import BindingJsonRpc + from Bridge import DynamicBridge + schema = os.path.join(constants.P_VDSM, 'vdsmapi-schema.json') + ip = self._getServerIP(config.get('addresses', 'management_ip')) + port = config.getint('addresses', 'json_port') + self.bindings['json'] = BindingJsonRpc(DynamicBridge(schema), + ip, port) + def _prepareBindings(self): self.bindings = {} if config.getboolean('vars', 'xmlrpc_enable'): @@ -172,6 +181,12 @@ except ImportError: self.log.warn('Unable to load the rest server module. ' 'Please make sure it is installed.') + if config.getboolean('vars', 'jsonrpc_enable'): + try: + self._loadBindingJsonRpc() + except ImportError: + self.log.warn('Unable to load the json rpc server module. ' + 'Please make sure it is installed.') def _prepareMOM(self): try: diff --git a/vdsm/config.py.in b/vdsm/config.py.in index df85e7e..51a7ad6 100644 --- a/vdsm/config.py.in +++ b/vdsm/config.py.in @@ -130,6 +130,8 @@ ('rest_enable', 'true', 'Enable the REST server'), + ('jsonrpc_enable', 'true', 'Enable the JSON RPC server'), + ('report_host_threads_as_cores', 'false', 'Count each cpu hyperthread as an individual core'), ]), @@ -251,6 +253,10 @@ 'Port on which the vdsmd REST server listens to network ' 'clients.'), + ('json_port', '4044', + 'Port on which the vdsmd Json RPC server listens to network ' + 'clients.'), + ('management_ip', '', None), ('guests_gateway_ip', '', None), diff --git a/vdsm_api/BindingJsonRpc.py b/vdsm_api/BindingJsonRpc.py new file mode 100644 index 0000000..9ac8b5d --- /dev/null +++ b/vdsm_api/BindingJsonRpc.py @@ -0,0 +1,99 @@ +# VDSM JsonRPC Server +# Copyright (C) 2012 Adam Litke, IBM Corporation +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public +# License along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +import threading +import SocketServer +import json +import logging + +import struct + +_Size = struct.Struct("!Q") + +__bridge__ = None + +class BindingJsonRpc: + def __init__(self, bridge, ip, port): + self.bridge = bridge + self.serverPort = port + self.serverIP = ip + self.log = logging.getLogger('BindingJsonRpc') + self._createServer() + + def _createServer(self): + global __bridge__ + __bridge__ = self.bridge + ip = self.serverIP or '0.0.0.0' + self.server = JsonRpcServer((ip, self.serverPort), JsonRpcTCPHandler) + + def start(self): + def threaded_start(): + self.server.serve_forever() + t = threading.Thread(target=threaded_start, + name='JsonRpc') + t.setDaemon(True) + t.start() + + def prepareForShutdown(self): + self.server.shutdown() + + +class JsonRpcServer(SocketServer.TCPServer): + def __init__(self, addrInfo, handler): + self.allow_reuse_address = True + SocketServer.TCPServer.__init__(self, addrInfo, handler) + + +class JsonRpcTCPHandler(SocketServer.StreamRequestHandler): + """ + The RequestHandler class for our server. + + It is instantiated once per connection to the server, and must + override the handle() method to implement communication to the + client. + """ + + def handle(self): + bridge = __bridge__ + log = logging.getLogger('JsonRpcTCPHandler') + while True: + # self.request is the TCP socket connected to the client + try: + data = self.request.recv(_Size.size) + if len(data) == 0: + log.debug("Connection closed") + return + msgLen = _Size.unpack(data)[0] + msg = json.loads(self.request.recv(msgLen)) + except: + log.warn("Unexpected exception", exc_info=True) + return + log.debug('--> %s', msg) + + try: + ret = bridge._dispatch(msg['methodName'], msg.get('args', {})) + except Exception: + log.error("Dispatch error", exc_info=True) + continue + ret['id'] = msg['id'] + msg = json.dumps(ret) + msg = msg.encode('utf-8') + msize = _Size.pack(len(msg)) + resp = msize + msg + log.debug('<-- %s', msg) + + self.wfile.write(resp) + self.wfile.flush() + diff --git a/vdsm_api/Bridge.py b/vdsm_api/Bridge.py new file mode 100644 index 0000000..7402e08 --- /dev/null +++ b/vdsm_api/Bridge.py @@ -0,0 +1,332 @@ +# VDSM API Connector +# Copyright (C) 2012 Adam Litke, IBM Corporation +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public +# License along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + +from functools import partial +import vdsmapi +import logging +import types +import API + +class VdsmError(Exception): + def __init__(self, code, message): + self.code = code + self.message = message + +class MethodBridge(object): + + def _dispatch(self, name, argobj): + method = name.replace('.', '_') + result = None + error = {'code': 0, 'message': 'Success'} + try: + fn = getattr(self, method) + except AttributeError: + error = {'code': 4, + 'message': "Operation '%s' not supported" % name} + return {'result': result, 'error': error} + try: + result = fn(argobj) + except VdsmError, e: + error = {'code': e.code, 'message': e.message} + except Exception, e: + error = {'code': 5, 'message': 'Internal error: %s' % e} + return {'result': result, 'error': error} + + def getArgs(self, argobj, arglist): + ret = () + for arg in arglist: + if arg in argobj: + ret += (argobj[arg],) + return ret + + def getResult(self, response, member=None): + if member is None: + return None + try: + return response[member] + except KeyError: + raise VdsmError(5, "Response is missing '%s' member" % member) + +class DynamicBridge(MethodBridge): + def __init__(self, schema): + self.parseSchema(schema) + + + def parseSchema(self, schema): + self.commands = {} + self.classes = {} + self.types = {} + with open(schema) as f: + symbols = vdsmapi.parse_schema(f) + for s in symbols: + if 'command' in s: + key = "%s_%s" % (s['command']['class'], + s['command']['name']) + self.commands[key] = s + elif 'class' in s: + cls = s['class'] + self.classes[cls] = s + elif 'type' in s: + t = s['type'] + self.types[t] = s + + + def __getattr__(self, attr): + if attr in self.commands: + cls, name = attr.split('_') + return partial(self.dynamicMethod, cls, name) + else: + raise AttributeError + + + def getArgList(self, cmd): + return self.commands[cmd].get('data', {}).keys() + + + def getObjargList(self, cls): + return self.classes[cls]['data'].keys() + + def getApiClass(self, cls): + name_map = { 'Host': 'Global' } + try: + cls = name_map[cls] + except KeyError: + pass + return getattr(API, cls) + + def typeFixup(self, sym_name, sym_type, obj): + logging.info("typeFixup: '%s': '%s'" % (sym_name, sym_type)) + isList = False + if type(sym_type) is list: + sym_type = sym_type[0] + isList = True + if sym_name[0] == '*': + sym_name = sym_name[1:] + + try: + symbol = self.types[sym_type] + except KeyError: + return + + if isList: + itemList = obj + else: + itemList = [obj] + + for item in itemList: + if sym_type in typefixups: + logging.error("Fixing up type %s", sym_type) + typefixups[sym_type](item) + for (k, v) in symbol.get('data', {}).items(): + if k[0] == '*': + k = k[1:] + if k in item: + self.typeFixup(k, v, item[k]) + + + def fixupArgs(self, cmd, args): + argInfo = zip(self.commands[cmd].get('data', {}).items(), args) + for typeInfo, val in argInfo: + argName, argType = typeInfo + if argType not in self.types: + continue + self.typeFixup(argName, argType, val) + + def fixupRet(self, cmd, result): + retType = self.commands[cmd].get('returns', None) + if retType is not None: + self.typeFixup('return', retType, result) + return result + + def dynamicMethod(self, cls, name, argobj): + cmd = '%s_%s' % (cls, name) + ctorArgObj = argobj.get('__obj__', {}) + ctorargs = self.getArgs(ctorArgObj, self.getObjargList(cls)) + args = self.getArgs(argobj, self.getArgList(cmd)) + apiobj = self.getApiClass(cls) + api = apiobj(*ctorargs) + + self.fixupArgs(cmd, args) + + # Call the override function (if given). Otherwise, just call directly + fn = command_info.get(cmd, {}).get('call') + if fn: + result = fn(api, argobj) + else: + fn = getattr(api, name) + result = fn(*args) + + if result['status']['code']: + code = result['status']['code'] + msg = result['status']['message'] + return {'code': code, 'message': msg} + + retfield = command_info.get(cmd, {}).get('ret') + if type(retfield) == types.FunctionType: + ret = retfield(result) + else: + ret = self.getResult(result, retfield) + return self.fixupRet(cmd, ret) + + +def Host_getStorageRepoStats_Ret(ret): + """ + The returned dictionary doesn't separate the stats from the status code + so we need to rebuild the result. + """ + del ret['status'] + return ret + + +def Host_getVMList_Call(api, args): + """ + This call is only interested in returning the VM UUIDs so pass False for + the first argument in order to suppress verbose results. + """ + vmList = args.get('vmList', []) + return API.Global().getVMList(False, vmList) + +def Host_getVMList_Ret(ret): + """ + Just return a list of VM UUIDs + """ + return [v['vmId'] for v in ret['vmList']] + + +def StoragePool_getInfo_Ret(ret): + """ + The result contains two data structures which must be merged + """ + return {'info': ret['info'], 'dominfo': ret['dominfo']} + + +def VM_getInfo_Call(api, args): + """ + The VM object has no getInfo method. We use the method from 'Global' and + pass arguments to get verbose information for only this one VM. + """ + vmId = api._UUID + return API.Global().getVMList(True, [vmId]) + +def VM_getInfo_Ret(ret): + """ + The result will be a list with only one element. + """ + return ret['vmList'][0] + +def VM_migrationCreate_Ret(ret): + """ + The result contains two data structures which must be merged + """ + return {'params': ret['params'], 'migrationPort': ret['migrationPort']} + + +def Volume_getsize_Ret(ret): + """ + Merge the two sizes into a single dictionary result. + """ + return {'truesize': ret['truesize'], 'apparentsize': ret['apparentsize']} + + +## +# Possible ways to override a command: +# - Supply a custom call function if the function name doesn't map directly to +# a vdsm API. +# - Specify the name of the field in the result that is the return value +# - Specify a custom function to post-process the result into a return value +## +command_info = { + 'ConnectionRefs_acquire': {'ret': 'results'}, + 'ConnectionRefs_release': {'ret': 'results'}, + 'ConnectionRefs_statuses': {'ret': 'connectionslist'}, + 'Host_fenceNode': {'ret': 'power'}, + 'Host_getAllTasksInfo': {'ret': 'allTasksInfo'}, + 'Host_getAllTasksStatuses': {'ret': 'allTasksStatus'}, + 'Host_getCapabilities': {'ret': 'info'}, + 'Host_getConnectedStoragePools': {'ret': 'poollist'}, + 'Host_getDeviceInfo': {'ret': 'info'}, + 'Host_getDeviceList': {'ret': 'devList'}, + 'Host_getDevicesVisibility': {'ret': 'visibility'}, + 'Host_getLVMVolumeGroups': {'ret': 'vglist'}, + 'Host_getStats': {'ret': 'info'}, + 'Host_getStorageDomains': {'ret': 'domlist'}, + 'Host_getStorageRepoStats': {'ret': Host_getStorageRepoStats_Ret}, + 'Host_getVMList': {'call': Host_getVMList_Call, 'ret': Host_getVMList_Ret}, + 'Image_delete': {'ret': 'uuid'}, + 'Image_deleteVolumes': {'ret': 'uuid'}, + 'Image_getVolumes': {'ret': 'uuidlist'}, + 'Image_mergeSnapshots': {'ret': 'uuid'}, + 'Image_move': {'ret': 'uuid'}, + 'ISCSIConnection_discoverSendTargets': {'ret': 'fullTargets'}, + 'LVMVolumeGroup_create': {'ret': 'uuid'}, + 'LVMVolumeGroup_getInfo': {'ret': 'info'}, + 'StorageDomain_getFileList': {'ret': 'files'}, + 'StorageDomain_getImages': {'ret': 'imageslist'}, + 'StorageDomain_getInfo': {'ret': 'info'}, + 'StorageDomain_getStats': {'ret': 'stats'}, + 'StorageDomain_getVolumes': {'ret': 'uuidlist'}, + 'StoragePool_connectStorageServer': {'ret': 'statuslist'}, + 'StoragePool_disconnectStorageServer': {'ret': 'statuslist'}, + 'StoragePool_fence': {'ret': 'spm_st'}, + 'StoragePool_getBackedUpVmsInfo': {'ret': 'vmlist'}, + 'StoragePool_getBackedUpVmsList': {'ret': 'vmlist'}, + 'StoragePool_getDomainsContainingImage': {'ret': 'domainslist'}, + 'StoragePool_getFloppyList': {'ret': 'isolist'}, + 'StoragePool_getInfo': {'ret': StoragePool_getInfo_Ret}, + 'StoragePool_getIsoList': {'ret': 'isolist'}, + 'StoragePool_getSpmStatus': {'ret': 'spm_st'}, + 'StoragePool_spmStart': {'ret': 'uuid'}, + 'StoragePool_upgrade': {'ret': 'upgradeStatus'}, + 'StoragePool_validateStorageServerConnection': {'ret': 'statuslist'}, + 'Task_getInfo': {'ret': 'TaskInfo'}, + 'Task_getStatus': {'ret': 'taskStatus'}, + 'VM_changeCD': {'ret': 'vmList'}, + 'VM_changeFloppy': {'ret': 'vmList'}, + 'VM_create': {'ret': 'vmList'}, + 'VM_getInfo': {'call': VM_getInfo_Call, 'ret': VM_getInfo_Ret}, + 'VM_getStats': {'ret': 'statsList'}, + 'VM_hotplugDisk': {'ret': 'vmList'}, + 'VM_hotplugNic': {'ret': 'vmList'}, + 'VM_hotUnplugDisk': {'ret': 'vmList'}, + 'VM_hotUnplugNic': {'ret': 'vmList'}, + 'VM_mergeStatus': {'ret': 'mergeStatus'}, + 'VM_migrationCreate': {'ret': VM_migrationCreate_Ret}, + 'Volume_copy': {'ret': 'uuid'}, + 'Volume_create': {'ret': 'uuid'}, + 'Volume_delete': {'ret': 'uuid'}, + 'Volume_getInfo': {'ret': 'info'}, + 'Volume_getPath': {'ret': 'path'}, + 'Volume_getSize': {'ret': Volume_getsize_Ret}, + 'Host_getAllTasks': {'ret': 'TasksDetails'}, +} + + +def fieldClone(oldName, newName, obj): + logging.warning("fieldClone: %s -> %s", oldName, newName) + if oldName in obj: + obj[newName] = obj[oldName] + elif newName in obj: + obj[oldName] = obj[newName] + +typefixups = { + 'VmDevice': partial(fieldClone, 'type', 'deviceType'), + 'BlockDevicePathInfo': partial(fieldClone, 'type', 'deviceType'), + 'VolumeGroupInfo': partial(fieldClone, 'type', 'deviceType'), + 'VmDeviceAddress': partial(fieldClone, 'type', 'addressType'), + 'IscsiCredentials': partial(fieldClone, 'type', 'authType'), + 'ConnectionRefArgs': partial(fieldClone, 'type', 'connType'), + 'VolumeInfo': partial(fieldClone, 'type', 'allocType'), +} diff --git a/vdsm_api/Makefile.am b/vdsm_api/Makefile.am new file mode 100644 index 0000000..655586f --- /dev/null +++ b/vdsm_api/Makefile.am @@ -0,0 +1,13 @@ +EXTRA_DIST = \ + vdsmapi-schema.json \ + $(NULL) + +dist_vdsm_PYTHON = \ + BindingJsonRpc.py \ + Bridge.py \ + vdsmapi.py \ + $(NULL) + +dist_vdsm_DATA = \ + vdsmapi-schema.json \ + $(NULL) -- To view, visit http://gerrit.ovirt.org/8614 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Idae0faa80ffc6a5af002a8a7151aa40dc9a6673d Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <agl(a)us.ibm.com>

4 10

Change in vdsm[master]: vdsm.spec: Add vdsm-xmlrpc as Requirement
by Douglas Schilling Landgraf 12 Nov '12

12 Nov '12

Douglas Schilling Landgraf has uploaded a new change for review. Change subject: vdsm.spec: Add vdsm-xmlrpc as Requirement ...................................................................... vdsm.spec: Add vdsm-xmlrpc as Requirement For vdsm-json stuff and vdsm initialization, we should require vdsm-xmlrpc package. Change-Id: I41e8507edb2e5b72d6492229d4323b6574b2afc0 Signed-off-by: Douglas Schilling Landgraf <dougsland(a)redhat.com> --- M vdsm.spec.in 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/36/9136/1 diff --git a/vdsm.spec.in b/vdsm.spec.in index c117ee5..2f171b7 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -67,6 +67,7 @@ Requires: rpm-python Requires: nfs-utils Requires: python-pthreading +Requires: %{name}-xmlrpc %ifarch x86_64 Requires: dmidecode -- To view, visit http://gerrit.ovirt.org/9136 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I41e8507edb2e5b72d6492229d4323b6574b2afc0 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Douglas Schilling Landgraf <dougsland(a)redhat.com>

2 10

Change in vdsm[master]: netinfo: added configuration to show dummy NIC's too
by hercinger.viktor＠gmail.com 10 Nov '12

10 Nov '12

Hercinger Viktor has uploaded a new change for review. Change subject: netinfo: added configuration to show dummy NIC's too ...................................................................... netinfo: added configuration to show dummy NIC's too With the vars.fake_nics configuration a list of comma-separated interfaces can be specified. If a dummy interface matches the fnmatch pattern, it will be included in the NIC list. Change-Id: Ica0ad974e6dd0d144a6bb0904fdcd6bbce99ff9f Signed-off-by: Hercinger Viktor <hercinger.viktor(a)gmail.com> --- M tests/netinfoTests.py M vdsm/netinfo.py 2 files changed, 18 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/17/8917/1 diff --git a/tests/netinfoTests.py b/tests/netinfoTests.py index f5a4b95..a678fbc 100644 --- a/tests/netinfoTests.py +++ b/tests/netinfoTests.py @@ -23,6 +23,7 @@ from testrunner import VdsmTestCase as TestCaseBase from vdsm import netinfo +from vdsm import config from monkeypatch import MonkeyPatch # speeds defined in ethtool @@ -66,3 +67,6 @@ # just log a traceback. If it raises an exception the test will fail as # it should. netinfo.get() + + def testMatchNicName(self): + self.assertTrue(netinfo.match_nic_name('test1', ['test0', 'test1'])) diff --git a/vdsm/netinfo.py b/vdsm/netinfo.py index c3c5a6f..26ad247 100644 --- a/vdsm/netinfo.py +++ b/vdsm/netinfo.py @@ -19,6 +19,7 @@ # import os +import os.path import errno import glob import ethtool @@ -43,13 +44,22 @@ LIBVIRT_NET_PREFIX = 'vdsm-' +def match_nic_name(nic, patterns): + return any(map(lambda p: fnmatch(nic, p), patterns)) + def nics(): res = [] - for b in glob.glob('/sys/class/net/*/device'): - nic = b.split('/')[-2] - if not any(map(lambda p: fnmatch(nic, p), - config.get('vars', 'hidden_nics').split(','))): + hidden_nics = config.get('vars', 'hidden_nics').split(',') + fake_nics = config.get('vars', 'fake_nics').split(',') + + for b in glob.glob('/sys/class/net/*'): + nic = b.split('/')[-1] + if not os.path.exists(os.path.join(b, 'device')): + if match_nic_name(nic, fake_nics): + res.append(nic) + elif not match_nic_name(nic, hidden_nics): res.append(nic) + return res -- To view, visit http://gerrit.ovirt.org/8917 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ica0ad974e6dd0d144a6bb0904fdcd6bbce99ff9f Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Hercinger Viktor <hercinger.viktor(a)gmail.com>

4 10

Change in vdsm[master]: Report hsm tasks in getAllTasksStatuses and getAllTasksInfo
by smizrahi＠redhat.com 09 Nov '12

09 Nov '12

Saggi Mizrahi has uploaded a new change for review. Change subject: Report hsm tasks in getAllTasksStatuses and getAllTasksInfo ...................................................................... Report hsm tasks in getAllTasksStatuses and getAllTasksInfo Change-Id: I40a42db6554e5145d05f913c742d00dc26969899 Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M vdsm/storage/hsm.py 1 file changed, 2 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/72/6672/1 -- To view, visit http://gerrit.ovirt.org/6672 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I40a42db6554e5145d05f913c742d00dc26969899 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

4 10

Change in vdsm[master]: storage/threadPool: fix copyright notice
by Dan Kenigsberg 09 Nov '12

09 Nov '12

Dan Kenigsberg has uploaded a new change for review. Change subject: storage/threadPool: fix copyright notice ...................................................................... storage/threadPool: fix copyright notice Change-Id: Ib6a06dc60f53f12793a93bb71cd3d5bfec4cc0d1 Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M vdsm/storage/threadPool.py 1 file changed, 5 insertions(+), 17 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/19/9119/1 diff --git a/vdsm/storage/threadPool.py b/vdsm/storage/threadPool.py index 4cf4568..22633b7 100644 --- a/vdsm/storage/threadPool.py +++ b/vdsm/storage/threadPool.py @@ -1,21 +1,9 @@ # -# Copyright 2011 Red Hat, Inc. -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -# -# Refer to the README and COPYING files for full details of the license +# Adapted by Red Hat from +# http://code.activestate.com/recipes/203871-a-generic-programming-thread-poo… +# Author: Tim Lesher +# License: PSF License +# http://wiki.python.org/moin/PythonSoftwareFoundationLicenseV2Easy # import threading -- To view, visit http://gerrit.ovirt.org/9119 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib6a06dc60f53f12793a93bb71cd3d5bfec4cc0d1 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

2 4

Change in vdsm[master]: clienteIF: replace MOM self.log.error to warn
by Douglas Schilling Landgraf 09 Nov '12

09 Nov '12

Douglas Schilling Landgraf has uploaded a new change for review. Change subject: clienteIF: replace MOM self.log.error to warn ...................................................................... clienteIF: replace MOM self.log.error to warn If mom package is not installed, at this point vdsm should throw warning message instead of error. Change-Id: I860c5d0833ae0e7d9feb326d340a34dd68104d98 Signed-off-by: Douglas Schilling Landgraf <dougsland(a)redhat.com> --- M vdsm/clientIF.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/32/9132/1 diff --git a/vdsm/clientIF.py b/vdsm/clientIF.py index 6c5a670..ed2a3c1 100644 --- a/vdsm/clientIF.py +++ b/vdsm/clientIF.py @@ -178,7 +178,7 @@ momconf = config.get("mom", "conf") self.mom = MomThread(momconf) except: - self.log.error("MOM initialization failed and fall " + self.log.warn("MOM initialization failed and fall " "back to KsmMonitor", exc_info=True) self.ksmMonitor = ksm.KsmMonitorThread(self) -- To view, visit http://gerrit.ovirt.org/9132 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I860c5d0833ae0e7d9feb326d340a34dd68104d98 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Douglas Schilling Landgraf <dougsland(a)redhat.com>

3 4

Change in vdsm[master]: libvirtvm: set volume the format in live snapshots
by ewarszaw＠redhat.com 08 Nov '12

08 Nov '12

Hello Federico Simoncelli, I'd like you to do a code review. Please visit http://gerrit.ovirt.org/9129 to review the following change. Change subject: libvirtvm: set volume the format in live snapshots ...................................................................... libvirtvm: set volume the format in live snapshots When we update the drive objects (and configurations) at the end of a live snapshot we should also update the format to be "cow" in order to start requesting the drive extensions on block devices. Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=874481 Change-Id: Ia5535cc8ad55441046e81aa329fac82a354592b9 Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M vdsm/libvirtvm.py 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/29/9129/1 diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py index 86e39a3..ca31a43 100644 --- a/vdsm/libvirtvm.py +++ b/vdsm/libvirtvm.py @@ -1853,6 +1853,7 @@ newDrives[vmDevName] = tgetDrv.copy() newDrives[vmDevName]["poolID"] = vmDrive.poolID newDrives[vmDevName]["name"] = vmDevName + newDrives[vmDevName]["format"] = "cow" try: newDrives[vmDevName]["path"] = \ -- To view, visit http://gerrit.ovirt.org/9129 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia5535cc8ad55441046e81aa329fac82a354592b9 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com> Gerrit-Reviewer: Federico Simoncelli <fsimonce(a)redhat.com>

2 2

Change in vdsm[master]: Using only globalPool allow only 10 processes
by ybronhei＠redhat.com 08 Nov '12

08 Nov '12

Yaniv Bronhaim has uploaded a new change for review. Change subject: Using only globalPool allow only 10 processes ...................................................................... Using only globalPool allow only 10 processes For each domain we want to have process pool with limit slots of processes, this way we optimize glob over mounted links. This fix uses different pool for each domain, instead of the global one. Change-Id: I9a4024b8d8be8b9cbbb4836987dfb94e3a11d7d3 Signed-off-by: Yaniv Bronhaim <ybronhei(a)redhat.com> Bug-Id: https://bugzilla.redhat.com/show_bug.cgi?id=853011 --- M vdsm/storage/fileSD.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/29/9029/1 diff --git a/vdsm/storage/fileSD.py b/vdsm/storage/fileSD.py index a6a718f..f221eb0 100644 --- a/vdsm/storage/fileSD.py +++ b/vdsm/storage/fileSD.py @@ -479,7 +479,7 @@ def collectMetaFiles(possibleDomain): try: - metaFiles = oop.getGlobalProcPool().glob.glob( + metaFiles = oop.getProcessPool(possibleDomain).glob.glob( os.path.join(possibleDomain, constants.UUID_GLOB_PATTERN, sd.DOMAIN_META_DATA)) -- To view, visit http://gerrit.ovirt.org/9029 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I9a4024b8d8be8b9cbbb4836987dfb94e3a11d7d3 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim <ybronhei(a)redhat.com>

4 6

Change in vdsm[master]: schema: New type VmParameters
by agl＠us.ibm.com 07 Nov '12

07 Nov '12

Adam Litke has uploaded a new change for review. Change subject: schema: New type VmParameters ...................................................................... schema: New type VmParameters Although the format of the data passed via VM.create is very similar to a VmDefinition, some of the parameters don't make sense to pass in the context of creating a new VM. Solve this by creating a new type for VM parameters. Change-Id: I00d1b9aed55cbfc2210c1a4091bce17d45b90e67 Signed-off-by: Adam Litke <agl(a)us.ibm.com> --- M vdsm_api/vdsmapi-schema.json 1 file changed, 48 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/39/7839/1 diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index f6d3c77..b1dc60c 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -2346,6 +2346,53 @@ 'vmId': 'UUID', 'vmName': 'str', 'vmType': 'VmType'}} ## +# @VmDefinition: +# +# Full virtual machine status and properties. +# +# @acpiEnable: Indicates if ACPI is enabled inside the VM +# +# @custom: #opional A dictionary of custom, free-form properties +# +# @devices: #optional An array of VM devices requested +# +# @display: The type of display +# +# @kvmEnable: Indicates if KVM hardware acceleration is enabled +# +# @memSize: The amount of memory assigned to the VM in MB +# +# @nice: The host scheduling priority +# +# @smp: The number of CPUs presented to the VM +# +# @smpCoresPerSocket: #optional Indicates the number of CPU cores per socket +# +# @smpThreadsPerCore: #optional Indicates the number of CPU threads per core +# +# @timeOffset: The time difference from host to the VM in seconds +# +# @transparentHugePages: Indicates if the Transparent Huge Pages feature is +# enabled for this virtual machine +# +# @vmId: The VM UUID +# +# @vmName: The VM name +# +# @vmType: The type of VM +# +# Since: 4.10.0 +## +{'type': 'VmParameters', + 'data': {'acpiEnable': 'bool', + '*custom': 'StringMap', '*devices': ['VmDevice'], + 'display': 'VmDisplayType', 'kvmEnable': 'bool', 'memSize': 'uint', + 'nice': 'int', 'smp': 'uint', 'smpCoresPerSocket': 'uint', + 'smpThreadsPerCore': 'uint', 'timeOffset': 'uint', + 'transparentHugePages': 'bool', 'vmId': 'UUID', 'vmName': 'str', + 'vmType': 'VmType'}} + +## # @VmFullStatus: # # A VmDefinition uses the same format as a VmFullStatus. @@ -4278,7 +4325,7 @@ # Since: 4.10.0 ## {'command': {'class': 'VM', 'name': 'create'}, - 'data': {'vmParams': 'VmDefinition'}, + 'data': {'vmParams': 'VmParameters'}, 'returns': 'VmDefinition'} ## -- To view, visit http://gerrit.ovirt.org/7839 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I00d1b9aed55cbfc2210c1a4091bce17d45b90e67 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <agl(a)us.ibm.com>

4 12

Change in vdsm[master]: BZ#855924 -- fix 3.1 -> 3.0 migration again
by peet＠redhat.com 07 Nov '12

07 Nov '12

Peter V. Saveliev has uploaded a new change for review. Change subject: BZ#855924 -- fix 3.1 -> 3.0 migration again ...................................................................... BZ#855924 -- fix 3.1 -> 3.0 migration again The issue was that fixing self._vm.conf is not enough -- we should fix self._machineParams, that will work in all the cases. Signed-off-by: Peter V. Saveliev <peet(a)redhat.com> Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=855924 Change-Id: I334cd39cad3eed30c2c3f1efa7085526454a44fb --- M vdsm/vm.py 1 file changed, 4 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/36/9036/1 diff --git a/vdsm/vm.py b/vdsm/vm.py index 2229882..591ac1a 100644 --- a/vdsm/vm.py +++ b/vdsm/vm.py @@ -236,15 +236,15 @@ """ # care only about "drives" list, since # "devices" doesn't cause errors - if 'drives' in self._vm.conf: + if 'drives' in self._machineParams: for item in ("cdrom", "floppy"): new_drives = [] - for drive in self._vm.conf['drives']: + for drive in self._machineParams['drives']: if drive['device'] == item: - self._vm.conf[item] = drive['path'] + self._machineParams[item] = drive['path'] else: new_drives.append(drive) - self._vm.conf['drives'] = new_drives + self._machineParams['drives'] = new_drives def run(self): try: -- To view, visit http://gerrit.ovirt.org/9036 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I334cd39cad3eed30c2c3f1efa7085526454a44fb Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Peter V. Saveliev <peet(a)redhat.com>

4 9

Change in vdsm[master]: schema: Return objects rather than UUIDs
by agl＠us.ibm.com 07 Nov '12

07 Nov '12

Adam Litke has uploaded a new change for review. Change subject: schema: Return objects rather than UUIDs ...................................................................... schema: Return objects rather than UUIDs Rather than returning UUIDs and forcing the API user to instantiate their own objects using those UUIDs, just return initialized objects. This patch just changes the API schema. The specific implementation of this idea appears in the patch that implements the code generator. This is not exhaustive, but an example of the idea. The rest can be converted over time. Change-Id: I7fdfe69436d65ef8190d8a5f1d941e7a628c77c0 Signed-off-by: Adam Litke <agl(a)us.ibm.com> --- M vdsm_api/vdsmapi-schema.json 1 file changed, 87 insertions(+), 71 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/40/7840/1 diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index b1dc60c..7e8b9d3 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -939,12 +939,12 @@ # Get a list of all Storage Pools that are connected to this host. # # Returns: -# A list of Storage Pool UUIDs +# A list of Storage Pools # # Since: 4.10.0 ## {'command': {'class': 'Host', 'name': 'getConnectedStoragePools'}, - 'returns': ['UUID']} + 'returns': ['StoragePool']} ## # @BlockDeviceType: @@ -1561,15 +1561,15 @@ ## # @Host.getStorageDomains: # -# Get a list of known Storage Domain UUIDs. +# Get a list of known Storage Domains. # -# @spUUID: #optional Limit to Domains belonging to this Storage Pool +# @storagepoolID: #optional Limit to Domains belonging to this Storage Pool # -# @domainClass: #optional Limit to Domains of this @StorageDomainImageClass +# @domainClass: #optional Limit to Domains of this @StorageDomainImageClass # -# @storageType: #optional Limit to Domains of this @StorageDomainType +# @storageType: #optional Limit to Domains of this @StorageDomainType # -# @remotePath: #optional Limit to Domains having this remotePath +# @remotePath: #optional Limit to Domains having this remotePath # # Returns: # A list of known Storage Domains @@ -1577,9 +1577,9 @@ # Since: 4.10.0 ## {'command': {'class': 'Host', 'name': 'getStorageDomains'}, - 'data': {'*spUUID': 'UUID', '*domainClass': 'StorageDomainImageClass', + 'data': {'*storagepoolID': 'UUID', '*domainClass': 'StorageDomainImageClass', '*storageType': 'StorageDomainType', '*remotePath': 'str'}, - 'returns': ['UUID']} + 'returns': ['StorageDomain']} ## # @Host.getStorageRepoStats: @@ -2419,18 +2419,16 @@ # # Get information about the current virtual machines. # -# @fullStatus: #optional Set to request complete VM information -# # @vmList: #optional Filter the results by a list of UUIDs # # Returns: -# A list of VM definitions +# A list of VMs # # Since: 4.10.0 ## {'command': {'class': 'Host', 'name': 'getVMList'}, - 'data': {'*fullStatus': 'bool', '*vmList': ['UUID']}, - 'returns': ['VmInfo']} + 'data': {'*vmList': ['UUID']}, + 'returns': ['VM']} ## # @Host.ping: @@ -2810,18 +2808,19 @@ # # Image API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: The UUID of the Image +# @imageID: The UUID of the Image # -# @spUUID: The UUID of the Storage Pool associated with the Image +# @storagepoolID: The UUID of the Storage Pool associated with the Image # -# @sdUUID: The UUID of the Storage Domain associated with the Image +# @storagedomainID: The UUID of the Storage Domain associated with the Image # # Since: 4.10.0 ## {'class': 'Image', - 'data': {'conn': 'Host', 'UUID': 'UUID', 'spUUID': 'UUID', 'sdUUID': 'UUID'}} + 'data': {'conn': 'Host', 'imageID': 'UUID', 'storagepoolID': 'UUID', + 'storagedomainID': 'UUID'}} ## # @Image.delete: @@ -2867,12 +2866,12 @@ # Get a list of Volumes associated with this Image. # # Returns: -# A list of Image UUIDs +# A list of Volumes # # Since: 4.10.0 ## {'command': {'class': 'Image', 'name': 'getVolumes'}, - 'returns': ['UUID']} + 'returns': ['Volume']} ## # @Image.mergeSnapshots: @@ -2938,13 +2937,15 @@ # # LVMVolumeGroup API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: #optional Associate this object with an existing LVM Volume Group +# @lvmvolumegroupID: #optional Associate this object with an existing LVM +# Volume Group # # Since: 4.10.0 ## -{'class': 'LVMVolumeGroup', 'data': {'conn': 'Host', 'UUID': 'UUID'}} +{'class': 'LVMVolumeGroup', + 'data': {'conn': 'Host', 'lvmvolumegroupID': 'UUID'}} ## # @LVMVolumeGroup.create: @@ -2997,16 +2998,17 @@ # # StorageDomain API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: Associate this object with a new or existing Storage Domain +# @storagedomainID: Associate this object with a new or existing Storage Domain # -# @spUUID: #optional The Storage Pool UUID if this Storage Domain is attached +# @storagepoolID: #optional The Storage Pool UUID if this Storage Domain is +# attached # # Since: 4.10.0 ## {'class': 'StorageDomain', - 'data': {'conn': 'Host', 'UUID': 'UUID', '*spUUID': 'UUID'}} + 'data': {'conn': 'Host', 'storagedomainID': 'UUID', 'storagepoolID': 'UUID'}} ## # @StorageDomain.activate: @@ -3022,12 +3024,12 @@ # # Attach a Storage Domain to a Storage Pool. # -# @spUUID: The Storage Pool to which the Storage Domain should be attached +# @storagepoolID: The UUID of the pool to which this domain should be attached # # Since: 4.10.0 ## {'command': {'class': 'StorageDomain', 'name': 'attach'}, - 'data': {'spUUID': 'UUID'}} + 'data': {'storagepoolID': 'UUID'}} ## # @StorageDomainCreateArgumentsBlock: @@ -3209,12 +3211,12 @@ # Get a list of Images associated with this Storage Domain. # # Returns: -# An array of Image UUIDs +# An array of Imags # # Since: 4.10.0 ## {'command': {'class': 'StorageDomain', 'name': 'getImages'}, - 'returns': ['UUID']} + 'returns': ['Image']} ## # @StorageDomainRole: @@ -3316,16 +3318,16 @@ # # Get a list of Volumes contained within a Storage Domain. # -# @imgUUID: Limit results to Volumes associated with a single Image +# @imageID: Limit results to Volumes associated with a single Image # # Returns: -# A list of Volume UUIDs +# A list of Volumes # # Since: 4.10.0 ## {'command': {'class': 'StorageDomain', 'name': 'getVolumes'}, - 'data': {'imgUUID': 'UUID'}, - 'returns': ['UUID']} + 'data': {'imageID': 'UUID'}, + 'returns': ['Volume']} ## # @StorageDomain.setDescription: @@ -3357,21 +3359,21 @@ # # Upload a Volume into a Storage Domain from a remote location. # -# @imgUUID: The UUID of the image that is associated with the Volume +# @imageID: The UUID of the image that is associated with the Volume # -# @volUUID: The UUID of an existing Volume where the data will be uploaded +# @volumeID: The UUID of an existing Volume where the data will be uploaded # -# @srcPath: The remote location of the Volume data. Must be in a format that -# can be understood by the command indicated in @method. +# @srcPath: The remote location of the Volume data. Must be in a format that +# can be understood by the command indicated in @method. # -# @size: The size of the volume data in blocks +# @size: The size of the volume data in blocks # -# @method: The method (command) to use to upload the data +# @method: The method (command) to use to upload the data # # Since: 4.10.0 ## {'command': {'class': 'StorageDomain', 'name': 'uploadVolume'}, - 'data': {'imgUUID': 'UUID', 'volUUID': 'UUID', 'srcPath': 'str', + 'data': {'imageID': 'UUID', 'volumeID': 'UUID', 'srcPath': 'str', 'size': 'int', 'method': 'UploadVolumeMethod'}} ## @@ -3389,13 +3391,13 @@ # # StoragePool API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: Associate this object with a new or existing Storage Pool +# @storagepoolID: Associate this object with a new or existing Storage Pool # # Since: 4.10.0 ## -{'class': 'StoragePool', 'data': {'conn': 'Host', 'UUID': 'UUID'}} +{'class': 'StoragePool', 'data': {'conn': 'Host', 'storagepoolID': 'UUID'}} ## # @StoragePool.connect: @@ -3600,7 +3602,7 @@ # # Get information about backed-up virtual machines from a Backup Storage Domain. # -# @sdUUID: The UUID of the Backup Storage Domain to check +# @storagedomainID: The UUID of the Backup Storage Domain to check # # @vmList: Limit results to a list of VM UUIDs # @@ -3613,7 +3615,7 @@ # previously injected it into the Backup Storage Domain. ## {'command': {'class': 'StoragePool', 'name': 'getBackedUpVmsInfo'}, - 'data': {'sdUUID': 'UUID', 'vmList': ['UUID']}, + 'data': {'storagedomainID': 'UUID', 'vmList': ['UUID']}, 'returns': 'OVFMap'} ## @@ -3621,7 +3623,7 @@ # # Get a list of backed up virtual machines from a Backup Storage Domain. # -# @sdUUID: The UUID of the Backup Storage Domain to check +# @storagedomainID: The UUID of the Backup Storage Domain to check # # Returns: # A list of VM UUIDs @@ -3629,7 +3631,7 @@ # Since: 4.10.0 ## {'command': {'class': 'StoragePool', 'name': 'getBackedUpVmsList'}, - 'data': {'sdUUID': 'UUID'}, + 'data': {'storagedomainID': 'UUID'}, 'returns': ['UUID']} ## @@ -3650,18 +3652,18 @@ # # Get a list of Storage Domains that contain an Image. # -# @imgUUID: The UUID of the Image to search for +# @imageID: The UUID of the Image to search for # # @onlyDataDomains: #optional Only include Data Storage Domains # # Returns: -# A list of Storage Domain UUIDs that contain the Image +# A list of Storage Domains that contain the Image # # Since: 4.10.0 ## {'command': {'class': 'StoragePool', 'name': 'getDomainsContainingImage'}, - 'data': {'imgUUID': 'UUID', '*onlyDataDomains': 'bool'}, - 'returns': ['UUID']} + 'data': {'imageID': 'UUID', '*onlyDataDomains': 'bool'}, + 'returns': ['StorageDomain']} ## # @StoragePool.getIsoList: @@ -4065,26 +4067,26 @@ # # @vmList: A list of virtual machine definitions to store # -# @sdUUID: The Storage Domain to use for storing the VM definitions +# @storagedomainID: The Storage Domain to use for storing the VM definitions # # Since: 4.10.0 ## {'command': {'class': 'StoragePool', 'name': 'updateVMs'}, - 'data': {'vmList': ['UpdateVmDefinition'], 'sdUUID': 'UUID'}} + 'data': {'vmList': ['UpdateVmDefinition'], 'storagedomainID': 'UUID'}} ## # @StoragePool.removeVM: # # Remove a previously saved virtual machine definition. # -# @vmUUID: Remove the saved definition of the VM with this UUID +# @vmUUID: Remove the saved definition of the VM with this UUID # -# @sdUUID: The Storage Domain where the VM is stored +# @storagedomainID: The Storage Domain where the VM is stored # # Since: 4.10.0 ## {'command': {'class': 'StoragePool', 'name': 'removeVM'}, - 'data': {'vmUUID': 'UUID', 'sdUUID': 'UUID'}} + 'data': {'vmUUID': 'UUID', 'storagedomainID': 'UUID'}} ## Category: @Task ############################################################# ## @@ -4092,13 +4094,13 @@ # # Task API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: Associate this object with an existing Task +# @taskID: Associate this object with an existing Task # # Since: 4.10.0 ## -{'class': 'Task', 'data': {'conn': 'Host', 'UUID': 'UUID'}} +{'class': 'Task', 'data': {'conn': 'Host', 'taskID': 'UUID'}} ## # @Task.clear: @@ -4161,11 +4163,11 @@ # # @conn: A connected base API object # -# @UUID: Associate this object with an existing VM +# @vmID: Associate this object with an existing VM # # Since: 4.10.0 ## -{'class': 'VM', 'data': {'conn': 'Host', 'UUID': 'UUID'}} +{'class': 'VM', 'data': {'conn': 'Host', 'vmID': 'UUID'}} ## # @DriveSpecVolume: @@ -4387,6 +4389,20 @@ # Since: 4.10.0 ## {'command': {'class': 'VM', 'name': 'destroy'}} + +## +# @VM.getInfo: +# +# Get detailed information about a VM. +# +# Returns: +# VM information +# +# Since: 4.10.0 +## +{'command': {'class': 'VM', 'name': 'getInfo'}, + 'returns': 'VmFullStatus'} + ## # @VM.getMigrationStatus: @@ -5179,21 +5195,21 @@ # # Volume API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: The UUID of the Volume +# @volumeID: The UUID of the Volume # -# @spUUID: The Storage Pool associated with @UUID +# @storagepoolID: The Storage Pool associated with @UUID # -# @sdUUID: The Storage Domain associated with @UUID +# @storagedomainID: The Storage Domain associated with @UUID # -# @imgUUID: The Image associated with @UUID +# @imageID: The Image associated with @UUID # # Since: 4.10.0 ## {'class': 'Volume', - 'data': {'conn': 'Host', 'UUID': 'UUID', 'spUUID': 'UUID', 'sdUUID': 'UUID', - 'imgUUID': 'UUID'}} + 'data': {'conn': 'Host', 'volumeID': 'UUID', 'storagepoolID': 'UUID', + 'storagedomainID': 'UUID', 'imageID': 'UUID'}} ## # @VolumeRole: -- To view, visit http://gerrit.ovirt.org/7840 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7fdfe69436d65ef8190d8a5f1d941e7a628c77c0 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <agl(a)us.ibm.com>

3 4

Change in vdsm[master]: schema: Remove VmFullStatus
by agl＠us.ibm.com 07 Nov '12

07 Nov '12

Adam Litke has uploaded a new change for review. Change subject: schema: Remove VmFullStatus ...................................................................... schema: Remove VmFullStatus The VmFullStatus alias is a meaningless indirection that simply causes confusion to API users. In all cases, its use just means VmDefinition so use that type instead. Change-Id: Id16979a105521e3c059ba27029493b71158f8815 Signed-off-by: Adam Litke <agl(a)us.ibm.com> --- M vdsm_api/vdsmapi-schema.json 1 file changed, 2 insertions(+), 11 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/13/8613/1 diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index 5d4e10f..3f8f0a0 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -2394,15 +2394,6 @@ 'vmType': 'VmType'}} ## -# @VmFullStatus: -# -# A VmFullStatus uses the same format as a VmDefinition. -# -# Since: 4.10.0 -## -{'alias': 'VmFullStatus', 'data': 'VmDefinition'} - -## # @VmInfo: # # A discriminated record of VM status information. @@ -2413,7 +2404,7 @@ ## {'type': 'VmInfo', 'data': {}, - 'union': ['VmFullStatus', 'VmShortStatus']} + 'union': ['VmDefinition', 'VmShortStatus']} ## # @Host.getVMList: @@ -4910,7 +4901,7 @@ # Since: 4.10.0 ## {'type': 'MigrationCreateState', - 'data': {'migrationPort': 'int', 'params': 'VmFullStatus'}} + 'data': {'migrationPort': 'int', 'params': 'VmDefinition'}} ## # @VM.migrationCreate: -- To view, visit http://gerrit.ovirt.org/8613 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id16979a105521e3c059ba27029493b71158f8815 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <agl(a)us.ibm.com>

4 4

Change in vdsm[master]: fix REMOVED_IMAGE_PREFIX not defined error
by zhshzhou＠linux.vnet.ibm.com 07 Nov '12

07 Nov '12

Zhou Zheng Sheng has uploaded a new change for review. Change subject: fix REMOVED_IMAGE_PREFIX not defined error ...................................................................... fix REMOVED_IMAGE_PREFIX not defined error Recently http://gerrit.ovirt.org/#/c/8506 is merged, it moves REMOVED_IMAGE_PREFIX = "_remove_me_" from vdsm/storage/image.py to vdsm/storage/sd.py. So all occurrence of REMOVED_IMAGE_PREFIX in image.py must be changed to sd.REMOVED_IMAGE_PREFIX, otherwise pyflakes will say REMOVED_IMAGE_PREFIX not defined. Change-Id: Ifc6ec49ff20c7f8a18ae14fe205570dfc6ce7634 Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- M vdsm/storage/image.py 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/98/9098/1 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 6ae5908..4cd9785 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -234,7 +234,7 @@ # Otherwise move it out of the way newImgUUID = self.deletedVolumeName(imgUUID) self.log.info("Rename image %s -> %s", imgUUID, newImgUUID) - if not imgUUID.startswith(REMOVED_IMAGE_PREFIX): + if not imgUUID.startswith(sd.REMOVED_IMAGE_PREFIX): removedImage = os.path.join(os.path.dirname(imageDir), newImgUUID) os.rename(imageDir, removedImage) else: @@ -242,7 +242,7 @@ volumes = [volclass(self.repoPath, sdUUID, newImgUUID, volUUID) for volUUID in uuidlist] for vol in volumes: - if not vol.volUUID.startswith(REMOVED_IMAGE_PREFIX): + if not vol.volUUID.startswith(sd.REMOVED_IMAGE_PREFIX): vol.rename(self.deletedVolumeName(vol.volUUID), recovery=False) else: self.log.warning("Volume %s of image %s already renamed", vol.volUUID, imgUUID) -- To view, visit http://gerrit.ovirt.org/9098 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ifc6ec49ff20c7f8a18ae14fe205570dfc6ce7634 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com>

3 3

Re: [oVirt Jenkins] vdsm_unit_tests - Build # 1502 - Still Failing!
by Dan Kenigsberg 07 Nov '12

07 Nov '12

On Tue, Nov 06, 2012 at 12:41:42PM -0500, Jenkins oVirt Server wrote: > > BUILD FAILURE > Build URL: http://jenkins.ovirt.org/job/vdsm_unit_tests/1502/ > Project: vdsm_unit_tests > Date of build: Tue, 06 Nov 2012 12:41:22 -0500 > Build duration: 20 sec Eyal, please take a look at this machine: it seems that /home/jenkins/workspace/vdsm_unit_tests/ has some trash remainder, an old vdsm/pthread.pyc. Would you make sure that the directory is clean before checking out from git? Dan.

2 1

Change in vdsm[master]: Make utils pep8 compliant
by smizrahi＠redhat.com 07 Nov '12

07 Nov '12

Saggi Mizrahi has uploaded a new change for review. Change subject: Make utils pep8 compliant ...................................................................... Make utils pep8 compliant Change-Id: I514660d73071f8fb417fea0cc54973a55002992f Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M Makefile.am M vdsm/utils.py 2 files changed, 117 insertions(+), 69 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/92/9092/1 diff --git a/Makefile.am b/Makefile.am index bd262af..1aa6e86 100644 --- a/Makefile.am +++ b/Makefile.am @@ -97,6 +97,7 @@ vdsm/supervdsm.py \ vdsm/supervdsmServer.py \ vdsm/tc.py \ + vdsm/utils.py \ vdsm/vdsm \ vdsm/vdsmDebugPlugin.py \ vdsm/vm.py \ diff --git a/vdsm/utils.py b/vdsm/utils.py index 002094a..b4614e7 100644 --- a/vdsm/utils.py +++ b/vdsm/utils.py @@ -19,7 +19,8 @@ # """ -A module containing miscellaneous functions and classes that are user plentifuly around vdsm. +A module containing miscellaneous functions and classes that are user +plentifuly around vdsm. .. attribute:: utils.symbolerror @@ -28,7 +29,8 @@ from SimpleXMLRPCServer import SimpleXMLRPCServer import SocketServer import threading -import os, time +import os +import time import logging import errno import subprocess @@ -47,6 +49,7 @@ if not os.path.exists(_THP_STATE_PATH): _THP_STATE_PATH = '/sys/kernel/mm/redhat_transparent_hugepage/enabled' + def isBlockDevice(path): path = os.path.abspath(path) return stat.S_ISBLK(os.stat(path).st_mode) @@ -63,6 +66,7 @@ os.unlink(fileToRemove) except: pass + def readMemInfo(): """ @@ -92,9 +96,12 @@ raise time.sleep(0.1) + #Threaded version of SimpleXMLRPCServer -class SimpleThreadedXMLRPCServer(SocketServer.ThreadingMixIn, SimpleXMLRPCServer): +class SimpleThreadedXMLRPCServer(SocketServer.ThreadingMixIn, + SimpleXMLRPCServer): allow_reuse_address = True + class InterfaceSample: """ @@ -123,7 +130,8 @@ try: s = file(f).read() except IOError, e: - if e.errno != errno.ENOENT: # silently ignore missing wifi stats + # silently ignore missing wifi stats + if e.errno != errno.ENOENT: logging.debug("Could not read %s", f, exc_info=True) return 0 try: @@ -131,7 +139,7 @@ except: if s != '': logging.warning("Could not parse statistics (%s) from %s", - f, s, exc_info=True) + f, s, exc_info=True) logging.debug('bad %s: (%s)', f, s) if not tries: raise @@ -140,7 +148,8 @@ """ Return the operational state of the interface. - :returns: ``'up'`` if interface is up. ``'down'`` or ``0`` id it's down. + :returns: ``'up'`` if interface is up. ``'down'`` or ``0`` id it's + down. """ try: flags = ethtool.get_flags(ifid) @@ -166,8 +175,9 @@ """ def __init__(self): self.user, userNice, self.sys, self.idle = \ - map(int, file('/proc/stat').readline().split()[1:5]) + map(int, file('/proc/stat').readline().split()[1:5]) self.user += userNice + class PidCpuSample: """ @@ -177,11 +187,13 @@ """ def __init__(self, pid): self.user, self.sys = \ - map(int, file('/proc/%s/stat' % pid).read().split()[13:15]) + map(int, file('/proc/%s/stat' % pid).read().split()[13:15]) + class TimedSample: def __init__(self): self.timestamp = time.time() + class BaseSample(TimedSample): """ @@ -189,10 +201,11 @@ """ def __init__(self, pid, ifids): TimedSample.__init__(self) - self.interfaces= {} + self.interfaces = {} for ifid in ifids: self.interfaces[ifid] = InterfaceSample(ifid) self.pidcpu = PidCpuSample(pid) + class HostSample(BaseSample): """ @@ -200,7 +213,8 @@ Contains the sate of the host in the time of initialization. """ - MONITORED_PATHS = ['/tmp', '/var/log', '/var/log/core', constants.P_VDSM_RUN] + MONITORED_PATHS = ['/tmp', '/var/log', '/var/log/core', + constants.P_VDSM_RUN] def _getDiskStats(self): d = {} @@ -208,7 +222,7 @@ free = 0 try: stat = os.statvfs(p) - free = stat.f_bavail * stat.f_bsize / 2**20 + free = stat.f_bavail * stat.f_bsize / (2 ** 20) except: pass d[p] = {'free': str(free)} @@ -238,9 +252,10 @@ try: with file(_THP_STATE_PATH) as f: s = f.read() - self.thpState = s[s.index('[')+1:s.index(']')] + self.thpState = s[s.index('[') + 1:s.index(']')] except: self.thpState = 'never' + class AdvancedStatsFunction(object): """ @@ -265,8 +280,8 @@ return self._interval def __repr__(self): - return "<AdvancedStatsFunction %s at 0x%x>" % \ - (self._function.__name__, id(self._function.__name__)) + return "<AdvancedStatsFunction %s at 0x%x>" % ( + self._function.__name__, id(self._function.__name__)) def __call__(self, *args, **kwargs): retValue = self._function(*args, **kwargs) @@ -284,12 +299,14 @@ the first and the last return value in the defined 'window' and the time difference. """ - if len(self._sample) < 2: return None, None, None + if len(self._sample) < 2: + return None, None, None bgn_time, bgn_sample = self._sample[0] end_time, end_sample = self._sample[-1] return bgn_sample, end_sample, (end_time - bgn_time) + class AdvancedStatsThread(threading.Thread): """ @@ -386,8 +403,8 @@ waitInterval = maxInterval for statsFunction in self._statsFunctions: - thisInterval = statsFunction.interval - \ - (intervalAccum % statsFunction.interval) + thisInterval = statsFunction.interval - ( + intervalAccum % statsFunction.interval) waitInterval = min(waitInterval, thisInterval) if intervalAccum % statsFunction.interval == 0: @@ -401,6 +418,7 @@ self._stopEvent.wait(waitInterval) intervalAccum = (intervalAccum + waitInterval) % maxInterval + class StatsThread(threading.Thread): """ A thread that periodically checks the stats of interfaces @@ -408,6 +426,7 @@ AVERAGING_WINDOW = 5 SAMPLE_INTERVAL_SEC = 2 MBITTOBYTES = 1000000 / 8 + def __init__(self, log, ifids, ifrates, ifmacs): threading.Thread.__init__(self) self._log = log @@ -418,7 +437,8 @@ self._ifrates = ifrates self._ifmacs = ifmacs self._ncpus = 1 - self._lineRate = (sum(ifrates) or 1000) * 10**6 / 8 # in bytes-per-second + # in bytes-per-second + self._lineRate = (sum(ifrates) or 1000) * (10 ** 6) / 8 self._paused = False self._lastSampleTime = time.time() @@ -431,7 +451,7 @@ def cont(self): self._paused = False - def sample(self): # override + def sample(self): # override pass def run(self): @@ -449,7 +469,7 @@ self._samples.pop(0) except libvirtvm.TimeoutError: self._log.error("Timeout while sampling stats", - exc_info=True) + exc_info=True) self._stopEvent.wait(self.SAMPLE_INTERVAL_SEC) except: if not self._stopEvent.isSet(): @@ -480,27 +500,33 @@ for ifid, ifrate, ifmac in zip(self._ifids, self._ifrates, self._ifmacs): ifrate = ifrate or 1000 - Mbps2Bps = 10**6 / 8 - thisRx = (hs1.interfaces[ifid].rx - hs0.interfaces[ifid].rx) % 2**32 - thisTx = (hs1.interfaces[ifid].tx - hs0.interfaces[ifid].tx) % 2**32 + Mbps2Bps = (10 ** 6) / 8 + thisRx = (hs1.interfaces[ifid].rx - hs0.interfaces[ifid].rx) % \ + (2 ** 32) + thisTx = (hs1.interfaces[ifid].tx - hs0.interfaces[ifid].tx) % \ + (2 ** 32) rxRate = 100.0 * thisRx / interval / ifrate / Mbps2Bps txRate = 100.0 * thisTx / interval / ifrate / Mbps2Bps if txRate > 100 or rxRate > 100: txRate = min(txRate, 100.0) rxRate = min(rxRate, 100.0) - self._log.debug('Rate above 100%%. DEBUG: ifid %s interval: %s thisRx %s thisTx %s samples %s', ifid, interval, thisRx, thisTx, [(hs.timestamp, hs.interfaces[ifid].rx, hs.interfaces[ifid].tx) for hs in self._samples]) + self._log.debug('Rate above 100%%. DEBUG: ifid %s interval: ' + '%s thisRx %s thisTx %s samples %s', ifid, + interval, thisRx, thisTx, + [(hs.timestamp, hs.interfaces[ifid].rx, + hs.interfaces[ifid].tx) + for hs in self._samples]) + iface = hs1.interfaces[ifid] stats['network'][ifid] = {'name': ifid, 'speed': str(ifrate), - 'rxDropped': str(hs1.interfaces[ifid].rxDropped), - 'txDropped': str(hs1.interfaces[ifid].txDropped), - 'rxErrors': str(hs1.interfaces[ifid].rxErrors), - 'txErrors': str(hs1.interfaces[ifid].txErrors), - 'state': hs1.interfaces[ifid].operstate, - 'rxRate': '%.1f' % rxRate, - 'txRate': '%.1f' % txRate, - 'macAddr': ifmac, -# 'interval': str(interval), 'thisRx': str(thisRx), 'thisTx': str(thisTx), -# 'samples': str([(hs.timestamp, hs.interfaces[ifid].rx, hs.interfaces[ifid].tx) for hs in self._samples]) - } + 'rxDropped': str(iface.rxDropped), + 'txDropped': str(iface.txDropped), + 'rxErrors': str(iface.rxErrors), + 'txErrors': str(iface.txErrors), + 'state': iface.operstate, + 'rxRate': '%.1f' % rxRate, + 'txRate': '%.1f' % txRate, + 'macAddr': ifmac, + } rx += thisRx tx += thisTx rxDropped += hs1.interfaces[ifid].rxDropped @@ -518,6 +544,7 @@ return stats + class HostStatsThread(StatsThread): """ A thread that periodically samples host statistics. @@ -525,7 +552,7 @@ def __init__(self, cif, log, ifids, ifrates): self.startTime = time.time() StatsThread.__init__(self, log, ifids, ifrates, - [''] * len(ifids)) # fake ifmacs + [''] * len(ifids)) # fake ifmacs self._imagesStatus = ImagePathStatus(cif) self._pid = os.getpid() self._ncpus = len(os.listdir('/sys/class/cpuid/')) @@ -540,8 +567,8 @@ i = 0 for ifid in self._ifids: - if hs0.interfaces[ifid].operstate != \ - hs1.interfaces[ifid].operstate: + if (hs0.interfaces[ifid].operstate != + hs1.interfaces[ifid].operstate): self._ifrates[i] = nicspeed(ifid) i += 1 @@ -560,25 +587,25 @@ now = time.time() for sd, d in self._imagesStatus.storageDomains.iteritems(): stats['storageDomains'][sd] = {'code': d['code'], - 'delay': d['delay'], - 'lastCheck': d['lastCheck'], - 'valid': d['valid']} + 'delay': d['delay'], + 'lastCheck': d['lastCheck'], + 'valid': d['valid']} stats['elapsedTime'] = int(now - self.startTime) if len(self._samples) < 2: return stats hs0, hs1 = self._samples[0], self._samples[-1] interval = hs1.timestamp - hs0.timestamp - jiffies = (hs1.pidcpu.user - hs0.pidcpu.user) % 2**32 - stats['cpuUserVdsmd'] = (jiffies / interval) % 2**32 + jiffies = (hs1.pidcpu.user - hs0.pidcpu.user) % (2 ** 32) + stats['cpuUserVdsmd'] = (jiffies / interval) % (2 ** 32) jiffies = hs1.pidcpu.sys - hs0.pidcpu.sys - stats['cpuSysVdsmd'] = (jiffies / interval) % 2**32 + stats['cpuSysVdsmd'] = (jiffies / interval) % (2 ** 32) - jiffies = (hs1.totcpu.user - hs0.totcpu.user) % 2**32 + jiffies = (hs1.totcpu.user - hs0.totcpu.user) % (2 ** 32) stats['cpuUser'] = jiffies / interval / self._ncpus - jiffies = (hs1.totcpu.sys - hs0.totcpu.sys) % 2**32 + jiffies = (hs1.totcpu.sys - hs0.totcpu.sys) % (2 ** 32) stats['cpuSys'] = jiffies / interval / self._ncpus stats['cpuIdle'] = max(0.0, - 100.0 - stats['cpuUser'] - stats['cpuSys']) + 100.0 - stats['cpuUser'] - stats['cpuSys']) stats['memUsed'] = hs1.memUsed stats['anonHugePages'] = hs1.anonHugePages stats['cpuLoad'] = hs1.cpuLoad @@ -587,7 +614,8 @@ stats['thpState'] = hs1.thpState return stats -def convertToStr (val): + +def convertToStr(val): varType = type(val) if varType is float: return '%.2f' % (val) @@ -596,10 +624,12 @@ else: return val + def execCmd(*args, **kwargs): # import only after config as been initialized from storage.misc import execCmd return execCmd(*args, **kwargs) + def checkPathStat(pathToCheck): try: @@ -610,8 +640,9 @@ except: return (False, 0) + class ImagePathStatus(threading.Thread): - def __init__ (self, cif, interval=None): + def __init__(self, cif, interval=None): if interval is None: interval = config.getint('irs', 'images_check_times') self._interval = interval @@ -622,7 +653,7 @@ if self._interval > 0: self.start() - def stop (self): + def stop(self): self._stopEvent.set() def _refreshStorageDomains(self): @@ -631,7 +662,7 @@ if "args" in self.storageDomains: del self.storageDomains["args"] - def run (self): + def run(self): try: while not self._stopEvent.isSet(): if self._cif.irs: @@ -639,7 +670,8 @@ self._stopEvent.wait(self._interval) except: logging.error("Error while refreshing storage domains", - exc_info=True) + exc_info=True) + def getPidNiceness(pid): """ @@ -651,9 +683,10 @@ stat = file('/proc/%s/stat' % (pid)).readlines()[0] return int(stat.split(') ')[-1].split()[16]) + def tobool(s): try: - if s == None: + if s is None: return False if type(s) == bool: return s @@ -662,6 +695,7 @@ return bool(int(s)) except: return False + def _getAllMacs(): @@ -689,6 +723,8 @@ return set(macs) - set(["", "00:00:00:00:00:00"]) __hostUUID = '' + + def getHostUUID(): global __hostUUID if __hostUUID: @@ -702,12 +738,14 @@ __hostUUID = f.readline().replace("\n", "") else: p = subprocess.Popen([constants.EXT_SUDO, - constants.EXT_DMIDECODE, "-s", "system-uuid"], + constants.EXT_DMIDECODE, "-s", + "system-uuid"], close_fds=True, stdin=subprocess.PIPE, - stdout=subprocess.PIPE, stderr=subprocess.PIPE) + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) out, err = p.communicate() - out = '\n'.join( line for line in out.splitlines() - if not line.startswith('#') ) + out = '\n'.join(line for line in out.splitlines() + if not line.startswith('#')) if p.returncode == 0 and 'Not' not in out: #Avoid error string - 'Not Settable' or 'Not Present' @@ -734,6 +772,7 @@ for code, symbol in errno.errorcode.iteritems(): symbolerror[os.strerror(code)] = symbol + def getUserPermissions(userName, path): """ Return a dictionary with user specific permissions with respect to the @@ -759,19 +798,20 @@ isSameOwner = userInfo.pw_uid == fileStats.st_uid # 'Other' permissions are the base permissions - permissions['read'] = isRead(otherBits) or \ - isSameGroup and isRead(groupBits) or \ - isSameOwner and isRead(ownerBits) + permissions['read'] = (isRead(otherBits) or + isSameGroup and isRead(groupBits) or + isSameOwner and isRead(ownerBits)) - permissions['write'] = isWrite(otherBits) or \ - isSameGroup and isWrite(groupBits) or \ - isSameOwner and isWrite(ownerBits) + permissions['write'] = (isWrite(otherBits) or + isSameGroup and isWrite(groupBits) or + isSameOwner and isWrite(ownerBits)) - permissions['exec'] = isExec(otherBits) or \ - isSameGroup and isExec(groupBits) or \ - isSameOwner and isExec(ownerBits) + permissions['exec'] = (isExec(otherBits) or + isSameGroup and isExec(groupBits) or + isSameOwner and isExec(ownerBits)) return permissions + def listSplit(l, elem, maxSplits=None): splits = [] @@ -783,13 +823,14 @@ except ValueError: break - splits.append( l[:splitOffset] ) - l = l[splitOffset+1:] - splitCount +=1 + splits.append(l[:splitOffset]) + l = l[splitOffset + 1:] + splitCount += 1 if maxSplits is not None and splitCount >= maxSplits: break return splits + [l] + def listJoin(elem, *lists): if lists == []: @@ -799,9 +840,11 @@ l += [elem] + i return l + def closeOnExec(fd): old = fcntl.fcntl(fd, fcntl.F_GETFD, 0) fcntl.fcntl(fd, fcntl.F_SETFD, old | fcntl.FD_CLOEXEC) + class memoized(object): """Decorator that caches a function's return value each time it is called. @@ -815,6 +858,7 @@ self.func = func self.cache = {} functools.update_wrapper(self, func) + def __call__(self, *args): try: return self.cache[args] @@ -822,14 +866,17 @@ value = self.func(*args) self.cache[args] = value return value + def __get__(self, obj, objtype): """Support instance methods.""" return functools.partial(self.__call__, obj) + def validateMinimalKeySet(dictionary, reqParams): if not all(key in dictionary for key in reqParams): raise ValueError + class CommandPath(object): def __init__(self, name, *args): self.name = name -- To view, visit http://gerrit.ovirt.org/9092 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I514660d73071f8fb417fea0cc54973a55002992f Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

4 4

Change in vdsm[master]: BZ#836161 - Refactored deleteImage.
by ewarszaw＠redhat.com 07 Nov '12

07 Nov '12

Eduardo has uploaded a new change for review. Change subject: BZ#836161 - Refactored deleteImage. ...................................................................... BZ#836161 - Refactored deleteImage. Change-Id: I304ff5cd70186ffc9789cd1ac9337efa6c5ff695 Signed-off-by: Eduardo <ewarszaw(a)redhat.com> --- M vdsm/storage/blockSD.py M vdsm/storage/fileSD.py M vdsm/storage/hsm.py M vdsm/storage/sd.py 4 files changed, 142 insertions(+), 14 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/06/8506/1 diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index e596fb6..d26e53c 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -163,13 +163,88 @@ res[vName]['parent'] = vPar if vImg not in res[vName]['imgs']: res[vName]['imgs'].insert(0, vImg) - if vPar != sd.BLANK_UUID and \ - not vName.startswith(blockVolume.image.REMOVED_IMAGE_PREFIX) \ + if vPar != sd.BLANK_UUID \ + and not vName.startswith(sd.REMOVED_IMAGE_PREFIX) \ + and not vName.startswith(sd.ZERO_ME_IMAGE_PREFIX) \ and vImg not in res[vPar]['imgs']: res[vPar]['imgs'].append(vImg) return dict((k, sd.ImgsPar(tuple(v['imgs']), v['parent'])) for k, v in res.iteritems()) + + +def deleteVolumes(sdUUID, vols): + lvm.removeLVs(sdUUID, vols) + + +def _zeroVolume(sdUUID, volUUID): + """ + This function requires an active LV. + """ + # Change for zero 128 M chuncks and log. + # 128 M is the vdsm extent size default + cmd = tuple(constants.EXT_DD, "if=/dev/zero", + "of=%s" % os.path.join("/dev", sdUUID, volUUID)) + p = misc.execCmd(cmd, sudo=False, sync=False) + return p + + +def _getZerosFinished(zeroPs): + for volUUID, zeroProc in zeroPs.items(): + finished = {} + if not zeroProc.wait(0): + continue + else: + zeroPs.pop(volUUID) + finished[volUUID] = zeroProc.returncode + if zeroProc.returncode != 0: + log.warning("zeroing %s failed. %s", volUUID, + zeroProc.stderr) + else: + log.debug("zeroing %s finished", volUUID) + return finished + + +def zeroImgVolumes(sdUUID, imgUUID, volUUIDs): + # Put a sensible value for dd zeroing a 128 M or 1 G chunk and lvremove + # spent time. + ZERO_SLEEP = 60 # [seconds] + # Prepare for zeroing + try: + lvm.changelv(sdUUID, volUUIDs, (("-a", "y"), ("--deltag", imgUUID), + ("--addtag", sd.ZERO_ME_IMAGE_PREFIX + imgUUID))) + except se.StorageException as e: + log.debug("SD %s, Image %s pre zeroing ops failed", sdUUID, imgUUID, + volUUIDs) + try: + lvm.changelv(sdUUID, volUUIDs, ("--permission", "rw")) + except se.StorageException as e: + # Hope this only means that some volumes were already writable. + log.debug("IGNORED: %s", e) + # blank the volumes. + zeroPs = {} + for volUUID in volUUIDs: + zeroPs[volUUID] = _zeroVolume(sdUUID, volUUID) + + # Yes, this is a potentially infinite loop. Kill the vdsm task. + last = time.time() + while len(zeroPs) > 0: + time.sleep(ZERO_SLEEP) + toDelete = [] + for volUUID, rc in _getZerosFinished(zeroPs): + if rc != 0: + log.warning("zero leftover: %s/%s rc = %s", sdUUID, volUUID, rc) + else: + toDelete.append(volUUID) + if toDelete: + deleteVolumes(sdUUID, toDelete) + + now = time.time() + if now - (last + 10 * ZERO_SLEEP) > 0: + log.warning("Still zeroing VG:%s LVs: %s", + sdUUID, zeroPs.iterkeys()) + + log.debug("All rm_zeroed VG:%s LVs: %s, img: %s", sdUUID, volUUIDs, imgUUID) class VGTagMetadataRW(object): @@ -827,6 +902,12 @@ if i.startswith(blockVolume.TAG_PREFIX_IMAGE) ] return images + def deleteImage(self, sdUUID, imgUUID, volsImgs): + return deleteVolumes(sdUUID, volsImgs) + + def zeroImage(self, sdUUID, imgUUID, volsImgs): + return zeroImgVolumes(sdUUID, imgUUID, volsImgs) + def getAllVolumes(self): """ Return all the images that depend on a volume. diff --git a/vdsm/storage/fileSD.py b/vdsm/storage/fileSD.py index 054fadb..d5b298f 100644 --- a/vdsm/storage/fileSD.py +++ b/vdsm/storage/fileSD.py @@ -62,6 +62,21 @@ return True +def getDomPath(sdUUID): + # /rhev/data-center/mnt/export-path/sdUUID/dom_md/metadata + # /rhev/data-center/mnt/blockSD/sdUUID/dom_md/metadata + pattern = os.path.join(sd.mountBasePath, '*', sdUUID) + # Warning! You need a global proc pool big as the number of NFS domains. + domPaths = getProcPool.glob.glob(pattern) + if len(domPaths) != 1: + raise se.StorageDomainLayoutError("domain", sdUUID) + return domPaths[0] + + +def getImagePath(sdUUID, imgUUID): + return os.path.join(getDomPath(sdUUID), 'images', imgUUID) + + def getDomUuidFromMetafilePath(metafile): # Metafile path has pattern: # /rhev/data-center/mnt/export-path/sdUUID/dom_md/metadata @@ -298,6 +313,31 @@ imgList.append(os.path.basename(i)) return imgList + def deleteImage(self, sdUUID, imgUUID, volsImgs): + oPath = getImagePath(sdUUID, imgUUID) + head, tail = self.oop.os.path.split(oPath) + nPath = os.tempnam(head, sd.REMOVED_IMAGE_PREFIX + tail) + try: + self.oop.os.rename(oPath, nPath) + except OSError as e: + self.log.error("image: %s can't be moved", oPath) + raise se.ImageDeleteError("%s %s" % (imgUUID, str(e))) + for volUUID in volsImgs: + volPath = os.path.join(nPath, volUUID) + try: + self.oop.os.remove(volPath) + self.oop.os.remove(volPath + '.meta') + except OSError as e: + self.log.error("vol: %s can't be removed.", volPath) + try: + self.oop.os.remove(nPath) + except OSError as e: + self.log.error("removed image dir: %s can't be removed", nPath) + raise se.ImageDeleteError("%s %s" % (imgUUID, str(e))) + + def zeroImage(self, sdUUID, imgUUID, volsImgs): + return self.deleteImage(sdUUID, imgUUID, volsImgs) + def getAllVolumes(self): """ Return dict {volUUID: ((imgUUIDs,), parentUUID)} of the domain. diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 78eef77..2a8bc59 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -1260,24 +1260,26 @@ Delete Image folder with all volumes """ #vars.task.setDefaultException(se.ChangeMeError("%s" % args)) - pool = self.getPool(spUUID) #Validates that the pool is connected. WHY? - self.validateSdUUID(sdUUID) + self.getPool(spUUID) #Validates that the pool is connected. WHY? + dom = self.validateSdUUID(sdUUID) vars.task.getExclusiveLock(STORAGE, imgUUID) vars.task.getSharedLock(STORAGE, sdUUID) + allVols = dom.getAllVolumes() + imgsByVol = sd.getVolsOfImage(allVols, imgUUID) # Do not validate if forced. - if not misc.parseBool(force): - pool.validateDelete(sdUUID, imgUUID) - # Rename image if postZero and perform delete as async operation - # else delete image in sync. stage + if not misc.parseBool(force) and not dom.isBackup() \ + and not all(len(v.imgs) == 1 for v in imgsByVol.itervalues()): + msg = "Cannot delete shared image %s. volImgs: %s" \ + % (imgUUID, imgsByVol) + raise se.CannotDeleteSharedVolume(msg) + + # zeroImage will delete zeroed volumes at the end. if misc.parseBool(postZero): - newImgUUID = pool.preDeleteRename(sdUUID, imgUUID) - self._spmSchedule(spUUID, "deleteImage", pool.deleteImage, sdUUID, newImgUUID, - misc.parseBool(postZero), misc.parseBool(force) - ) + self._spmSchedule(spUUID, "zeroImage_%s" % imgUUID, dom.zeroImage, + sdUUID, imgUUID, imgsByVol.keys()) else: - pool.deleteImage(sdUUID, imgUUID, - misc.parseBool(postZero), misc.parseBool(force)) + dom.deleteImage(sdUUID, imgUUID, imgsByVol) # This is a hack to keep the interface consistent # We currently have race conditions in delete image, to quickly fix # this we delete images in the "synchronous" state. This only works diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index 55c5b12..fcb796c 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -134,12 +134,17 @@ ImgsPar = namedtuple("ImgsPar", "imgs,parent") ISO_IMAGE_UUID = '11111111-1111-1111-1111-111111111111' BLANK_UUID = '00000000-0000-0000-0000-000000000000' +REMOVED_IMAGE_PREFIX = "_remove_me_" +ZERO_ME_IMAGE_PREFIX = "_zero_me_" # Blocks used for each lease (valid on all domain types) LEASE_BLOCKS = 2048 UNICODE_MINIMAL_VERSION = 3 +storage_repository = config.get('irs', 'repository') +mountBasePath = os.path.join(storage_repository, DOMAIN_MNT_POINT) + def getVolsOfImage(allVols, imgUUID): """ Filter allVols dict for volumes related to imgUUID. -- To view, visit http://gerrit.ovirt.org/8506 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I304ff5cd70186ffc9789cd1ac9337efa6c5ff695 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>

8 37

Change in vdsm[master]: Remove rebundant plus between two strings in parenthesis
by shaohef＠linux.vnet.ibm.com 06 Nov '12

06 Nov '12

ShaoHe Feng has uploaded a new change for review. Change subject: Remove rebundant plus between two strings in parenthesis ...................................................................... Remove rebundant plus between two strings in parenthesis Python will join them automatically without plus Change-Id: Ie24a04bb2002a84179b9b281d0a5fa25c333bf10 Signed-off-by: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> --- M tests/miscTests.py M tests/persistentDictTests.py M tests/resourceManagerTests.py M vdsm/API.py M vdsm/storage/safelease.py M vdsm_reg/register-to-engine.py 6 files changed, 20 insertions(+), 20 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/12/8812/1 diff --git a/tests/miscTests.py b/tests/miscTests.py index 6339986..19ae5f8 100644 --- a/tests/miscTests.py +++ b/tests/miscTests.py @@ -182,7 +182,7 @@ self.assertEquals(list(misc.tmap(dummy, data)), data) def testErrMethod(self): - exceptionStr = ("It's time to kick ass and chew bubble gum... " + + exceptionStr = ("It's time to kick ass and chew bubble gum... " "and I'm all outta gum.") def dummy(arg): @@ -221,8 +221,8 @@ """ #Prepare prefix = "prefix" - stubContent = ('"Multiple exclamation marks", ' + - 'he went on, shaking his head, ' + + stubContent = ('"Multiple exclamation marks", ' + 'he went on, shaking his head, ' '"are a sure sign of a diseased mind."') # (C) Terry Pratchet - Small Gods dir = tempfile.mkdtemp() @@ -801,8 +801,8 @@ Test if method works when given a valid file. """ #create - writeData = ("Trust me, I know what self-loathing is," + - "but to kill myself? That would put a damper on my " + + writeData = ("Trust me, I know what self-loathing is," + "but to kill myself? That would put a damper on my " "search for answers. Not at all productive.") # (C) Jhonen Vasquez - Johnny the Homicidal Maniac fd, path = tempfile.mkstemp() diff --git a/tests/persistentDictTests.py b/tests/persistentDictTests.py index fe7c391..c223d67 100644 --- a/tests/persistentDictTests.py +++ b/tests/persistentDictTests.py @@ -24,7 +24,7 @@ class DummyFailWriter(object): def writelines(self, lines): - raise RuntimeError("You might have a very minor case of " + + raise RuntimeError("You might have a very minor case of " "serious brain damage") # (C) Valve - Portal 2 diff --git a/tests/resourceManagerTests.py b/tests/resourceManagerTests.py index 84708cf..7575bf4 100644 --- a/tests/resourceManagerTests.py +++ b/tests/resourceManagerTests.py @@ -197,7 +197,7 @@ except AttributeError: return except Exception as ex: - self.fail("Wrong exception was raised. " + + self.fail("Wrong exception was raised. " "Expected AttributeError got %s", ex.__class__.__name__) @@ -217,7 +217,7 @@ except AttributeError: return except Exception as ex: - self.fail("Wrong exception was raised. " + + self.fail("Wrong exception was raised. " "Expected AttributeError got %s", ex.__class__.__name__) finally: req.wait() diff --git a/vdsm/API.py b/vdsm/API.py index aa256a1..08dfafa 100644 --- a/vdsm/API.py +++ b/vdsm/API.py @@ -176,7 +176,7 @@ self.log.error('Missing required parameter %s' % (param)) return {'status': {'code': errCode['MissParam'] ['status']['code'], - 'message': 'Missing required ' + + 'message': 'Missing required ' 'parameter %s' % (param)}} try: storage.misc.validateUUID(vmParams['vmId']) @@ -359,7 +359,7 @@ except ValueError: self.log.error('Missing one of required parameters: vmId, nic') return {'status': {'code': errCode['MissParam']['status']['code'], - 'message': 'Missing one of required ' + + 'message': 'Missing one of required ' 'parameters: vmId, nic'}} try: curVm = self._cif.vmContainer[self._UUID] @@ -375,7 +375,7 @@ except ValueError: self.log.error('Missing one of required parameters: vmId, nic') return {'status': {'code': errCode['MissParam']['status']['code'], - 'message': 'Missing one of required ' + + 'message': 'Missing one of required ' 'parameters: vmId, nic'}} try: curVm = self._cif.vmContainer[self._UUID] @@ -391,7 +391,7 @@ except ValueError: self.log.error('Missing one of required parameters: vmId, drive') return {'status': {'code': errCode['MissParam']['status']['code'], - 'message': 'Missing one of required ' + + 'message': 'Missing one of required ' 'parameters: vmId, drive'}} try: curVm = self._cif.vmContainer[self._UUID] @@ -407,7 +407,7 @@ except ValueError: self.log.error('Missing one of required parameters: vmId, drive') return {'status': {'code': errCode['MissParam']['status']['code'], - 'message': 'Missing one of required ' + + 'message': 'Missing one of required ' 'parameters: vmId, drive'}} try: curVm = self._cif.vmContainer[self._UUID] @@ -1019,7 +1019,7 @@ cleantext += line return cleantext - self.log.debug('fenceNode(addr=%s,port=%s,agent=%s,user=%s,' + + self.log.debug('fenceNode(addr=%s,port=%s,agent=%s,user=%s,' 'passwd=%s,action=%s,secure=%s,options=%s)', addr, port, agent, username, 'XXXX', action, secure, options) @@ -1037,7 +1037,7 @@ return errCode['fenceAgent'] raise - inp = ('agent=fence_%s\nipaddr=%s\nlogin=%s\noption=%s\n' + + inp = ('agent=fence_%s\nipaddr=%s\nlogin=%s\noption=%s\n' 'passwd=%s\n') % (agent, addr, username, action, password) if port != '': inp += 'port=%s\n' % (port,) @@ -1120,7 +1120,7 @@ Doesn't survive a restart """ - logging.getLogger('clientIF.setLogLevel').info('Setting loglevel ' + + logging.getLogger('clientIF.setLogLevel').info('Setting loglevel ' 'to %s' % level) handlers = logging.getLogger().handlers [fileHandler] = [h for h in handlers if @@ -1205,7 +1205,7 @@ if vlan or bond or nics: # Backwards compatibility - self.log.warn('Specifying vlan, ' + + self.log.warn('Specifying vlan, ' 'bond or nics to delNetwork is deprecated') _netinfo = netinfo.NetInfo() try: @@ -1215,7 +1215,7 @@ configNetwork.validateVlanId(vlan) if nics and bond and set(nics) != \ set(_netinfo.bondings[bond]["slaves"]): - self.log.error('delNetwork: not all nics specified ' + + self.log.error('delNetwork: not all nics specified ' 'are enslaved (%s != %s)' % (nics, _netinfo.bondings[bond]["slaves"])) diff --git a/vdsm/storage/safelease.py b/vdsm/storage/safelease.py index 551dda5..88a4eae 100644 --- a/vdsm/storage/safelease.py +++ b/vdsm/storage/safelease.py @@ -120,7 +120,7 @@ (rc, out, err) = misc.execCmd(releaseLockCommand, sudo=False, cwd=self.lockUtilPath) if rc != 0: - self.log.error("Could not release cluster lock " + + self.log.error("Could not release cluster lock " "rc=%s out=%s, err=%s" % (str(rc), out, err)) self.log.debug("Cluster lock released successfully") diff --git a/vdsm_reg/register-to-engine.py b/vdsm_reg/register-to-engine.py index d397c84..6c6cc45 100755 --- a/vdsm_reg/register-to-engine.py +++ b/vdsm_reg/register-to-engine.py @@ -112,7 +112,7 @@ try: port = config.get('vars', 'vdc_host_port') except ConfigParser.NoOptionError: - sys.stderr.write("Failed to retrieve port number " + + sys.stderr.write("Failed to retrieve port number " "from config file: %s\n" % VDSM_REG_CONF_FILE) sys.exit(CONF_FILE_READ_ERROR) -- To view, visit http://gerrit.ovirt.org/8812 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie24a04bb2002a84179b9b281d0a5fa25c333bf10 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com>

6 11

Change in vdsm[master]: Don't forget to close thread pool when stopping the SPM mail...
by smizrahi＠redhat.com 06 Nov '12

06 Nov '12

Saggi Mizrahi has uploaded a new change for review. Change subject: Don't forget to close thread pool when stopping the SPM mailer ...................................................................... Don't forget to close thread pool when stopping the SPM mailer Change-Id: I0fd696a20254380e0a0067dd33c7709445e38afb Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M vdsm/storage/storage_mailbox.py 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/80/8980/1 diff --git a/vdsm/storage/storage_mailbox.py b/vdsm/storage/storage_mailbox.py index c537883..041e872 100644 --- a/vdsm/storage/storage_mailbox.py +++ b/vdsm/storage/storage_mailbox.py @@ -687,5 +687,6 @@ time.sleep(self._monitorInterval) finally: self._stopped = True + self._mailman.tp.joinAll(waitForTasks=False) self.log.info("SPM_MailMonitor - Incoming mail monitoring thread stopped") -- To view, visit http://gerrit.ovirt.org/8980 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0fd696a20254380e0a0067dd33c7709445e38afb Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

5 10

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches November 2012