Assaf Muller has uploaded a new change for review.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Extend setupNetworks API to accept defaultRoute
Previously defaultRoute was True for the management network, now it is received from the client.
Change-Id: I9b2ff711155eb0bd79ab84e979eb82c846362374 Bug-Url: https://bugzilla.redhat.com/1015009 Signed-off-by: Assaf Muller amuller@redhat.com --- M lib/vdsm/tool/unified_persistence.py M vdsm/configNetwork.py M vdsm_api/vdsmapi-schema.json 3 files changed, 31 insertions(+), 12 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/20/22720/1
diff --git a/lib/vdsm/tool/unified_persistence.py b/lib/vdsm/tool/unified_persistence.py index d1b3d0a..b412915 100644 --- a/lib/vdsm/tool/unified_persistence.py +++ b/lib/vdsm/tool/unified_persistence.py @@ -37,14 +37,26 @@ _persist(networks, bondings)
-def _toIfcfgFormat(value): - filter = {'on': 'yes', 'off': 'no'} - # If value is in the dict and hashable - Get its filtered value. - # Otherwise: Return the original value +def _ifcfgFilter(filter, value): + """ + If value is in the dict and hashable - Get its filtered value. + Otherwise: Return the original value + :param filter: Dictionary of 'from' -> 'to' + :param value: The value to filter + :return: Filtered value + """ try: return filter.get(value, value) except TypeError: return value + + +def _toIfcfgFormat(value): + return _ifcfgFilter({'on': 'yes', 'off': 'no'}, value) + + +def _fromIfcfgFormat(value): + return _ifcfgFilter({'yes': True, 'no': False}, value)
def _getNetInfo(): @@ -72,7 +84,8 @@ netParams['bridged'] else physicalDevice
# Copy ip addressing information - bootproto = str(getIfaceCfg(topLevelDevice).get('BOOTPROTO')) + ifcfg = getIfaceCfg(topLevelDevice) + bootproto = str(ifcfg.get('BOOTPROTO')) if bootproto == 'dhcp': networks[network]['bootproto'] = bootproto else: @@ -83,6 +96,9 @@ if netParams['gateway'] != '': networks[network]['gateway'] = netParams['gateway']
+ networks[network]['defaultRoute'] = _fromIfcfgFormat( + ifcfg.get('DEFROUTE', False)) + # What if the 'physical device' is actually a VLAN? if physicalDevice in netinfo.vlans: vlanDevice = physicalDevice diff --git a/vdsm/configNetwork.py b/vdsm/configNetwork.py index 5db2a41..e928740 100755 --- a/vdsm/configNetwork.py +++ b/vdsm/configNetwork.py @@ -210,7 +210,8 @@ netmask=None, prefix=None, mtu=None, gateway=None, ipv6addr=None, ipv6gateway=None, force=False, configurator=None, bondingOptions=None, bridged=True, - _netinfo=None, qosInbound=None, qosOutbound=None, **options): + _netinfo=None, qosInbound=None, qosOutbound=None, + defaultRoute=False, **options): nics = nics or () if _netinfo is None: _netinfo = netinfo.NetInfo() @@ -245,16 +246,14 @@ bridged)
logging.info("Adding network %s with vlan=%s, bonding=%s, nics=%s," - " bondingOptions=%s, mtu=%s, bridged=%s, options=%s", - network, vlan, bonding, nics, bondingOptions, - mtu, bridged, options) + " bondingOptions=%s, mtu=%s, bridged=%s, defaultRoute=%s," + "options=%s", network, vlan, bonding, nics, bondingOptions, + mtu, bridged, defaultRoute, options)
if configurator is None: configurator = Ifcfg()
bootproto = options.pop('bootproto', None) - - defaultRoute = network == constants.MANAGEMENT_NETWORK
netEnt = objectivizeNetwork(network if bridged else None, vlan, bonding, bondingOptions, nics, mtu, ipaddr, netmask, @@ -527,6 +526,7 @@ ipv6gateway="<ipv6>" ipv6autoconf="0|1" dhcpv6="0|1" + defaultRoute=True|False (other options will be passed to the config file AS-IS) -- OR -- remove=True (other attributes can't be specified) diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index ce9204f..64e5613 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -164,6 +164,9 @@ # # @qosOutbound: #optional BandwidthParams for outgoing traffic. # +# @defaultRoute: #optional boolean - Is this network's gateway the host's +# default gateway? +# # Since: 4.10.0 ## {'type': 'SetupNetworkNetAttributes', @@ -171,7 +174,7 @@ '*netmask': 'str', '*gateway': 'str', '*bootproto': 'str', '*remove': 'bool', '*qosInbound': 'BandwidthParams', - '*qosOutbound': 'BandwidthParams'}} + '*qosOutbound': 'BandwidthParams', '*defaultRoute': 'bool'}} ## # @SetupNetworkBondAttributes: #
Assaf Muller has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 1: Code-Review-1
Do *NOT* merge before equivalent patch is merged to the engine.
oVirt Jenkins CI Server has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 1:
Build Successful
http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/6257/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_network_functional_tests/1095/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit_el/5470/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/6363/ : SUCCESS
Assaf Muller has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 1: Verified+1
Verification: Generated local setupNetworks. Not setting defaultRoute, and setting defaultRoute to True or False has the expected results. Upgrade wise, defaultRoute is taken from the ifcfg file correctly both for true and false values. The generated persistence file can be fed back to setupNetworks successfully.
Dan Kenigsberg has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 1: Code-Review-1
(4 comments)
.................................................... File lib/vdsm/tool/unified_persistence.py Line 71 Line 72 Line 73 Line 74 Line 75 assigning bootproto the string value "None" just because BOOTPROTO was missing is a bug waiting to pop up: with it, bootproto is one of "dhcp", "none", and "None". Please drop the str().
Line 36: UPGRADE_NAME, networks, bondings) Line 37: _persist(networks, bondings) Line 38: Line 39: Line 40: def _ifcfgFilter(filter, value): "filter" is an old Python builtin. This function does not do filtering, but rather conversion of values. Line 41: """ Line 42: If value is in the dict and hashable - Get its filtered value. Line 43: Otherwise: Return the original value Line 44: :param filter: Dictionary of 'from' -> 'to'
Line 54: def _toIfcfgFormat(value): Line 55: return _ifcfgFilter({'on': 'yes', 'off': 'no'}, value) Line 56: Line 57: Line 58: def _fromIfcfgFormat(value): I'd prefer a simpler function, named yesnoToBool - here there's certainly no reason for allowing unexpected values through. Line 59: return _ifcfgFilter({'yes': True, 'no': False}, value) Line 60: Line 61: Line 62: def _getNetInfo():
.................................................... File vdsm/configNetwork.py Line 253 Line 254 Line 255 Line 256 Line 257 We have to maintain this code until ovirt-3.3 is extinct.
Assaf Muller has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 1:
(5 comments)
.................................................... File lib/vdsm/tool/unified_persistence.py Line 71 Line 72 Line 73 Line 74 Line 75 Notice that I only assign into networks[network]['bootproto'] if bootproto == 'dhcp'
Line 36: UPGRADE_NAME, networks, bondings) Line 37: _persist(networks, bondings) Line 38: Line 39: Line 40: def _ifcfgFilter(filter, value): Filter like audio filters you can pass on sound waves... Something that changes the output, not necessarily filtering out values. Anyway, if 'filter' has a very specific connotation in Python I'll change it. Line 41: """ Line 42: If value is in the dict and hashable - Get its filtered value. Line 43: Otherwise: Return the original value Line 44: :param filter: Dictionary of 'from' -> 'to'
Line 54: def _toIfcfgFormat(value): Line 55: return _ifcfgFilter({'on': 'yes', 'off': 'no'}, value) Line 56: Line 57: Line 58: def _fromIfcfgFormat(value): I'll make the function assume that it can only expect 'yes' and 'no', but I find 'fromIfcfgFormat' vastly more expressive and clear than 'yesnoToBool'. It explains why the line of code exists, and not what it does. Line 59: return _ifcfgFilter({'yes': True, 'no': False}, value) Line 60: Line 61: Line 62: def _getNetInfo():
Line 95: networks[network]['netmask'] = netParams['netmask'] Line 96: if netParams['gateway'] != '': Line 97: networks[network]['gateway'] = netParams['gateway'] Line 98: Line 99: networks[network]['defaultRoute'] = _fromIfcfgFormat( I just noticed that this bit is wrong...
What we really want is to persist defaultRoute=True only for the management network (IE: Emulate proper engine behavior). Line 100: ifcfg.get('DEFROUTE', False)) Line 101: Line 102: # What if the 'physical device' is actually a VLAN? Line 103: if physicalDevice in netinfo.vlans:
.................................................... File vdsm/configNetwork.py Line 253 Line 254 Line 255 Line 256 Line 257 Done
Assaf Muller has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 2:
(2 comments)
.................................................... File lib/vdsm/tool/unified_persistence.py Line 97: if netParams['gateway'] != '': Line 98: networks[network]['gateway'] = netParams['gateway'] Line 99: Line 100: networks[network]['defaultRoute'] = \ Line 101: str(network in MANAGEMENT_NETWORKS) str conversion is needed because for whatever reason when serializing to json, True is converted to true. true cannot be sent to setupNetworks (True, 'true', 'True' are all good. true is not). Line 102: Line 103: # What if the 'physical device' is actually a VLAN? Line 104: if physicalDevice in netinfo.vlans: Line 105: vlanDevice = physicalDevice
.................................................... File vdsm/configNetwork.py Line 49: Line 50: def objectivizeNetwork(bridge=None, vlan=None, bonding=None, Line 51: bondingOptions=None, nics=None, mtu=None, ipaddr=None, Line 52: netmask=None, gateway=None, bootproto=None, Line 53: ipv6addr=None, ipv6gateway=None, ipv6autoconf=None, If setupNetworks gets an explicit defaultRoute, then with patch set 1 defaultRoute is sent as a keyword argument and in **opts as well. Line 54: dhcpv6=None, defaultRoute=None, _netinfo=None, Line 55: configurator=None, blockingdhcp=None, Line 56: implicitBonding=None, **opts): Line 57: """
oVirt Jenkins CI Server has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 2: Verified-1
Build Failed
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/6694/ : FAILURE
http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/6607/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit_el/5801/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_network_functional_tests/1140/ : FAILURE
Dan Kenigsberg has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 2:
(2 comments)
.................................................... File lib/vdsm/tool/unified_persistence.py Line 29: Line 30: UPGRADE_NAME = 'UnifiedPersistence' Line 31: Line 32: Line 33: # TODO: Upgrade currently gets bootproto from ifcfg files, I do not understand this TODO and certainty not why it's part of this patch.
Iproute2 and unified persistence are not get feature complete, and dhcp is one of their shortcomings. True. But why is this related to this file, which is reposible to upgrade from ifcfg? Line 34: # as we assume we're upgrading from oVirt <= 3.4, where users still used Line 35: # ifcfg files. Once we start dealing with new installations on OS that don't Line 36: # use ifcfg files, we need to stop getting information from ifcfg files. Line 37: # bootproto = 'dhcp' if there's a lease on the NIC at the moment of upgrade
Line 97: if netParams['gateway'] != '': Line 98: networks[network]['gateway'] = netParams['gateway'] Line 99: Line 100: networks[network]['defaultRoute'] = \ Line 101: str(network in MANAGEMENT_NETWORKS) I do not follow. If you serialize a Boolean into json, it does not recover its type when loaded? Line 102: Line 103: # What if the 'physical device' is actually a VLAN? Line 104: if physicalDevice in netinfo.vlans: Line 105: vlanDevice = physicalDevice
Assaf Muller has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 2:
(2 comments)
http://gerrit.ovirt.org/#/c/22720/2/lib/vdsm/tool/unified_persistence.py File lib/vdsm/tool/unified_persistence.py:
Line 29: Line 30: UPGRADE_NAME = 'UnifiedPersistence' Line 31: Line 32: Line 33: # TODO: Upgrade currently gets bootproto from ifcfg files,
I do not understand this TODO and certainty not why it's part of this patch
It has nothing to do with this patch, you're right. I got carried away :)
More importantly though - This upgrade isn't an upgrade from ifcfg files. It's an upgrade that builds the initial running conf for old and new hosts alike. Take a future oVirt 4.0 host - We need to build its initial running and persistent conf because otherwise it just won't exist. Meaning, we won't be able to boot from the persistent conf. Line 34: # as we assume we're upgrading from oVirt <= 3.4, where users still used Line 35: # ifcfg files. Once we start dealing with new installations on OS that don't Line 36: # use ifcfg files, we need to stop getting information from ifcfg files. Line 37: # bootproto = 'dhcp' if there's a lease on the NIC at the moment of upgrade
Line 97: if netParams['gateway'] != '': Line 98: networks[network]['gateway'] = netParams['gateway'] Line 99: Line 100: networks[network]['defaultRoute'] = \ Line 101: str(network in MANAGEMENT_NETWORKS)
I do not follow. If you serialize a Boolean into json, it does not recover
If you serialize True to json, it is for whatever reason converted to true (Which is different from 'true'). When I copy/paste that into setupNetworks, Python complains that true isn't defined. (As it isn't 'true', 'True', or True).
On another note - It's incorrect to check if the network is the management network... I'll just get the default route from the main routing table and check if the device fits. Line 102: Line 103: # What if the 'physical device' is actually a VLAN? Line 104: if physicalDevice in netinfo.vlans: Line 105: vlanDevice = physicalDevice
Dan Kenigsberg has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 2:
(2 comments)
http://gerrit.ovirt.org/#/c/22720/2/lib/vdsm/tool/unified_persistence.py File lib/vdsm/tool/unified_persistence.py:
Line 29: Line 30: UPGRADE_NAME = 'UnifiedPersistence' Line 31: Line 32: Line 33: # TODO: Upgrade currently gets bootproto from ifcfg files,
It has nothing to do with this patch, you're right. I got carried away :)
Confusing... An "upgrade" that is required to run on a clean install, too. I suppose that a doc about this should appear somewhere. Line 34: # as we assume we're upgrading from oVirt <= 3.4, where users still used Line 35: # ifcfg files. Once we start dealing with new installations on OS that don't Line 36: # use ifcfg files, we need to stop getting information from ifcfg files. Line 37: # bootproto = 'dhcp' if there's a lease on the NIC at the moment of upgrade
Line 97: if netParams['gateway'] != '': Line 98: networks[network]['gateway'] = netParams['gateway'] Line 99: Line 100: networks[network]['defaultRoute'] = \ Line 101: str(network in MANAGEMENT_NETWORKS)
If you serialize True to json, it is for whatever reason converted to true
Oh dear, I see that
json.loads(json.dumps({True: False})) == {u'True': False}
Note that when it comes to values, not keys, Booleans are conserved. It is not clear to me why this extra level of serialization needed here.
Nevertheless, I note that this is prevalent in this module. So let us figure this out in a future patch. Line 102: Line 103: # What if the 'physical device' is actually a VLAN? Line 104: if physicalDevice in netinfo.vlans: Line 105: vlanDevice = physicalDevice
Assaf Muller has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 2:
(1 comment)
http://gerrit.ovirt.org/#/c/22720/2/vdsm/configNetwork.py File vdsm/configNetwork.py:
Line 49: Line 50: def objectivizeNetwork(bridge=None, vlan=None, bonding=None, Line 51: bondingOptions=None, nics=None, mtu=None, ipaddr=None, Line 52: netmask=None, gateway=None, bootproto=None, Line 53: ipv6addr=None, ipv6gateway=None, ipv6autoconf=None,
If setupNetworks gets an explicit defaultRoute, then with patch set 1 defau
Just to make this perfectly clear: Since VDSM 3.3, if the engine sends defaultRoute, then objectivize gets defaultRoute twice (And VDSM crashes), once as a named argument, and one as a keyword argument in **opts. Patch set 2 fixes that issue. Line 54: dhcpv6=None, defaultRoute=None, _netinfo=None, Line 55: configurator=None, blockingdhcp=None, Line 56: implicitBonding=None, **opts): Line 57: """
Dan Kenigsberg has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 2: Code-Review-1
(1 comment)
http://gerrit.ovirt.org/#/c/22720/2/vdsm/configNetwork.py File vdsm/configNetwork.py:
Line 249: # the management network. Line 250: # TODO: When oVirt 3.3 is deprecated, change the default to False and Line 251: # remove reference to constants.MANAGEMENT_NETWORKS Line 252: if defaultRoute is None: Line 253: defaultRoute = network in constants.MANAGEMENT_NETWORKS needs renaming to LEGACY_MANAGEMENT_NETWORKS Line 254: Line 255: logging.info("Adding network %s with vlan=%s, bonding=%s, nics=%s," Line 256: " bondingOptions=%s, mtu=%s, bridged=%s, defaultRoute=%s," Line 257: "options=%s", network, vlan, bonding, nics, bondingOptions,
Assaf Muller has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 2:
(1 comment)
http://gerrit.ovirt.org/#/c/22720/2/vdsm/configNetwork.py File vdsm/configNetwork.py:
Line 249: # the management network. Line 250: # TODO: When oVirt 3.3 is deprecated, change the default to False and Line 251: # remove reference to constants.MANAGEMENT_NETWORKS Line 252: if defaultRoute is None: Line 253: defaultRoute = network in constants.MANAGEMENT_NETWORKS
needs renaming to LEGACY_MANAGEMENT_NETWORKS
Oh I didn't rebase yet. Line 254: Line 255: logging.info("Adding network %s with vlan=%s, bonding=%s, nics=%s," Line 256: " bondingOptions=%s, mtu=%s, bridged=%s, defaultRoute=%s," Line 257: "options=%s", network, vlan, bonding, nics, bondingOptions,
Assaf Muller has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 3: Verified+1
Dan Kenigsberg has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 3: Code-Review+1
Assaf Muller has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 4: Verified+1
Dan Kenigsberg has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 4: Code-Review+1
oVirt Jenkins CI Server has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 3: Verified-1
Build Failed
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit_el/5904/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_network_functional_tests/1155/ : FAILURE
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/6797/ : FAILURE
http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/6693/ : SUCCESS
oVirt Jenkins CI Server has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 4: Verified-1
Build Failed
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit_el/5923/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_network_functional_tests/1166/ : FAILURE
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/6816/ : FAILURE
http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/6712/ : SUCCESS
Antoni Segura Puimedon has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 4: Code-Review+1
Dan Kenigsberg has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 4: Code-Review+2
Antoni Segura Puimedon has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 5: Code-Review+1
Dan Kenigsberg has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 5: Verified+1 Code-Review+2
Dan Kenigsberg has submitted this change and it was merged.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Extend setupNetworks API to accept defaultRoute
Previously defaultRoute was True for the management network, now it is received from the client. For backwards compatability, if defaultRoute is not received from the client, it is set if the network being configured is the management network.
Change-Id: I9b2ff711155eb0bd79ab84e979eb82c846362374 Bug-Url: https://bugzilla.redhat.com/1015009 Signed-off-by: Assaf Muller amuller@redhat.com Reviewed-on: http://gerrit.ovirt.org/22720 Reviewed-by: Antoni Segura Puimedon asegurap@redhat.com Reviewed-by: Dan Kenigsberg danken@redhat.com Tested-by: Dan Kenigsberg danken@redhat.com --- M lib/vdsm/tool/unified_persistence.py M vdsm/configNetwork.py M vdsm_api/vdsmapi-schema.json 3 files changed, 46 insertions(+), 17 deletions(-)
Approvals: Antoni Segura Puimedon: Looks good to me, but someone else must approve Dan Kenigsberg: Verified; Looks good to me, approved
oVirt Jenkins CI Server has posted comments on this change.
Change subject: Extend setupNetworks API to accept defaultRoute ......................................................................
Patch Set 4:
Build Failed
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit_el/5923/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/6816/ : FAILURE
http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/6712/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_network_functional_tests/1167/ : FAILURE
vdsm-patches@lists.fedorahosted.org