Re: [netcf-devel] [libvirt] 802.1q vlan-tagging support for libvirt

[Klaus Heinrich Kiwi]

On Thu, 2009-04-02 at 16:34 -0700, David Lutterkort wrote:
>  Hi Klaus,
>  
>  On Thu, 2009-04-02 at 15:50 -0300, Klaus Heinrich Kiwi wrote:
>  > My understanding is that libvirt would use vconfig to create tagged
>  > interfaces, while using a physical interface as trunk (e.g., eth0 is the
>  > trunk, eth0.20 the interface with the '20' vlan tag).
>  > 
>  > Then it would add the tagged interface (eth0.20) to a bridge with the
>  > guest virtual interface. 
I was thinking about the semantics I described above. It ultimately
means that we'll have a bridge for each VLAN tag that crosses the trunk
interface. So for example if guests A, B and C are all associated with
VLAN ID 20, then:

eth0 -> eth0.20 -> br0 -> [tap0, tap1, tap2]

(where tap[0-3] are associated with guests A, B, C respectively)

Adding a new guest to a VLAN would mean searching the host system and
check if there is already a bridge running on that VLAN ID. Create a new
one if needed, or use the existing one.

The things that concerns me the most are:
 1) How scalable this really is
 2) The semantics are really different from how physical, 802.1q-enabled
switches would work.

Because (2) really creates new switches for each new VLAN tag, I wonder
how management would be different from what we have today with physical
switches (i.e., defining a port with a VLAN ID, assigning that port to a
physical machine) - unless we hide it behind libvirt somehow.

Still, things like SNMP-management can have issues with such approach
(snmp-based network managers would go crazy identifying several
switches/bridges per box - not really useful from a management PoV).

Are there other options? Since a tagged interface like eth0.20 is kind
of a virtual interface itself, would it be appropriate to use those
directly?

Or maybe extending the existing bridging code to be 802.1q-aware?

 Thanks,

 -Klaus



>  as Dan said, the actual functionality will be provided by netcf[1]
>  
>  VLAN is very high on the list of things that need to be done next - and
>  any help with that would be much appreciated ;)
>  
>  David
>  
>  [1] https://fedorahosted.org/netcf/
>   
-- 
Klaus Heinrich Kiwi <klausk(a)linux.vnet.ibm.com&gt;
Linux Security Development, IBM Linux Technology Center

[libvirt] 802.1q vlan-tagging support for libvirt

[John Levon]

On Tue, Apr 07, 2009 at 06:39:17PM -0300, Klaus Heinrich Kiwi wrote:

>  I was thinking about the semantics I described above. It ultimately
I just caught this discussion. We have implemented basic vlan support
(for xend, bridged only). Is there an existing patch from you guys for
the user interface? Below is our current libvirt patch

regards
john

Support setting of vlan ID for network interfaces

Signed-off-by: John Levon <john.levon(a)sun.com&gt;
Signed-off-by: Max Zhen <max.zhen(a)sun.com&gt;

diff --git a/docs/schemas/domain.rng b/docs/schemas/domain.rng
--- a/docs/schemas/domain.rng
+++ b/docs/schemas/domain.rng
@@ -692,6 +692,13 @@
          </optional>
         </element>
       </optional>
+      <optional>
+       <element name='vlan'>
+        <attribute name='id'>
+         <ref name='unsignedInt'/>
+        </attribute>
+       </element>
+      </optional>
     </interleave>
   </define>
 
diff --git a/src/domain_conf.c b/src/domain_conf.c
--- a/src/domain_conf.c
+++ b/src/domain_conf.c
@@ -918,6 +918,7 @@ virDomainNetDefParseXML(virConnectPtr co
     char *address = NULL;
     char *port = NULL;
     char *model = NULL;
+    char *vlanid = NULL;
 
     if (VIR_ALLOC(def) < 0) {
         virReportOOMError(conn);
@@ -983,6 +984,8 @@ virDomainNetDefParseXML(virConnectPtr co
                 model = virXMLPropString(cur, "type");
             } else if (xmlStrEqual (cur->name, BAD_CAST "networkresource"))
{
                 virDomainNetDefParseXMLRate(def, cur);
+            } else if (xmlStrEqual (cur->name, BAD_CAST "vlan")) {
+                vlanid = virXMLPropString(cur, "id");
             }
         }
         cur = cur->next;
@@ -1093,6 +1096,16 @@ virDomainNetDefParseXML(virConnectPtr co
         model = NULL;
     }
 
+    def->vlanid = 0;
+
+    if (vlanid != NULL) {
+        if (virStrToLong_i(vlanid, NULL, 10, &def->vlanid) < 0) {
+            virDomainReportError(conn, VIR_ERR_INTERNAL_ERROR, "%s",
+                                 _("Cannot parse vlan ID"));
+            goto error;
+        }
+    }
+
 cleanup:
     VIR_FREE(macaddr);
     VIR_FREE(network);
@@ -1104,6 +1117,7 @@ cleanup:
     VIR_FREE(bridge);
     VIR_FREE(model);
     VIR_FREE(type);
+    VIR_FREE(vlanid);
 
     return def;
 
@@ -3035,6 +3049,9 @@ virDomainNetDefFormat(virConnectPtr conn
                           unit, period, def->rate.value);
     }
 
+    if (def->vlanid)
+        virBufferVSprintf(buf, "      <vlan id='%d' />\n",
def->vlanid);
+
     virBufferAddLit(buf, "    </interface>\n");
 
     return 0;
diff --git a/src/domain_conf.h b/src/domain_conf.h
--- a/src/domain_conf.h
+++ b/src/domain_conf.h
@@ -186,6 +186,7 @@ struct _virDomainNetDef {
         int period;
         long value;
     } rate;
+    int vlanid;
 };
 
 enum virDomainChrSrcType {
diff --git a/src/virsh.c b/src/virsh.c
--- a/src/virsh.c
+++ b/src/virsh.c
@@ -4762,6 +4762,7 @@ static const vshCmdOptDef opts_attach_in
     {"mac",    VSH_OT_DATA, 0, gettext_noop("MAC address")},
     {"script", VSH_OT_DATA, 0, gettext_noop("script used to bridge network
interface")},
     {"capped-bandwidth", VSH_OT_STRING, 0, gettext_noop("bandwidth limit
for this interface")},
+    {"vlanid", VSH_OT_INT, 0, gettext_noop("VLAN ID attached to this
interface")},
     {NULL, 0, 0, NULL}
 };
 
@@ -4769,7 +4770,7 @@ cmdAttachInterface(vshControl *ctl, cons
 cmdAttachInterface(vshControl *ctl, const vshCmd *cmd)
 {
     virDomainPtr dom = NULL;
-    char *mac, *target, *script, *type, *source, *bw;
+    char *mac, *target, *script, *type, *source, *bw, *vlanid;
     int typ, ret = FALSE;
     char *buf = NULL, *tmp = NULL;
 
@@ -4787,6 +4788,7 @@ cmdAttachInterface(vshControl *ctl, cons
     mac = vshCommandOptString(cmd, "mac", NULL);
     script = vshCommandOptString(cmd, "script", NULL);
     bw = vshCommandOptString(cmd, "capped-bandwidth", NULL);
+    vlanid = vshCommandOptString(cmd, "vlanid", NULL);
 
     /* check interface type */
     if (STREQ(type, "network")) {
@@ -4866,6 +4868,21 @@ cmdAttachInterface(vshControl *ctl, cons
         tmp = vshRealloc(ctl, tmp, strlen(unit) + strlen(bw) + strlen(format));
         if (!tmp) goto cleanup;
         sprintf(tmp, format, unit, r);
+        buf = vshRealloc(ctl, buf, strlen(buf) + strlen(tmp) + 1);
+        if (!buf) goto cleanup;
+        strcat(buf, tmp);
+    }
+
+    if (vlanid != NULL) {
+        char *left;
+        long r =  strtol(vlanid, &left, 10);
+        if ((r <= 0) || (r >= 4095) || (*left != '\0')) {
+            vshError(ctl, FALSE, _("Bad VLAN ID: %s in command
'attach-interface'"), vlanid);
+            goto cleanup;
+        }
+        tmp = vshRealloc(ctl, tmp, strlen(vlanid) + 20);
+        if (!tmp) goto cleanup;
+        sprintf(tmp, "      <vlan id='%s'/>\n", vlanid);
         buf = vshRealloc(ctl, buf, strlen(buf) + strlen(tmp) + 1);
         if (!buf) goto cleanup;
         strcat(buf, tmp);
diff --git a/src/xend_internal.c b/src/xend_internal.c
--- a/src/xend_internal.c
+++ b/src/xend_internal.c
@@ -1896,7 +1896,10 @@ xend_parse_rate(const char *ratestr, int
     if (sscanf(ratestr, "%lu,%lu", &amount, &interval) == 2) {
         *unit = VIR_DOMAIN_NET_RATE_KB;
         *period = VIR_DOMAIN_NET_PERIOD_S;
-        *val = (amount * 8) / (interval / 1000000);
+	/* bytes to kilobits */
+        *val = (amount / 125);
+	/* factor in period interval */
+	*val /= (interval / 1000000);
         return 0;
     }
 
@@ -2046,6 +2049,15 @@ xenDaemonParseSxprNets(virConnectPtr con
                                  _("ignoring malformed rate limit
'%s'"), tmp);
                 } else {
                     net->rate.enabled = 1;
+                }
+            }
+
+            tmp = sexpr_node(node, "device/vif/vlanid");
+            if (tmp) {
+                if (virStrToLong_i(tmp, NULL, 0, &net->vlanid) != 0) {
+                    virXendError(conn, VIR_ERR_INTERNAL_ERROR,
+                                 _("malformed vlanid '%s'"), tmp);
+                    goto cleanup;
                 }
             }
 
@@ -5553,6 +5565,9 @@ xenDaemonFormatSxprNet(virConnectPtr con
                           unit, period);
     }
 
+    if (def->vlanid)
+        virBufferVSprintf(buf, "(vlanid '%d')", def->vlanid);
+
     /*
      * apparently (type ioemu) breaks paravirt drivers on HVM so skip this
      * from Xen 3.1.0

[libvirt] 802.1q vlan-tagging support for libvirt

[David Lutterkort]

On Tue, 2009-04-07 at 18:39 -0300, Klaus Heinrich Kiwi wrote:
>  I was thinking about the semantics I described above. It ultimately
>  means that we'll have a bridge for each VLAN tag that crosses the trunk
>  interface. So for example if guests A, B and C are all associated with
>  VLAN ID 20, then:
>  
>  eth0 -> eth0.20 -> br0 -> [tap0, tap1, tap2]
>  
>  (where tap[0-3] are associated with guests A, B, C respectively) 
Yes, I think that's how it should work; it would also mean that you'd
first set up eth0 as a separate interface, and new bridge/vlan interface
combos afterwards. AFAIK, for the bridge, only bootproto=none would make
sense.

>  The things that concerns me the most are:
>   1) How scalable this really is 
I don't know either ... we'll find out ;)

>   2) The semantics are really different from how physical,
> 802.1q-enabled
>  switches would work.
>  
>  Because (2) really creates new switches for each new VLAN tag, I wonder
>  how management would be different from what we have today with physical
>  switches (i.e., defining a port with a VLAN ID, assigning that port to a
>  physical machine) - unless we hide it behind libvirt somehow. 
The reason we are creating all those bridges isn't the VLAN's - it's
that we want to share the same physical interface amongst several
guests. And I don't know of another way to do that.

>  Are there other options? Since a tagged interface like eth0.20 is
> kind
>  of a virtual interface itself, would it be appropriate to use those
>  directly? 
You can use it directly, I just don't know how else you would share it
amongst VM's without a bridge.

David

[libvirt] 802.1q vlan-tagging support for libvirt

[Daniel P. Berrange]

On Tue, Apr 07, 2009 at 06:32:43PM -0700, David Lutterkort wrote:
>  On Tue, 2009-04-07 at 18:39 -0300, Klaus Heinrich Kiwi wrote:
>  > I was thinking about the semantics I described above. It ultimately
>  > means that we'll have a bridge for each VLAN tag that crosses the trunk
>  > interface. So for example if guests A, B and C are all associated with
>  > VLAN ID 20, then:
>  > 
>  > eth0 -> eth0.20 -> br0 -> [tap0, tap1, tap2]
>  > 
>  > (where tap[0-3] are associated with guests A, B, C respectively)
>  
>  Yes, I think that's how it should work; it would also mean that you'd
>  first set up eth0 as a separate interface, and new bridge/vlan interface
>  combos afterwards. AFAIK, for the bridge, only bootproto=none would make
>  sense.
>  
>  > The things that concerns me the most are:
>  >  1) How scalable this really is
>  
>  I don't know either ... we'll find out ;) 
I don't think that's really a scalability problem from libvirt's POV. I
know people use this setup quite widely already even with plain ifcfg-XXX
scripts. Any scalability problems nmost likely fall into the kernel /
networking code and whether it is good at avoiding unneccessary data
copies when you have stacked  NIC -> VLAN -> BRIDGE -> TAP

>  
>  >  2) The semantics are really different from how physical, 802.1q-enabled
>  > switches would work.
>  > 
>  > Because (2) really creates new switches for each new VLAN tag, I wonder
>  > how management would be different from what we have today with physical
>  > switches (i.e., defining a port with a VLAN ID, assigning that port to a
>  > physical machine) - unless we hide it behind libvirt somehow. 
I think one thing to consider is the difference between the physical and
logical models.  The libvirt API / representation here is fairly low 
level, dealing in individuals NICs. I think management apps would likely
want to present this in a slightly alternate way dealing more in logical
entities than physical NICs. eg oVirt's network model is closer to the
one you describe where the user defines a new switch for each VLAN tag.
It then maps this into the low level physical model of individual NICs
as needed. I think it si important that libvirt use the physical model
here  to give apps flexibility in how they expose it to users.

>  
>  The reason we are creating all those bridges isn't the VLAN's - it's
>  that we want to share the same physical interface amongst several
>  guests. And I don't know of another way to do that.
>  
>  > Are there other options? Since a tagged interface like eth0.20 is kind
>  > of a virtual interface itself, would it be appropriate to use those
>  > directly?
>  
>  You can use it directly, I just don't know how else you would share it
>  amongst VM's without a bridge. 
In the (nearish) future NICs will start appearing with SR-IOV capabilities.
This gives you one physical PCI device, whcih exposes multiple functions.
So a single physical NIC appears as 8 NICs to the OS. You can thus directly
assign each of these virtual NICs to a different VM directly,a voiding the
need to bridge them.

I don't think its worth spending too much time trying to come up with other
non-bridged NIC sharing setups when hardware is about todo it all for us :-)

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|