Fedora CoreOS Meeting Minutes 2020-09-30
by Dusty Mabe
Minutes: https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-30/fedora_core...
Minutes (text): https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-30/fedora_core...
Log: https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-30/fedora_core...
========================================
#fedora-meeting-1: fedora_coreos_meeting
========================================
Meeting started by dustymabe at 16:31:11 UTC. The full logs are
available at
https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-30/fedora_core...
.
Meeting summary
---------------
* roll call (dustymabe, 16:31:18)
* Action items from last meeting (dustymabe, 16:35:55)
* Move rpmdb path from /usr/share/rpm to /usr/lib/sysimage/rpm
(dustymabe, 16:36:37)
* LINK: https://github.com/coreos/fedora-coreos-tracker/issues/639
(lucab, 16:38:57)
* LINK:
http://lists.rpm.org/pipermail/rpm-maint/2017-October/006681.html
(dustymabe, 16:40:55)
* ACTION: jlebon to reach out to the rpm maintainers to see if the
relocation of the rpmdb path is something their willing to own for
F34 (dustymabe, 16:47:11)
* Add FCOS to AWS Marketplace (dustymabe, 16:47:59)
* LINK: https://github.com/coreos/fedora-coreos-tracker/issues/635
(dustymabe, 16:48:10)
* LINK: https://pagure.io/Fedora-Council/tickets/issue/332
(bgilbert, 16:50:19)
* LINK: https://github.com/coreos/fedora-coreos-tracker/issues/225
(bgilbert, 16:52:09)
* Add basic monitoring tools to the base image (dustymabe, 17:07:04)
* LINK: https://github.com/coreos/fedora-coreos-tracker/issues/628
(dustymabe, 17:07:10)
* LINK: https://src.fedoraproject.org/container/tools (cverna,
17:21:56)
* LINK:
https://src.fedoraproject.org/container/tools/blob/master/f/Dockerfile
(rtsisyk, 17:22:24)
* ACTION: we'll file a tracker ticket to come up with rough criteria
for adding packages (dustymabe, 17:28:12)
* LINK: https://github.com/coreos/fedora-coreos-docs/issues/187
(bgilbert, 17:28:45)
* open floor (dustymabe, 17:31:36)
* next week's stable release will require PXE booting with the rootfs
image if anyone hasn't switched over yet, now is the time :-)
(dustymabe, 17:33:50)
* LINK:
https://docs.fedoraproject.org/en-US/fedora-coreos/bare-metal/#_pxe_rootf...
(bgilbert, 17:34:58)
Meeting ended at 17:37:38 UTC.
Action Items
------------
* jlebon to reach out to the rpm maintainers to see if the relocation of
the rpmdb path is something their willing to own for F34
* we'll file a tracker ticket to come up with rough criteria for adding
packages
Action Items, by person
-----------------------
* jlebon
* jlebon to reach out to the rpm maintainers to see if the relocation
of the rpmdb path is something their willing to own for F34
* **UNASSIGNED**
* we'll file a tracker ticket to come up with rough criteria for
adding packages
People Present (lines said)
---------------------------
* dustymabe (102)
* bgilbert (80)
* rtsisyk (57)
* aoei (44)
* jlebon (30)
* zodbot (21)
* lucab (17)
* cverna (7)
* cyberpear (5)
* nasirhm1 (5)
* lorbus (3)
* misc (1)
* skunkerk (1)
Generated by `MeetBot`_ 0.1.4
.. _`MeetBot`: http://wiki.debian.org/MeetBot
3 years, 5 months
Fedora CoreOS Meeting Minutes 2020-09-23
by Dusty Mabe
Minutes: https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-23/fedora_core...
Minutes (text): https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-23/fedora_core...
Log: https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-23/fedora_core...
========================================
#fedora-meeting-1: fedora_coreos_meeting
========================================
Meeting started by dustymabe at 16:30:40 UTC. The full logs are
available at
https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-23/fedora_core...
.
Meeting summary
---------------
* roll call (dustymabe, 16:30:44)
* Action items from last meeting (dustymabe, 16:35:49)
* jlebon filed
https://github.com/coreos/fedora-coreos-tracker/issues/623 (jlebon,
16:36:22)
* Need dnsmasq for podman to create CNI networks (dustymabe, 16:37:50)
* LINK: https://github.com/coreos/fedora-coreos-tracker/issues/519
(dustymabe, 16:37:55)
* LINK: https://github.com/coreos/fedora-coreos-tracker/issues/186
(cyberpear, 16:54:45)
* LINK: https://github.com/coreos/fedora-coreos-tracker/issues/186
(cyberpear, 16:54:45)
* AGREED: We'll try to get the podman team to break the hard
requirement of podman on podman plugins and continue the discussion
upstream about potentially revisiting CNI plugin design.
(dustymabe, 17:04:24)
* tracker: Fedora 33 rebase work (dustymabe, 17:04:58)
* LINK: https://github.com/coreos/fedora-coreos-tracker/issues/609
(dustymabe, 17:05:03)
* LINK:
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/issues/539
(dustymabe, 17:08:53)
* open floor (dustymabe, 17:11:15)
* the stable stream release 32.20200907.3.0 went out this morning with
recent kernel fixes (dustymabe, 17:12:26)
Meeting ended at 17:31:40 UTC.
Action Items
------------
Action Items, by person
-----------------------
* **UNASSIGNED**
* (none)
People Present (lines said)
---------------------------
* dustymabe (91)
* jlebon (32)
* bgilbert (31)
* cyberpear (20)
* zodbot (17)
* lucab (11)
* walters (8)
* nasirhm (6)
* skunkerk (2)
* darkmuggle (1)
* lorbus (1)
Generated by `MeetBot`_ 0.1.4
.. _`MeetBot`: http://wiki.debian.org/MeetBot
3 years, 5 months
Fedora CoreOS Meeting Minutes 2020-09-16
by Dusty Mabe
Minutes: https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-16/fedora_core...
Minutes (text): https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-16/fedora_core...
Log: https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-16/fedora_core...
========================================
#fedora-meeting-1: fedora_coreos_meeting
========================================
Meeting started by dustymabe at 16:28:43 UTC. The full logs are
available at
https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-16/fedora_core...
.
Meeting summary
---------------
* roll call (dustymabe, 16:28:47)
* Action items from last meeting (dustymabe, 16:35:19)
* F33 rebase rpmdb migration (dustymabe, 16:36:54)
* LINK:
https://github.com/coreos/fedora-coreos-tracker/issues/609#issuecomment-6...
(dustymabe, 16:37:03)
* AGREED: we'll keep bdb for the rpm database for at least the first
release of Fedora 33 and then make the transition with a barrier in
the fedora 33 timeframe (dustymabe, 16:53:20)
* ACTION: jlebon to create a tracking ticket to move the rpm database
from bdb to sqlite in a barrier release in the f33 time frame
(dustymabe, 16:55:09)
* ad-hoc testing stream release to update the kernel (dustymabe,
17:01:16)
* open floor (dustymabe, 17:04:12)
* the fedora archive repo is in place, currently just waiting on infra
team to create a fedora URL to front it rather than an aws.s3 URL,
which will allow us to move it in the future if we need to.
(dustymabe, 17:05:06)
* LINK: https://github.com/fedora-silverblue/issue-tracker/issues/73
(dustymabe, 17:08:50)
Meeting ended at 17:14:45 UTC.
Action Items
------------
* jlebon to create a tracking ticket to move the rpm database from bdb
to sqlite in a barrier release in the f33 time frame
Action Items, by person
-----------------------
* jlebon
* jlebon to create a tracking ticket to move the rpm database from bdb
to sqlite in a barrier release in the f33 time frame
* **UNASSIGNED**
* (none)
People Present (lines said)
---------------------------
* dustymabe (82)
* jlebon (36)
* zodbot (19)
* lucab (11)
* cyberpear (6)
* ravanelli (3)
* walters (3)
* darkmuggle (1)
* misc (1)
* skunkerk (1)
* bgilbert (1)
* bh7cw (0)
Generated by `MeetBot`_ 0.1.4
.. _`MeetBot`: http://wiki.debian.org/MeetBot
3 years, 6 months
FCOS / Kubernetes (using docker) / Multus / CNI Bridge plugin - pods unable to access the additional network
by Gary Richards
I'm trying to migrate some container linux nodes to FCOS.
For my main pod to pod networking I use Calico and that continues to work fine on the new FCOS nodes.
In addition I have Multus setup which allows me to attach additional network interfaces to pods by creating network attachment definitions (which define the CNI plugin to use for that network interface and its config) and then associating those with the few pods that need them.
A NetworkAttachmentDefinition I have has the following config:
{
"cniVersion": "0.3.1",
"name": "lan",
"type": "bridge",
"bridge": "br-lan",
"ipam": {
"type": "static",
"addresses": [
{ "address": "10.1.0.2/16" }
]
}
}
Which really is as simple as it looks. Basically use the bridge CNI plugin to create a veth pair so that the additional network interface in the container is added to the br-lan bridge and configure the container side veth interface with the specified IP address.
If the pod that it annotated with this network attachment definition is moved onto an FCOS node, its networking on that interface fails to work. From outside, if I tcpdump on the FCOS node I see traffic traverse eno1 -> bond0 -> br-lan, but then it goes nowhere. If the pod is on the container linux node that traffic additionally makes its way onto the veth interface associated with the pod and everything works as expected.
In reverse I see the same thing. If I do something from the pod I see its traffic make it to the veth interface on the host, but never onto the bridge, but on the CL node it makes it to the bridge, through the other interfaces and out to the network. It's almost like the veth interface hasn't been added to the bridge... But it has!
Like I said, this works totally fine on container linux. The networking is configured exactly the same (other than via NetworkManager rather than systemd-networkd). So my network interfaces are as follows:
eno1 -> bond0
eno2 -> bond0
bond0 -> br-lan
bond0.100 -> br-something
bond0.200 -> br-somethingelse
When the pod comes up, the bridge CNI plugin via Multus creates the veth interface and adds the host side interface of the pair into br-lan. This output is from the FCOS node (the 'master br-lan' part is apparently shows which devices are added to the br-lan bridge if you haven't got brctl available to see the output that most people are probably used to seeing):
# ip link | grep br-lan
7: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
11: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP mode DEFAULT group default qlen 1000
31: veth1f6cabd9@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP mode DEFAULT group default
And this from the CL node, which matches:
# ip link | grep br-lan
6: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
10: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP mode DEFAULT group default qlen 1000
218: vetha2c4788e@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP mode DEFAULT group default
ipv4 ip_forward is enabled on both nodes
# cat /proc/sys/net/ipv4/ip_forward
1
rp_filter for all of the above mentioned interfaces is set to 1 on the CL node and 2 on the FCOS node. As far as I can tell 2 is the same as 1 but more loose about what it allows so long as it can still route to the source of a packet somehow. So I can't see it being the problem
Examining the bridge config, the CL node has stp disabled. The FCOS node has stp enabled. If I investigate further, both bond0 and the veth interface are in the forwarding state. (3 seems to be the FORWARDING state)
# cat /sys/devices/virtual/net/br-lan/brif/bond0/state
3
# cat /sys/devices/virtual/net/br-lan/brif/veth1f6cabd9/state
3
If I disable stp on the bridge on the FCOS node it makes no difference
I even started comparing everything under /sys/devices/virtual/net/br-lan on both nodes, but still can't see anything out of the ordinary.
I really can't believe that anything has changed too much in how any of this works between CL and FCOS, but i've run out of ideas. I've been doing VMs in a similar way for 10 years or so too, so as far as I can tell the networking itself is sound.
Any thoughts welcomed.
3 years, 6 months
Fedora CoreOS Meeting Minutes 2020-09-02
by Dusty Mabe
Minutes: https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-02/fedora_core...
Minutes (text): https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-02/fedora_core...
Log: https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-02/fedora_core...
========================================
#fedora-meeting-1: fedora_coreos_meeting
========================================
Meeting started by dustymabe at 16:31:19 UTC. The full logs are
available at
https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-02/fedora_core...
.
Meeting summary
---------------
* roll call (dustymabe, 16:31:24)
* Action items from last meeting (dustymabe, 16:35:16)
* LINK: https://hackmd.io/aVJiL9DUQpCDSafSBcs6ZQ?view (dustymabe,
16:36:12)
* tracker: Fedora 33 rebase work (dustymabe, 16:36:58)
* LINK: https://github.com/coreos/fedora-coreos-tracker/issues/609
(dustymabe, 16:37:04)
* please help us identify any potential issues with F33 that we need
to consider when migrating FCOS. (dustymabe, 16:39:45)
* we'll hopefully switch the `next` stream to F33 in the next cycle or
two (dustymabe, 16:40:09)
* Discuss adding systemd-networkd to FCOS (dustymabe, 16:43:02)
* LINK: https://github.com/coreos/fedora-coreos-tracker/issues/610
(dustymabe, 16:43:10)
* LINK: https://github.com/coreos/fedora-coreos-config/pull/574
(dustymabe, 16:43:28)
* AGREED: we'll reach out to the systemd team to see how they feel
about making a systemd-networkd subpackage. If they refuse or are
not interested we will explore option A (including systemd-networkd
in the base layer) but with a dropin that disables it by default.
(dustymabe, 17:35:07)
* open floor (dustymabe, 17:35:48)
* jdoss helped us get our twitter account re-activated
https://twitter.com/fedoracoreos (dustymabe, 17:36:32)
* There is a new `next` stream release out today with podman 2.0.6. If
all goes well we'll move to podman 2.x in next week's testing
release. (dustymabe, 17:37:58)
Meeting ended at 17:40:11 UTC.
Action Items
------------
Action Items, by person
-----------------------
* **UNASSIGNED**
* (none)
People Present (lines said)
---------------------------
* dustymabe (138)
* jdoss (47)
* bgilbert (27)
* lorbus (26)
* zodbot (21)
* aoei (19)
* walters (18)
* cyberpear (10)
* slowrie (8)
* miabbott (5)
* lucab (5)
* skunkerk (1)
Generated by `MeetBot`_ 0.1.4
.. _`MeetBot`: http://wiki.debian.org/MeetBot
3 years, 6 months