infrastructure March 2017

infrastructure@lists.fedoraproject.org

48 participants
84 discussions

[PATCH] keep i386 off of the primary mirror space
by dennis＠ausil.us 09 Mar '17

09 Mar '17

From: Dennis Gilmore <ausil(a)fedoraproject.org> Signed-off-by: Dennis Gilmore <ausil(a)fedoraproject.org> --- roles/bodhi2/backend/files/fedora-updates-push | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/bodhi2/backend/files/fedora-updates-push b/roles/bodhi2/backend/files/fedora-updates-push index aff2a31..6e0a647 100755 --- a/roles/bodhi2/backend/files/fedora-updates-push +++ b/roles/bodhi2/backend/files/fedora-updates-push @@ -8,7 +8,7 @@ ATOMICDEST=/mnt/koji/atomic/ OPTIONS="-rlptDvHh --stats --delay-updates $RSYNC_OPTS" #f26 exlcude -PRIMARY_EXCLUDES=" --exclude=aarch64 --exclude=ppc64 --exclude=ppc64le --exclude=s390x " +PRIMARY_EXCLUDES=" --exclude=aarch64 --exclude=i386 --exclude=ppc64 --exclude=ppc64le --exclude=s390x " ALT_EXCLUDES=" --exclude=armhfp --exclude=SRPMS --exclude=x86_64 --exclude=s390x" #old style simple sync -- 1.8.3.1

2 1

[PATCH] sync to ALTDEST for alternative arches
by dennis＠ausil.us 09 Mar '17

09 Mar '17

From: Dennis Gilmore <ausil(a)fedoraproject.org> Signed-off-by: Dennis Gilmore <ausil(a)fedoraproject.org> --- roles/bodhi2/backend/files/fedora-updates-push | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/roles/bodhi2/backend/files/fedora-updates-push b/roles/bodhi2/backend/files/fedora-updates-push index 201c2e2..aff2a31 100755 --- a/roles/bodhi2/backend/files/fedora-updates-push +++ b/roles/bodhi2/backend/files/fedora-updates-push @@ -2,6 +2,7 @@ SOURCE=/mnt/koji/mash/updates DEST=/pub/fedora/linux/updates/ +ALTDEST=/pub/fedora-secondary/updates/ ATOMICSOURCE=/mnt/koji/mash/atomic/ ATOMICDEST=/mnt/koji/atomic/ @@ -100,9 +101,9 @@ done for rel in 26; do OUTPUT1=$(rsync $OPTIONS --exclude "repodata/*" $ALT_EXCLUDES \ - $SOURCE/f$rel-updates/ $DEST/$rel/ --link-dest $DEST/testing/$rel/) + $SOURCE/f$rel-updates/ $ALTDEST/$rel/ --link-dest $ALTDEST/testing/$rel/) OUTPUT2=$(rsync $OPTIONS --delete --delete-delay $ALT_EXCLUDES --exclude=Live --exclude=Images \ - $SOURCE/f$rel-updates/ $DEST/$rel/) + $SOURCE/f$rel-updates/ $ALTDEST/$rel/) # Grep out some signals from the stats bytes=$(echo "$OUTPUT1" | grep "Literal data" | awk ' { print $3 } ') @@ -121,9 +122,9 @@ done for rel in 26; do OUTPUT1=$(rsync $OPTIONS --exclude "repodata/*" $ALT_EXCLUDES \ - $SOURCE/f$rel-updates-testing/ $DEST/testing/$rel/) + $SOURCE/f$rel-updates-testing/ $ALTDEST/testing/$rel/) OUTPUT2=$(rsync $OPTIONS --delete --delete-delay $ALT_EXCLUDES --exclude=Live --exclude=Images \ - $SOURCE/f$rel-updates-testing/ $DEST/testing/$rel/) + $SOURCE/f$rel-updates-testing/ $ALTDEST/testing/$rel/) # Grep out some signals from the stats bytes=$(echo "$OUTPUT1" | grep "Literal data" | awk ' { print $3 } ') -- 1.8.3.1

3 2

Freeze break request: set new ppc64/ppc64le builders to not need nfs mount
by Kevin Fenzi 09 Mar '17

09 Mar '17

Greetings. Peter added 6 new builders a while back on ppc8-04. 4 of them ppc64 and 4 of them ppc64le. However, we have a problem with them not being able to mount the nfs storage from koji currently. There's some sort of very weird switch issue going on so we don't want to try and fix it right now. But, the only reason builders need the nfs mount is if they are going to do newrepo tasks. We can remove these from the createrepo channel and disable the nfs mount and they should be happy building away. We are low on these types of builders, so it would be good to get them added. +1s? kevin -- From 7bbcf534c7cabeea042bcab5a8256e549a5b9102 Mon Sep 17 00:00:00 2001 From: Kevin Fenzi <kevin(a)scrye.com> Date: Wed, 8 Mar 2017 22:12:14 +0000 Subject: [PATCH] try and add in new ppc64/ppc64le builders with no nfs/createrepo for now Signed-off-by: Kevin Fenzi <kevin(a)scrye.com> --- inventory/group_vars/all | 3 +++ inventory/host_vars/buildvm-ppc64-11.ppc.fedoraproject.org | 1 + inventory/host_vars/buildvm-ppc64-12.ppc.fedoraproject.org | 1 + inventory/host_vars/buildvm-ppc64-13.ppc.fedoraproject.org | 1 + inventory/host_vars/buildvm-ppc64le-11.ppc.fedoraproject.org | 1 + inventory/host_vars/buildvm-ppc64le-12.ppc.fedoraproject.org | 1 + inventory/host_vars/buildvm-ppc64le-13.ppc.fedoraproject.org | 1 + playbooks/groups/buildvm.yml | 2 +- 8 files changed, 10 insertions(+), 1 deletion(-) diff --git a/inventory/group_vars/all b/inventory/group_vars/all index 163a649..0f1ad44 100644 --- a/inventory/group_vars/all +++ b/inventory/group_vars/all @@ -238,3 +238,6 @@ collectd_apache: true # assume vpn is false vpn: False + +# assume createrepo is true and this builder has the koji nfs mount to do that +createrepo: True diff --git a/inventory/host_vars/buildvm-ppc64-11.ppc.fedoraproject.org b/inventory/host_vars/buildvm-ppc64-11.ppc.fedoraproject.org index 4f1dbed..80d38cd 100644 --- a/inventory/host_vars/buildvm-ppc64-11.ppc.fedoraproject.org +++ b/inventory/host_vars/buildvm-ppc64-11.ppc.fedoraproject.org @@ -5,3 +5,4 @@ eth1_ip: 10.5.127.118 # This is a lie, but the Power64 kernel/qemu/libvirt flips the eth0/eth1 main_bridge: br1 nfs_bridge: br0 +createrepo: False diff --git a/inventory/host_vars/buildvm-ppc64-12.ppc.fedoraproject.org b/inventory/host_vars/buildvm-ppc64-12.ppc.fedoraproject.org index 0392699..14fbfbc 100644 --- a/inventory/host_vars/buildvm-ppc64-12.ppc.fedoraproject.org +++ b/inventory/host_vars/buildvm-ppc64-12.ppc.fedoraproject.org @@ -5,3 +5,4 @@ eth1_ip: 10.5.127.119 # This is a lie, but the Power64 kernel/qemu/libvirt flips the eth0/eth1 main_bridge: br1 nfs_bridge: br0 +createrepo: False diff --git a/inventory/host_vars/buildvm-ppc64-13.ppc.fedoraproject.org b/inventory/host_vars/buildvm-ppc64-13.ppc.fedoraproject.org index e725c93..4e92132 100644 --- a/inventory/host_vars/buildvm-ppc64-13.ppc.fedoraproject.org +++ b/inventory/host_vars/buildvm-ppc64-13.ppc.fedoraproject.org @@ -5,3 +5,4 @@ eth1_ip: 10.5.127.120 # This is a lie, but the Power64 kernel/qemu/libvirt flips the eth0/eth1 main_bridge: br1 nfs_bridge: br0 +createrepo: False diff --git a/inventory/host_vars/buildvm-ppc64le-11.ppc.fedoraproject.org b/inventory/host_vars/buildvm-ppc64le-11.ppc.fedoraproject.org index 7b150d4..34afa7e 100644 --- a/inventory/host_vars/buildvm-ppc64le-11.ppc.fedoraproject.org +++ b/inventory/host_vars/buildvm-ppc64le-11.ppc.fedoraproject.org @@ -5,3 +5,4 @@ eth1_ip: 10.5.127.136 # This is a lie, but the Power64 kernel/qemu/libvirt flips the eth0/eth1 main_bridge: br1 nfs_bridge: br0 +createrepo: False diff --git a/inventory/host_vars/buildvm-ppc64le-12.ppc.fedoraproject.org b/inventory/host_vars/buildvm-ppc64le-12.ppc.fedoraproject.org index f4a26be..248ecb5 100644 --- a/inventory/host_vars/buildvm-ppc64le-12.ppc.fedoraproject.org +++ b/inventory/host_vars/buildvm-ppc64le-12.ppc.fedoraproject.org @@ -5,3 +5,4 @@ eth1_ip: 10.5.127.137 # This is a lie, but the Power64 kernel/qemu/libvirt flips the eth0/eth1 main_bridge: br1 nfs_bridge: br0 +createrepo: False diff --git a/inventory/host_vars/buildvm-ppc64le-13.ppc.fedoraproject.org b/inventory/host_vars/buildvm-ppc64le-13.ppc.fedoraproject.org index f2ea995..8986b1e 100644 --- a/inventory/host_vars/buildvm-ppc64le-13.ppc.fedoraproject.org +++ b/inventory/host_vars/buildvm-ppc64le-13.ppc.fedoraproject.org @@ -5,3 +5,4 @@ eth1_ip: 10.5.127.138 # This is a lie, but the Power64 kernel/qemu/libvirt flips the eth0/eth1 main_bridge: br1 nfs_bridge: br0 +createrepo: False diff --git a/playbooks/groups/buildvm.yml b/playbooks/groups/buildvm.yml index c4bf699..19f6087 100644 --- a/playbooks/groups/buildvm.yml +++ b/playbooks/groups/buildvm.yml @@ -22,7 +22,7 @@ - base - hosts - apache - - { role: nfs/client, mnt_dir: '/mnt/fedora_koji', nfs_src_dir: "{{ koji_hub_nfs }}" } + - { role: nfs/client, mnt_dir: '/mnt/fedora_koji', nfs_src_dir: "{{ koji_hub_nfs }}", when: createrepo } - { role: fas_client, when: not inventory_hostname.startswith('bkernel') } - { role: sudo, when: not inventory_hostname.startswith('bkernel') } - koji_builder -- 1.8.3.1

4 4

Meeting Agenda Item: Introduction Savitoj Singh
by Savitoj 09 Mar '17

09 Mar '17

Hello World! I would like to introduce myself to the team. My name is Savitoj Singh, I'm working in Red Hat (Pune) since 3.7 years as a Senior Systems Administrator and overall 5 years experience. We as a team takes care of 1300+ servers over a globe and manages 4 major Red Hat CEE Labs. I'm comfortable with Python and shell scripting, and I have a work experience on RH OpenStack, OpenShift, Docker, Vagrant, Jenkins, GitLab, CI/CD, RHV, CFME, Ansible, Puppet etc. My interests are more inclined towards DevOps. I would really like to learn Emacs and grind python skills. Here are more details about me: *GitHub*: https://github.com/savitojs *GitLab*: https://gitlab.com/savitojs *Freenode* *IRC*: Savitoj *Certifications*: https://www.redhat.com/rhtapps/certification/verify/?certId=110-473-592 *LinkedIn*: https://www.linkedin.com/in/savitojs Regards, Savitoj Singh

2 1

[PATCH] butt ugly hack to sync f26 updates.
by dennis＠ausil.us 08 Mar '17

08 Mar '17

From: Dennis Gilmore <ausil(a)fedoraproject.org> a releng issue has been filed @ https://pagure.io/releng/issue/6692 to do it in a much better way Signed-off-by: Dennis Gilmore <ausil(a)fedoraproject.org> --- roles/bodhi2/backend/files/fedora-updates-push | 91 ++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) diff --git a/roles/bodhi2/backend/files/fedora-updates-push b/roles/bodhi2/backend/files/fedora-updates-push index d8c9227..201c2e2 100755 --- a/roles/bodhi2/backend/files/fedora-updates-push +++ b/roles/bodhi2/backend/files/fedora-updates-push @@ -6,7 +6,11 @@ ATOMICSOURCE=/mnt/koji/mash/atomic/ ATOMICDEST=/mnt/koji/atomic/ OPTIONS="-rlptDvHh --stats --delay-updates $RSYNC_OPTS" +#f26 exlcude +PRIMARY_EXCLUDES=" --exclude=aarch64 --exclude=ppc64 --exclude=ppc64le --exclude=s390x " +ALT_EXCLUDES=" --exclude=armhfp --exclude=SRPMS --exclude=x86_64 --exclude=s390x" +#old style simple sync for rel in 24 25; do OUTPUT1=$(rsync $OPTIONS --exclude "repodata/*" \ @@ -49,6 +53,93 @@ for rel in 24 25; do fi done +#new style primary sync +for rel in 26; do + + OUTPUT1=$(rsync $OPTIONS --exclude "repodata/*" $PRIMARY_EXCLUDES \ + $SOURCE/f$rel-updates/ $DEST/$rel/ --link-dest $DEST/testing/$rel/) + OUTPUT2=$(rsync $OPTIONS --delete --delete-delay $PRIMARY_EXCLUDES --exclude=Live --exclude=Images \ + $SOURCE/f$rel-updates/ $DEST/$rel/) + + # Grep out some signals from the stats + bytes=$(echo "$OUTPUT1" | grep "Literal data" | awk ' { print $3 } ') + deleted=$(echo "$OUTPUT2" | grep "deleting " | wc -l) + + # If anything changed, then publish a fedmsg message as bodhi.updates.sync + if [ "$bytes" != "0" -o "$deleted" != "0" ]; then + echo "{\"bytes\": \"$bytes\", \"deleted\": \"$deleted\", \"repo\": \"updates\", \"release\": \"$rel\"}" | fedmsg-logger \ + --cert-prefix ftpsync \ + --modname bodhi \ + --topic updates.fedora.sync \ + --json-input &> /dev/null + fi + +done +for rel in 26; do + + OUTPUT1=$(rsync $OPTIONS --exclude "repodata/*" $PRIMARY_EXCLUDES \ + $SOURCE/f$rel-updates-testing/ $DEST/testing/$rel/) + OUTPUT2=$(rsync $OPTIONS --delete --delete-delay $PRIMARY_EXCLUDES --exclude=Live --exclude=Images \ + $SOURCE/f$rel-updates-testing/ $DEST/testing/$rel/) + + # Grep out some signals from the stats + bytes=$(echo "$OUTPUT1" | grep "Literal data" | awk ' { print $3 } ') + deleted=$(echo "$OUTPUT2" | grep "deleting " | wc -l) + + # If anything changed, then publish a fedmsg message as bodhi.updates.sync + if [ "$bytes" != "0" -o "$deleted" != "0" ]; then + echo "{\"bytes\": \"$bytes\", \"deleted\": \"$deleted\", \"repo\": \"updates-testing\", \"release\": \"$rel\"}" | fedmsg-logger \ + --cert-prefix ftpsync \ + --modname bodhi \ + --topic updates.fedora.sync \ + --json-input &> /dev/null + fi + +done +# new style alternative sync +for rel in 26; do + + OUTPUT1=$(rsync $OPTIONS --exclude "repodata/*" $ALT_EXCLUDES \ + $SOURCE/f$rel-updates/ $DEST/$rel/ --link-dest $DEST/testing/$rel/) + OUTPUT2=$(rsync $OPTIONS --delete --delete-delay $ALT_EXCLUDES --exclude=Live --exclude=Images \ + $SOURCE/f$rel-updates/ $DEST/$rel/) + + # Grep out some signals from the stats + bytes=$(echo "$OUTPUT1" | grep "Literal data" | awk ' { print $3 } ') + deleted=$(echo "$OUTPUT2" | grep "deleting " | wc -l) + + # If anything changed, then publish a fedmsg message as bodhi.updates.sync + if [ "$bytes" != "0" -o "$deleted" != "0" ]; then + echo "{\"bytes\": \"$bytes\", \"deleted\": \"$deleted\", \"repo\": \"updates\", \"release\": \"$rel\"}" | fedmsg-logger \ + --cert-prefix ftpsync \ + --modname bodhi \ + --topic updates.fedora.sync \ + --json-input &> /dev/null + fi + +done +for rel in 26; do + + OUTPUT1=$(rsync $OPTIONS --exclude "repodata/*" $ALT_EXCLUDES \ + $SOURCE/f$rel-updates-testing/ $DEST/testing/$rel/) + OUTPUT2=$(rsync $OPTIONS --delete --delete-delay $ALT_EXCLUDES --exclude=Live --exclude=Images \ + $SOURCE/f$rel-updates-testing/ $DEST/testing/$rel/) + + # Grep out some signals from the stats + bytes=$(echo "$OUTPUT1" | grep "Literal data" | awk ' { print $3 } ') + deleted=$(echo "$OUTPUT2" | grep "deleting " | wc -l) + + # If anything changed, then publish a fedmsg message as bodhi.updates.sync + if [ "$bytes" != "0" -o "$deleted" != "0" ]; then + echo "{\"bytes\": \"$bytes\", \"deleted\": \"$deleted\", \"repo\": \"updates-testing\", \"release\": \"$rel\"}" | fedmsg-logger \ + --cert-prefix ftpsync \ + --modname bodhi \ + --topic updates.fedora.sync \ + --json-input &> /dev/null + fi + +done + for rel in 24 25; do OUTPUT1=$(rsync $OPTIONS --ignore-existing \ -- 1.8.3.1

2 1

Plan for tomorrow's Fedora Infrastructure meeting - 2017-03-09
by Kevin Fenzi 08 Mar '17

08 Mar '17

The infrastructure team will be having it's weekly meeting tomorrow, 2017-03-09 at 19:00 UTC in #fedora-meeting on the freenode network. We have a gobby document (see: https://fedoraproject.org/wiki/Gobby ) fedora-infrastructure-meeting-next is the document. Please try and review and edit that document before the meeting and we will use it to have our agenda of things to discuss. A copy as of today is included in this email. If you have something to discuss, add the topic to the discussion area with your name. If you would like to teach other folks about some application or setup in our infrastructure, please add that topic and your name to the learn about section. kevin -- This shared document is for the next fedora infrastructure meeting. = Introduction = We will use it over the week before the meeting to gather status and info and discussion items and so forth, then use it in the irc meeting to transfer information to the meetbot logs. = Meeting start stuff = #startmeeting Infrastructure (2017-03-09) #meetingname infrastructure #topic aloha #chair smooge relrod nirik abadger1999 lmacken dgilmore threebean pingou puiterwijk pbrobinson #topic New folks introductions = Status / information / Trivia / Announcements = (We put things here we want others on the team to know, but don't need to discuss) (Please use #info <the thing> - your name) #topic announcements and information #info Fedora 26 Alpha freeze is on now! - everyone #info Updated composers and buildvm's before freeze - kevin #info more hosted requests trickling in since sunset, but not too many - kevin #info added jenkins f25 ppc64le instance for your secondary arch jenkins needs - kevin #info = Things we should discuss = We use this section to bring up discussion topics. Things we want to talk about as a group and come up with some consensus /suor decision or just brainstorm a problem or issue. If there are none of these we skip this section. (Use #topic your discussion topic - your username) #topic Setting aside one download server for tier1 mirrors only - kevin #topic = Apprentice office hours = #topic Apprentice Open office hours Here we will discuss any apprentice questions, try and match up people looking for things to do with things to do, progress, testing anything like that. = Learn about some application or gsetup in infrastructure = (This section, each week we get 1 person to talk about an application or setup that we have. Just going over what it is, how to contribute, ideas for improvement, etc. Whoever would like to do this, just add the i/nfo in this section. In the event we don't find someone to teach about something, we skip this section and just move on to open floor.) #topic Learn about: = Meeting end stuff = #topic Open Floor #endmeeting

1 0

[PATCH] setup Fedora 27 keys for koji-gc cleanup
by dennis＠ausil.us 07 Mar '17

07 Mar '17

From: Dennis Gilmore <ausil(a)fedoraproject.org> Signed-off-by: Dennis Gilmore <ausil(a)fedoraproject.org> --- roles/koji_hub/templates/koji-gc.conf.j2 | 3 +++ 1 file changed, 3 insertions(+) diff --git a/roles/koji_hub/templates/koji-gc.conf.j2 b/roles/koji_hub/templates/koji-gc.conf.j2 index 9ecb750..9c48a8e 100644 --- a/roles/koji_hub/templates/koji-gc.conf.j2 +++ b/roles/koji_hub/templates/koji-gc.conf.j2 @@ -35,6 +35,7 @@ key_aliases = 81B46521 fedora-24 FDB19C98 fedora-25 64DAB85D fedora-26 + F5282EE4 fedora-27 217521F6 fedora-epel 0608B895 fedora-epel-6 352C64E5 fedora-epel-7 @@ -52,6 +53,7 @@ unprotected_keys = fedora-24 fedora-25 fedora-26 + fedora-27 fedora-extras redhat-beta fedora-epel @@ -91,6 +93,7 @@ policy = sig fedora-24 && age < 12 weeks :: keep sig fedora-25 && age < 12 weeks :: keep sig fedora-26 && age < 12 weeks :: keep + sig fedora-27 && age < 12 weeks :: keep sig fedora-epel && age < 12 weeks :: keep sig fedora-epel-6 && age < 12 weeks :: keep sig fedora-epel-7 && age < 12 weeks :: keep -- 1.8.3.1

2 1

Freeze break request: sync f26 updates
by Kevin Fenzi 07 Mar '17

07 Mar '17

When setting things up, we failed to setup the updates sync job to sync new f26 updates. This is needed now that we have enabled bodhi. So we need to apply the following patch, ansible it out to bodhi-backend01, and then also create empty repos in /mnt/koji/mash/updates for f26-updates and f26-updates-testing. The f26-updates-testing link will repoint to the first f26-updates-testing bodhi push (later tonight) and the f26-updates one will stay pointed at the empty repo until we are ready to stage 0 day updates. +1s? kevin -- diff --git a/roles/bodhi2/backend/files/fedora-updates-push b/roles/bodhi2/backend/files/fedora-updates-push index d8c9227..9b5025d 100755 --- a/roles/bodhi2/backend/files/fedora-updates-push +++ b/roles/bodhi2/backend/files/fedora-updates-push @@ -7,7 +7,7 @@ ATOMICDEST=/mnt/koji/atomic/ OPTIONS="-rlptDvHh --stats --delay-updates $RSYNC_OPTS" -for rel in 24 25; do +for rel in 24 25 26; do OUTPUT1=$(rsync $OPTIONS --exclude "repodata/*" \ $SOURCE/f$rel-updates/ $DEST/$rel/ --link-dest $DEST/testing/$rel/) @@ -28,7 +28,7 @@ for rel in 24 25; do fi done -for rel in 24 25; do +for rel in 24 25 26; do OUTPUT1=$(rsync $OPTIONS --exclude "repodata/*" \ $SOURCE/f$rel-updates-testing/ $DEST/testing/$rel/)

1 1

[PATCH] - fallback to local FontAwesome fonts in apps.fp.org/global
by Ryan Lerch 07 Mar '17

07 Mar '17

This patch adds a final-level fallback to the FontAwesome font declaration to use the local version of the FontAwesome font. This is mainly for when devs are hacking on a local instance of apps that are using this font, as the CORS settings will resctrict them from downloading the fonts.

3 5

[release] Hotness 0.8.1
by Jeremy Cline 07 Mar '17

07 Mar '17

Hey all, I just tagged the-new-hotness 0.8.1 and it's running in the staging environment. If all goes well I will push it to production either Thursday or next Monday. Changelog ========= 0.8.1 is a bugfix release intended to improve the reliability of the-new-hotness, particularly the code responsible for creating Koji scratch builds for new updates. Bugfixes ^^^^^^^^ - Attempting to remove a working directory that's already gone no longer results in a traceback (#154) - Fetching package sources now has greatly improved error reporting (#161) - Packages with multiple source tarballs are now correctly handled. Previously, only the first source URL was used (#161) -- Jeremy Cline XMPP: jeremy(a)jcline.org IRC: jcline

1 0

← Newer
1
2
3
4
5
6
7
8
9
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure March 2017