infrastructure March 2017

infrastructure@lists.fedoraproject.org

48 participants
84 discussions

Freeze Break Request: fix iscsi/mpath
by Kevin Fenzi 22 Mar '17

22 Mar '17

Greetings. This thursday (2017-03-23) there's going to be some upgrades done on our netapp storage. These upgrades should not cause any outages if everything was working as expected, but there's an issue: A number of our machines are logged into iscsi luns, but not properly multipathing that connection, which means that those machines could see an outage to the iscsi storage since they don't have the redundant paths. However, we are not using iscsi as widely as we used to either, so a number of these machines are logged in, but don't otherwise use the storage at all. So, I'd like to get a freeze break to do the following: 1. Wait until our f26rc2 is composed (which might mean waiting until tomorrow). 2. Apply the following ansible patch: diff --git a/playbooks/groups/virthost.yml b/playbooks/groups/virthost.yml index e3d673f..e53d540 100644 --- a/playbooks/groups/virthost.yml +++ b/playbooks/groups/virthost.yml @@ -22,7 +22,7 @@ - hosts - fas_client - collectd/base - - { role: iscsi_client, when: datacenter == "phx2" } + - { role: iscsi_client, when: inventory_hostname.startswith(('bvirthost','buildvmhost')) } - sudo - { role: openvpn/client, when: datacenter != "phx2" } - virthost This will restrict the iscsi client setup to only bvirthost and buildvmhost machines. Those are the only ones that _should_ be using this storage (but see below): 3. Run ansible commands over the non bvirthost/buildvmhost machines to do a 'iscsi logout', disable iscsi and multipathd services and remove the /var/lib/iscsi/nodes/* files. 4. Re-provision docker-registry02.stg.phx2.fedoraproject.org and loopabull01.stg.phx2.fedoraproject.org that were provisioned on the iscsi storage and make them use local storage. 5. Sadly, that leaves bvirthost01/02/03 still not working right. However, they use 0 things from the iscsi storage, so we can fix them after freeze. That should hopefully do it. ;) +1s? kevin

3 2

SSH to Bastion
by InvalidPath 22 Mar '17

22 Mar '17

I had to change my ssh key, how long does replication take before I can access bastion? Thanks! Ben

4 6

Bodhi 2.5.0 beta deployed to stg
by Randy Barlow 22 Mar '17

22 Mar '17

Hello! After a lot of wrestling I got a Bodhi 2.5.0 beta deployed to staging. We have a stakeholder's meeting in 7.5 days so I wanted to have a new version to discuss there and here it is. This release has some exciting new features and bug fixes[0], the most exciting of which is a theme redesign done by Ryan Lerch. He put particular focus on the update page, and I think it greatly increases the usability of the page by putting the most common information you want to see first. For an example, check out a kernel update[1]. Another neat thing about 2.5.0 is that Bodhi is now hosting its own docs at /docs. The UI doesn't advertise this link yet, though I do have an issue tracking that[3]. Obviously I don't plan to deploy this to production until after the freeze, but I thought it would be good to get it deployed to stg before the stakeholder's meeting next week and I thought it would be good to get the new design "out there" sooner rather than later as well. [0] https://bodhi.stg.fedoraproject.org/docs/release_notes.html#id1 [1] https://bodhi.stg.fedoraproject.org/updates/FEDORA-2017-e8e717e2b7 [2] https://bodhi.stg.fedoraproject.org/docs/ [3] https://github.com/fedora-infra/bodhi/issues/1321

4 8

Meeting Agenda Item: Introduction Ivan Garcia
by Ivan Garcia 21 Mar '17

21 Mar '17

Hello! My name is Ivan Garcia, my irc nic is cyberworm54. I've been here before but due to some time constraints, I needed to go idle for a while. Now I have free time that I can invest on this wonderful community! I am Linux Engineer currently working for Salesforce, I also hold/held a RHCE/RHCSA. I have a broad and in depth knowledge on rhel,fedora,centos and openstack. When it comes to programming and scripting bash and python are my go to's. I am an avid chef user and since I know the team uses ansible, I really would like to get a grip on ansible as I haven't worked with it before. I can bring some useful sysadmin skills to the table when it comes to openstack and rhel/centos . I am really looking forward to this! Have a great night/day !

1 0

FBR: Bodhi should stop sending error emails on success, and should start sending error emails on error
by Randy Barlow 20 Mar '17

20 Mar '17

There is a 1-character diff patch on fedmsg-atomic-composer that I'd like to apply to production to stop us from getting error e-mails whenever Bodhi succeeds: https://github.com/fedora-infra/fedmsg-atomic-composer/pull/21 Basically, the logic is exactly backwards on whether the stderr should be considered an error or info based on the exit code. If the FBR is approved I will also need someone to review the PR, and then I'll just make an upstream release and get it deployed.

3 7

Weekly Koji Infra Tag Report
by Nobody 19 Mar '17

19 Mar '17

This is a list of packages in the various infrastructure koji tags Please check and make sure there are not any that can be removed/dropped epel6-infra (no matching packages) epel7-infra Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- freeipa-ktutils epel7-infra puiterwijk glusterfs epel7-infra kevin fedmsg-beaker-repoupdate epel7-infra tflink anitya epel7-infra jcline the-new-hotness epel7-infra jcline fedocal epel7-infra pingou python-IPy epel7-infra kevin python-robosignatory epel7-infra puiterwijk pdc-updater epel7-infra ralph python-pdc epel7-infra ralph mirrormanager2 epel7-infra puiterwijk blockerbugs epel7-infra tflink python-django-jsonfield epel7-infra ralph f23-infra Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- libphutil, f23-infra tflink arcanist, f23-infra tflink phabricator f23-infra tflink phabricator-extension-ipsilonauth f23-infra tflink libphutil f23-infra tflink arcanist f23-infra tflink f24-infra Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- mediawiki-openid f24-infra kevin phabricator-extension-oauth f24-infra tflink python-twill f24-infra codeblock stickynotes2modernpaste f24-infra codeblock python-flask-testing f24-infra codeblock modern-paste f24-infra codeblock mediawiki-skin-fedora f24-infra puiterwijk mediawiki-FedoraBadges f24-infra kevin basset f24-infra puiterwijk phabricator f24-infra tflink mediawiki-Lockdown f24-infra kevin libphutil f24-infra tflink arcanist f24-infra tflink mediawiki-RSS f24-infra kevin mirrormanager2 f24-infra puiterwijk f25-infra Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- python-flask-testing f25-infra codeblock modern-paste f25-infra codeblock python-coveralls f25-infra codeblock mdapi f25-infra pingou basset f25-infra puiterwijk mediawiki-FedoraBadges f25-infra kevin mediawiki-Lockdown f25-infra kevin mediawiki-RSS f25-infra kevin mediawiki-openid f25-infra kevin plus-plus-service f25-infra pingou python-pdc f25-infra ralph python-django-cors-headers f25-infra ralph python-django-rest-framework-composed-permissions f25-infra ralph patternfly1 f25-infra ralph fas f25-infra kevin libphutil, f25-infra tflink arcanist, f25-infra tflink phabricator f25-infra tflink phabricator-extension-ipsilonauth f25-infra tflink libphutil f25-infra tflink arcanist f25-infra tflink python-twill f25-infra codeblock stickynotes2modernpaste f25-infra codeblock mediawiki-skin-fedora f25-infra kevin f26-infra (no matching packages) f27-infra (no matching packages)

1 0

Freeze Break Request: increase threads for resultsdb wsgi daemon
by Tim Flink 19 Mar '17

19 Mar '17

Production resultsdb is still really slow and we're still seeing the occasional error on result posting so I'd like to bump the resources allocated to the wsgi app again. +1s? Tim From 3d45155959cbdcde39c1a98a584a100b590761db Mon Sep 17 00:00:00 2001 From: Tim Flink <tflink(a)fedoraproject.org> Date: Fri, 17 Mar 2017 15:33:49 +0000 Subject: [PATCH] bumping resultsdb wsgi resources again --- roles/taskotron/resultsdb-backend/templates/resultsdb.conf.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/taskotron/resultsdb-backend/templates/resultsdb.conf.j2 b/roles/taskotron/resultsdb-ba index 97e73b9..c3f4d5c 100644 --- a/roles/taskotron/resultsdb-backend/templates/resultsdb.conf.j2 +++ b/roles/taskotron/resultsdb-backend/templates/resultsdb.conf.j2 @@ -1,5 +1,5 @@ {% if deployment_type in ['stg', 'prod'] %} -WSGIDaemonProcess resultsdb user=apache group=apache threads=100 processes=5 +WSGIDaemonProcess resultsdb user=apache group=apache threads=200 processes=20 {% else %} WSGIDaemonProcess resultsdb user=apache group=apache threads=5 {% endif %} -- 1.8.3.1

4 3

Meeting Agenda Item: Introduction Mohammed Tayeh
by Mohammed Tayeh 18 Mar '17

18 Mar '17

Hello there. Here's my proper introduction to the list. My name is Mohammed Tayeh, my IRC name is mohammed94 . I live in Gaza, Palestine and work as a junior Linux Systems Administrator. I have a Linux experience since 2012. i am work on "Orange Palestine ISP company" My jobs is IPTV admin. am Fedora ambassador on Palestine, also i do some QA task and translate fedora to arabic language I thinking about Join the infrastructure team on Fedora project. i want to learn more Systems Administration tasks and help fedora community. now I don't now what is your plan and yours day task i want to learn and improve my skills. my skills : * Programming languages: JAVA (JavaFX, JSP), beginner at Bash Script, html+css * i am work as a Systems Administrator with ( Apache, MySql, cpanel and other software) now i work on IPTV (wowza, MD stalker, nginx-rtmp, ffmpeg) how i can help the fedora community and also improve my skills and learn more and more on Systems Administration Best Regards, Mohammed Tayeh

1 0

FBR: Set resultsdb wsgi to normal values
by Patrick Uiterwijk 17 Mar '17

17 Mar '17

Hi, Can I get retroactive +1s for the patch? This drops the number of threads and processes down to normal limits: having 200 * 20 + 100 * 10 = 5000 threads for wsgi is... not going to work. Patrick commit 66172d7738bbee3749520b025a24b8930e5fd2c0 Author: Patrick Uiterwijk <puiterwijk(a)redhat.com> Date: Sat Mar 18 01:11:30 2017 +0000 Use sane values for processess and threads for resultsdb Signed-off-by: Patrick Uiterwijk <puiterwijk(a)redhat.com> diff --git a/roles/taskotron/resultsdb-backend/templates/resultsdb.conf.j2 b/roles/taskotron/resultsdb-backend/templat index c3f4d5c..dcec940 100644 --- a/roles/taskotron/resultsdb-backend/templates/resultsdb.conf.j2 +++ b/roles/taskotron/resultsdb-backend/templates/resultsdb.conf.j2 @@ -1,5 +1,5 @@ {% if deployment_type in ['stg', 'prod'] %} -WSGIDaemonProcess resultsdb user=apache group=apache threads=200 processes=20 +WSGIDaemonProcess resultsdb user=apache group=apache threads=20 processes=4 {% else %} WSGIDaemonProcess resultsdb user=apache group=apache threads=5 {% endif %} diff --git a/roles/taskotron/resultsdb-frontend/templates/resultsdb_frontend.conf.j2 b/roles/taskotron/resultsdb-front index e71cf58..a6b3596 100644 --- a/roles/taskotron/resultsdb-frontend/templates/resultsdb_frontend.conf.j2 +++ b/roles/taskotron/resultsdb-frontend/templates/resultsdb_frontend.conf.j2 @@ -1,5 +1,5 @@ {% if deployment_type in ['stg', 'prod'] %} -WSGIDaemonProcess resultsdb_frontend user=apache group=apache threads=100 processes=10 +WSGIDaemonProcess resultsdb_frontend user=apache group=apache threads=20 processes=4 {% else %} WSGIDaemonProcess resultsdb_frontend user=apache group=apache threads=5 {% endif %}

2 1

FBR: Update Pagure, Anitya and Piwik to use secure cipher set
by Patrick Uiterwijk 17 Mar '17

17 Mar '17

Hi, Turns out that these three services were not yet using our global secure cipher set. This means that they have the Apache defaults, which are quite insecure (RC4 and no FS). Can I please get +1s to apply the underneath patch? Patrick commit 55183057fc95109df5d6b50258918c59c7930674 Author: Patrick Uiterwijk <puiterwijk(a)redhat.com> Date: Fri Mar 17 23:28:19 2017 +0000 Update Pagure, anitya and piwik to use the secure cipher set Signed-off-by: Patrick Uiterwijk <puiterwijk(a)redhat.com> diff --git a/roles/anitya/frontend/files/0_releasemonitoring.conf b/roles/anitya/frontend/files/0_releasemonitoring.co index 56a0bfb..e054147 100644 --- a/roles/anitya/frontend/files/0_releasemonitoring.conf +++ b/roles/anitya/frontend/files/0_releasemonitoring.conf @@ -7,8 +7,8 @@ ServerName release-monitoring.org:443 SSLEngine on - SSLProtocol all -SSLv2 -SSLv3 - # Use secure TLSv1.1 and TLSv1.2 ciphers + SSLProtocol {{ ssl_protocols }} + SSLCipherSuite {{ ssl_ciphers }} Header always add Strict-Transport-Security "max-age=15768000; includeSubDomains; preload" SSLCertificateFile /etc/pki/tls/certs/release-monitoring.org.cert diff --git a/roles/pagure/frontend/templates/0_pagure.conf b/roles/pagure/frontend/templates/0_pagure.conf index a7b7e70..3c3f353 100644 --- a/roles/pagure/frontend/templates/0_pagure.conf +++ b/roles/pagure/frontend/templates/0_pagure.conf @@ -64,7 +64,8 @@ WSGIDaemonProcess paguredocs user=git group=git maximum-requests=1000 display-na ServerAdmin admin(a)fedoraproject.org SSLEngine on - SSLProtocol all -SSLv2 -SSLv3 + SSLProtocol {{ ssl_protocols }} + SSLCipherSuite {{ ssl_ciphers }} # Use secure TLSv1.1 and TLSv1.2 ciphers Header always add Strict-Transport-Security "max-age=15768000; includeSubDomains; preload" @@ -113,7 +114,8 @@ WSGIDaemonProcess paguredocs user=git group=git maximum-requests=1000 display-na {% endif %} SSLEngine on - SSLProtocol all -SSLv2 -SSLv3 + SSLProtocol {{ ssl_protocols }} + SSLCipherSuite {{ ssl_ciphers }} # Use secure TLSv1.1 and TLSv1.2 ciphers Header always add Strict-Transport-Security "max-age=15768000; includeSubDomains; preload" @@ -138,7 +140,8 @@ WSGIDaemonProcess paguredocs user=git group=git maximum-requests=1000 display-na WSGIScriptAlias / /var/www/docs_pagure.wsgi SSLEngine on - SSLProtocol all -SSLv2 -SSLv3 + SSLProtocol {{ ssl_protocols }} + SSLCipherSuite {{ ssl_ciphers }} # Use secure TLSv1.1 and TLSv1.2 ciphers Header always add Strict-Transport-Security "max-age=15768000; includeSubDomains; preload" diff --git a/roles/piwik/files/piwik-httpd.conf b/roles/piwik/files/piwik-httpd.conf index 4b55fdc..881c509e 100644 --- a/roles/piwik/files/piwik-httpd.conf +++ b/roles/piwik/files/piwik-httpd.conf @@ -11,8 +11,8 @@ ServerName piwik.fedorainfracloud.org SSLEngine on - SSLProtocol all -SSLv2 -SSLv3 - # Use secure TLSv1.1 and TLSv1.2 ciphers + SSLProtocol {{ ssl_protocols }} + SSLCipherSuite {{ ssl_ciphers }} Header always add Strict-Transport-Security "max-age=15768000; includeSubDomains; preload" SSLCertificateFile /etc/pki/tls/certs/piwik.fedorainfracloud.org.cert

4 6

← Newer
1
2
3
4
5
6
7
8
9
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure March 2017