infrastructure October 2015

infrastructure@lists.fedoraproject.org

37 participants
79 discussions

fedrtc.org -> infrastructure?
by Daniel Pocock 14 Nov '15

14 Nov '15

Hi all, At some point, it would be good to see fedrtc.org migrate to Fedora infrastructure and use the fedoraproject.org domain I'd be happy to submit the full request for resources[1] but I just want to see if there is any initial comment on it. Here is a list of what is involved: - it uses a PostgreSQL database schema[2] - it requires some DNS entries (SRV and NAPTR), examples[3] - it needs a TLS cert for fedoraproject.org on the host(s) where it runs - it has static HTTP content and PHP that is currently hosted with all but one problem[4] on a RHEL7 httpd. Content is in Github[5], it could be presented as an RPM if necessary. - all packages are in EPEL7, except: cajun-json in EPEL6, in testing for EPEL7 resiprocate in Fedora, builds from SRPM on RHEL7 - the SIP proxy is a single daemon, managed by systemctl. All settings in a single file, /etc/repro/repro.config - the TURN server process is also a single daemon, managed by systemctl. All settings in a single file, /etc/reTurn/reTurnServer.config Just to clarify the scope of this: it is not a full telephony service like Asterisk, just a SIP proxy and TURN server. There is no persistent state information (as there would be for voicemail, email service, etc) and no customized routing. Ongoing maintenance requirements: - TLS certificate renewals - monitoring the ports - package updates from time to time It currently runs on a lab machine, I'd be happy to arrange SSH access to the Fedora Infrastructure team to see exactly what is involved and verify that it is manageable. Regards, Daniel 1. https://fedoraproject.org/wiki/Request_For_Resources 2. https://github.com/resiprocate/resiprocate/blob/master/repro/create_postgre… 3. http://rtcquickstart.org/guide/multi/dns.html 4. https://bugzilla.redhat.com/show_bug.cgi?id=1222649 5. https://github.com/opentelecoms-org/fedrtc-org

5 12

ansible csi group vars - dns
by Ali Khalidi 06 Nov '15

06 Nov '15

infra - here is the diffs for csi vars for group: dns applicable when we come out of freez. diff --git a/inventory/group_vars/dns b/inventory/group_vars/dns index 17da9d0..db01b32 100644 --- a/inventory/group_vars/dns +++ b/inventory/group_vars/dns @@ -16,3 +16,7 @@ nrpe_procs_warn: 300 nrpe_procs_crit: 500 sudoers: "{{ private }}/files/sudo/sysadmin-dns" + +csi_security_category: High +csi_primary_contact: Fedora Admins - admin(a)fedoraproject.org +csi_purpose: Domain Name Service diff --git a/inventory/host_vars/ns02.fedoraproject.org b/inventory/host_vars/ns02.fedoraproject.org index ec29b7e..ffc9479 100644 --- a/inventory/host_vars/ns02.fedoraproject.org +++ b/inventory/host_vars/ns02.fedoraproject.org @@ -15,3 +15,19 @@ datacenter: ibiblio ks_url: http://209.132.181.6/repo/rhel/ks/kvm-rhel-7-ext ks_repo: http://209.132.181.6/repo/rhel/RHEL7-x86_64/ + + +csi_relationship: | + + ns02 is a master dns server. + It serves about every domain under fedoraproject, fedo*, as well as others, both forward and reverse. + + * This host relies on: + - The virthost it's hosted on (ibiblio03.fedoraproject.org) + - batcave for dns git and keys + - connectivity to maxmind to create geoIP dns acl + + * Things that rely on this host: + - The Internet/Community to resolve everything related to fedora and reverse-IP for allocated subnets + - If this host is down, dns queries will slow down by the portion of this host to the total name servers responsible for the same domain set. + - secodary/slave dns servers diff --git a/inventory/host_vars/ns03.phx2.fedoraproject.org b/inventory/host_vars/ns03.phx2.fedoraproject.org index 4515d5e..669b5ff 100644 --- a/inventory/host_vars/ns03.phx2.fedoraproject.org +++ b/inventory/host_vars/ns03.phx2.fedoraproject.org @@ -10,3 +10,18 @@ ansible_ssh_host: ns03.phx2.fedoraproject.org vmhost: virthost21.phx2.fedoraproject.org datacenter: phx2 + +csi_relationship: | + + ns03 is a master dns server. + It serves about every domain under fedoraproject, fedo*, as well as others, both forward and reverse. + + * This host relies on: + - The virthost it's hosted on (virthost21.phx2.fedoraproject.org) + - batcave for dns git and keys + - connectivity to maxmind to create geoIP dns acl + + * Things that rely on this host: + - The Internet/Community to resolve everything related to fedora and reverse-IP for allocated subnets + - If this host is down, dns queries will slow down by the portion of this host to the total name servers responsible for the same domain set. + - secodary/slave dns servers diff --git a/inventory/host_vars/ns04.phx2.fedoraproject.org b/inventory/host_vars/ns04.phx2.fedoraproject.org index 9c72ca9..1a4089f 100644 --- a/inventory/host_vars/ns04.phx2.fedoraproject.org +++ b/inventory/host_vars/ns04.phx2.fedoraproject.org @@ -10,3 +10,18 @@ ansible_ssh_host: ns04.phx2.fedoraproject.org vmhost: virthost15.phx2.fedoraproject.org datacenter: phx2 + +csi_relationship: | + + ns04 is a master dns server. + It serves about every domain under fedoraproject, fedo*, as well as others, both forward and reverse. + + * This host relies on: + - The virthost it's hosted on (virthost15.phx2.fedoraproject.org) + - batcave for dns git and keys + - connectivity to maxmind to create geoIP dns acl + + * Things that rely on this host: + - The Internet/Community to resolve everything related to fedora and reverse-IP for allocated subnets + - If this host is down, dns queries will slow down by the portion of this host to the total name servers responsible for the same domain set. + - secodary/slave dns servers diff --git a/inventory/host_vars/ns05.fedoraproject.org b/inventory/host_vars/ns05.fedoraproject.org index 3955f44..e02f9a9 100644 --- a/inventory/host_vars/ns05.fedoraproject.org +++ b/inventory/host_vars/ns05.fedoraproject.org @@ -12,3 +12,18 @@ postfix_group: vpn vmhost: internetx01.fedoraproject.org datacenter: internetx + +csi_relationship: | + + ns05 is a master dns server. + It serves about every domain under fedoraproject, fedo*, as well as others, both forward and reverse. + + * This host relies on: + - The virthost it's hosted on (internetx01.fedoraproject.org) + - batcave for dns git and keys + - connectivity to maxmind to create geoIP dns acl + + * Things that rely on this host: + - The Internet/Community to resolve everything related to fedora and reverse-IP for allocated subnets + - If this host is down, dns queries will slow down by the portion of this host to the total name servers responsible for the same domain set. + - secodary/slave dns servers

2 4

Apprentice Workday Page available in Wiki - Need someone to add to fedocal please.
by Zach Villers 05 Nov '15

05 Nov '15

I created https://fedoraproject.org/wiki/Workday_planning_page it links from a section on the apprentice getting started page. Please edit/correct as you see fit. I do not have permission to add the event to fedocal. Can someone please add the date (Nov 18) for me or let me know who can add it or give me access? Thanks, Zach

2 1

Fedora 23 Final Freeze now in effect
by Kevin Fenzi 04 Nov '15

04 Nov '15

Greetings. we are now in the infrastructure freeze leading up to the Fedora 23 Final release. This is a final release freeze. This means that hosts that are marked as freezing should not have any changes made to them except as part of a freeze break request (see below) We do this to make sure that our infrastructure is stable to allow for building/composing/testing and distributing Fedora 23. This freeze will end 1 day after Fedora 23 is released. (Currently scheduled for 2015-10-27) Freeze breaks should be sent to this list, clearly describing the planned change and include patches (if applicable). Freeze breaks should not be applied until they have gotten at least 2 +1's from members of sysadmin-main and/or releng groups. You can see a list of hosts that do not freeze by checking out the ansible repo and running the freezelist script: git clone http://infrastructure.fedoraproject.org/infra/ansible.git scripts/freezelist -i inventory Thanks, kevin

1 1

mdapi in our infrastructure
by Pierre-Yves Chibon 02 Nov '15

02 Nov '15

Good Morning everyone, Over the last few days I have been working on a small app: mdapi. It is aimed at serving the metadata from our repos simply and *fast*, offering information from koji, rawhide, all our active branches and epel (you'll have to specify which you want) and for each it will return you the first hit it finds in the testing, updates or release repo (it says which in the json returned). I deployed it for testing in our cloud, at: http://209.132.184.236/ This morning I took a look at what it would take to package mdapi for epel7. mdapi has 5 dependencies: - aiohttp -> Has an epel7 branch but was never built, open a ticket on the bugzilla asking it - requests -> Exists in RHEL channels but not as py3, so will need a new package - simplejson -> Asked the current PoC if it could be built with the py3 sub-package in epel7 - sqlalchemy -> Looked at it (I'm the PoC for epel7), but requires: - python3-setuptools (same situation as requests) - requires: python3-pip, python3-wheel, python3-mock, python3-pytest - python3-nose (same situation as requests and setuptools as requests - werkzeug -> same situation as requests, setuptools... The dependency list as well as the current situation of the guidelines for packaging python3 app in epel [1], makes me proposing that we deploy mdapi in Fedora (at least for now). The second point I would like to raise is how we deploy this application. This isn't a standard wsgi application (since it's async) and it cannot run with apache (afaiu). So far the other async application we have (in pagure) have been deployed simply as a systemd service. I wonder if we want to use the same approach here or if we should investigate things like gunicorn/nginx or so. Does someone have experience in this field? Any advice/feedback? Thanks for your attention, Pierre [1] https://fedorahosted.org/fpc/ticket/567

5 16

[release] statscache: 0.0.3
by Ralph Bean 31 Oct '15

31 Oct '15

0.0.3 ----- Commits - 4baf09ea2 Include some forgotten things in packaging declarations. https://github.com/fedora-infra/statscache/commit/4baf09ea2

1 1

Freeze break request: New release of tunir for Autocloud backends
by Kushal Das 31 Oct '15

31 Oct '15

Hi, I would like to apply for an exception for this freeze for a new release of tunir for the Autocloud backends. This release has non-gating tests capability as the major change. Kushal -- Fedora Cloud Engineer CPython Core Developer CentOS Cloud SIG lead http://kushaldas.in

2 1

FAS3 status update
by Pierre-Yves Chibon 31 Oct '15

31 Oct '15

Dear all, Over the last couple of weeks Xavier and I managed to catch up a couple of time to discuss the current status of FAS3. I would like to summarize a little the output of these discussions. We can basically, consider 3 types of tasks left. * Development: Admin panel Xavier is working on providing an Admin panel to FAS3. This would provide a central place for admin tasks such as blocking an user, deleting an account and so on -> Due by next week Group membership model Xavier is thinking to rework the membership model for FAS groups. There is currently three levels: user, sponsors and admins while there are groups where sponsors might not be needed or where another level could be useful. -> Optional for 3.0, NOTABLOCKER Unit-tests FAS3 has currently little tests and needs much more -> Would be good to have for 3.0 but NOTABLOCKER (at least for a staging instance) 2FA 2 Factor Authentication is probably the biggest point remaining to develop. In FAS2, yubikey is integrated into FAS, while gauth isn't. For FAS3 we probably want to integrate both so that users have a place to set and reset their tokens. On the otherside, keeping the yubikey and the gauth servers out of FAS3 also make sense. So we may want to find a way for both to share some information where FAS3 would have read/write permissions while the yubikey/gauth servers would have only read. * Migration The idea here is two provide two scripts, a python script that will check the FAS2 DB and report problematic data (for example multiple users having the same IRC nick). Then a SQL script to be used once to convert the FAS2 DB into a FAS3 DB or migrate from the FAS2 DB to a different FAS3 DB (we'll see what is the easiest or makes the most sense). * Integration There are still quite a few integrations to do. - Packaging Xavier checked and said that all dependencies are present in RHEL, EPEL or Fedora already. We'll need to double-check and request the epel7 branch where needed. - Ipsilon This should likely be the first one to work on, in order to allow login on our apps in staging. - fedora-python Here the client needs to be ported from FAS2 to FAS3. The API has entirely changed and is not backward compatible but we *may* be able to make the API of the client backward compatible. Xavier already started working on this in the FAS3 branch - All our apps using FAS The idea is to deploy FAS3 in staging so that we can test/fix all our apps using FAS one by one and take our time to do this. As you can see, most of the development will be done by next week, except for the 2FA aspect which we need to figure out. I have volunteered to work on the migration scripts. Xavier and Patrick will need to coordinate for the Ipsilon integration. Then, we should be ready to push a FAS3 instance to staging. The idea is that we have FAS3 running in staging by Christmas. How does this sound? Anyone willing to step in? Thanks for your attention, Pierre & Xavier

3 2

Freeze break request: developer.fedoraproject.org to production
by Kevin Fenzi 30 Oct '15

30 Oct '15

Greetings. I've gotten the developer portal working in staging a few days ago: https://developer.stg.fedoraproject.org/ I'd like to move it into production now so it's ready for their announcement. +1s? Note that currently the site is built using jekyl on developer.fedorainfracloud.org and then synced with a github repo which we pull from. Moving forward we will come up with a better way to do all this, we just wanted to get things working for release now. kevin -- diff --git a/playbooks/groups/sundries.yml b/playbooks/groups/sundries.yml index 34dab01..c8bc7c0 100644 --- a/playbooks/groups/sundries.yml +++ b/playbooks/groups/sundries.yml @@ -49,7 +49,7 @@ - role: membership-map/build when: master_sundries_node - role: developer/build - when: master_sundries_node and env == "staging" + when: master_sundries_node tasks: - include: "{{ tasks }}/yumrepos.yml" diff --git a/playbooks/include/proxies-fedora-web.yml b/playbooks/include/proxies-fedora-web.yml index 250e75a..ce1f7d1 100644 --- a/playbooks/include/proxies-fedora-web.yml +++ b/playbooks/include/proxies-fedora-web.yml @@ -41,5 +41,4 @@ website: docs.fedoraproject.org - role: developer/website - website: developer.stg.fedoraproject.org - when: env == "staging" + website: developer.fedoraproject.org diff --git a/playbooks/include/proxies-websites.yml b/playbooks/include/proxies-websites.yml index 5feef25..0e60bed 100644 --- a/playbooks/include/proxies-websites.yml +++ b/playbooks/include/proxies-websites.yml @@ -422,13 +422,12 @@ cert_name: "{{wildcard_cert_name}}" - role: httpd/website - name: developer.stg.fedoraproject.org + name: developer.fedoraproject.org server_aliases: [developer.stg.fedoraproject.org] # Set this explicitly to stg here.. as per the original puppet config. SSLCertificateChainFile: wildcard-2014.stg.fedoraproject.org.intermediate.cert sslonly: true cert_name: "{{wildcard_cert_name}}" - when: env == "staging" # Kinda silly that we have two entries here, one for prod and one for stg. # This is inherited from our puppet setup -- we can collapse them as soon as

5 4

Plan for thursday's Fedora Infrastructure meeting - 2015-10-29
by Kevin Fenzi 29 Oct '15

29 Oct '15

The infrastructure team will be having it's weekly meeting tomorrow, 2015-10-15 at 18:00 UTC in #fedora-meeting on the freenode network. We have a gobby document (see: https://fedoraproject.org/wiki/Gobby ) NOTE: Gobby has changed, we are now using gobby05 and infinote. :) fedora-infrastructure-meeting-next is the document. Please try and review and edit that document before the meeting and we will use it to have our agenda of things to discuss. A copy as of today is included in this email. If you have something to discuss, add the topic to the discussion area with your name. If you would like to teach other folks about some application or setup in our infrastructure, please add that topic and your name to the learn about section. kevin -- = Introduction = This shared document is for the next fedora infrastructure meeting. We will use it over the week before the meeting to gather status and info and discussion items and so forth, then use it in the irc meeting to transfer information to the meetbot logs. = Meeting start stuff = #startmeeting Infrastructure (2015-10-29) #meetingname infrastructure #topic aloha #chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch threebean pingou puiterwijk pbrobinson #topic New folks introductions / Apprentice feedback = Status / information / Trivia / Announcements = (We put things here we want others on the team to know, but don't need to discuss) (Please use #info <the thing> - your name) #topic announcements and information #info Still in f23 final freeze - everyone #info = Things we should discuss = We use this section to bring up discussion topics. Things we want to talk about as a group and come up with some consensus or decision or just brainstorm a problem or issue. If there are none of these we skip this section. (Use #topic your discussion topic - your username) #topic = Learn about some application or setup in infrastructure = (This section, each week we get 1 person to talk about an application or setup that we have. Just going over what it is, how to contribute, ideas for improvement, etc. Whoever would like to do this, just add the info in this section. In the event we don't find someone to teach about something, we skip this section and just move on to open floor.) #topic Learn about: = Meeting end stuff = #topic Open Floor #endmeeting

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure October 2015