April 2017 - infrastructure - Fedora mailing-lists

About JS framework
by Pierre-Yves Chibon 17 Oct '22

17 Oct '22

Good Morning Everyone, Our infrastructure is mostly a python store, meaning almost all our apps are written in python and most using wsgi. However in python we are using a number of framework: * flask for most * pyramid for some of the biggest (bodhi, FAS3) * Django (askbot, Hyperkitty) * TurboGears2 (fedora-packages) * aiohttp (python3, async app: mdapi) While this makes sometime things difficult, these are fairly standard framework and most of our developers are able to help on all. However, as I see us starting to look at JS for some of our apps (fedora-hubs, wartaa...), I wonder if we could start the discussion early about the different framework and eventually see if we can unify around one. This would also allow those of us not familiar with any JS framework to look at the recommended one instead of picking one up semi-randomly. So has anyone experience with one or more JS framework? Do you have one that would you recommend? Why? Thanks for your inputs, Pierre

10 12

Re: Subscription lists
by Misha S. 19 Jul '17

19 Jul '17

I can not re-enable or change sending of mailing list mail to myself. https://lists.fedoraproject.org/admin/accounts/subscriptions Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request. Please contact the server administrator at root@localhost to inform them of the time this error occurred, and the actions you performed just before this error. More information about this error may be available in the server error log. Misha https://fedoraproject.org/wiki/User:Shnurapet English / Español / Italiano / Русский

2 2

The future of pkgdb
by Pierre-Yves Chibon 28 Apr '17

28 Apr '17

(adding the releng list on CC, please keep the reply on the infra list) Hi Everyone, Following up on the thread about pagure on the top of dist-git started a few days ago Ralph Bean, Matthew Prahl and I had a quick meeting just a few minutes ago about the future of pkgdb. Most of the content of the email is based on the notes we took during the meeting, so I hope it is understandable for everyone. I have added some context compared to the original notes, but feel free to ask questions if there are any missing pieces or things that are unclear. The idea is that with pagure becoming a front-end to dist-git, there is less needs for pkgdb. So we tried to list all what pkgdb does and propose solutions for each of these. Let's start with the start: what does pkgdb do? - Store point of contact for a package default assignee on bugzilla - Store commit access for a package commit ACL - Store admin access for a package approveacls ACL - Store CC list for bug reports on the package watchbugzilla ACL - Store CC list for changes made to the package watchcommits ACL - Store new package requests - Store new EPEL branch requests - Allow creating new Fedora branch on demand - Store which packages are orphaned - Store which packages are retired - Store collection/branches status - Store anitya integration flag - Store koshei integration flag - Store critpath status The main idea we have is to rely on pagure and PDC for as much as possible and having a special project on pagure to use as a ticket/queuing system for requests. This project could then be interacted with using a CLI tool. Now going through each of the requirements listed above - Store point of contact for a package (default assignee on bugzilla) - we could use the first committer, alphabetically - we could use the 'owner', but we need pagure to be able to "give" a repo which it currently cannot. - in order to "orphan" a package, we need this. - we could list the default assignee in the yaml file in dist-git - Not ideal since less "self-service" - Store commit access for a package (commit ACL) - Use the list of people with commit or admin access to the package - Store admin access for a package (approveacls ACL) - Handled in pagure directly by the people having admin access to the package - Need a way to prevent groups from having admin access (like we prevented groups from having approveacls in pkgdb) - Store CC list for changes made to the package (watchcommits ACL) - use the pagure "watch" feature. - Store CC list for bug reports on the package (watchbugzilla ACL) - grow the ability in pagure to watch issues versus watch commits. - in the meantime, just use watch commits for both. - Factory 2.0 will handle this - Store new package requests - matt prahl is already cooking up a way to do this using a https://pagure.io/repo-requests/issues/ queue and some scripts. - https://pagure.io/fedrepo_req/pull-request/1 - !!! we have problems using pagure ticket queue here (api tokens, need commit or really admin access...) - other options: - bugzilla - custom made queue - fpaste! - patch pagure to do what we need. -> Add the possibility to select a project in https://pagure.io/settings/token/new and allow there the issue_create, issue_update and issue_comment ACLs -> Add the possibility to set the duration of the token (with an upper limit: 365 days?) (per token with a default in the config file?) -> pingou will handle this - Store new EPEL branch requests - same as above. - for the first seven days, don't show these in pkgdb-admin (or equivalent) unless there are one or more comments from another package maintainer. - Allow creating new Fedora branch on demand - except we want this to be for creating *any* new branch request "like branch 2.0.x" - make a little itty bitty approving service (IBAS) for auto-approving these in a small way. - it should re-use our replacement for pkgdb-admin under the hood so we don't duplicate code. - if some hard-coded set of checks pass then, auto-approve. - checks like "is the EOL under 2 years?" "does the branch name not include any non-ascii characters?" "run it through a swear words dictionary" - otherwise, leave in the queue for manual processing. - Store which packages are orphaned - these are just owned by the orphan user in pagure - step 1, the orphan user is the only committer. - step 2, when pagure grows the ability to "give" a project, then make the orphan user the real owner. - Factory 2.0 may handle this. Let pingou know - Store which packages are retired - in consultation with dgilmore, we don't think we need this anymore. - the dead.package file is sufficient. we still need to propagate blockage to koji (so listen on fedmsg for changes to that file instead of listening for pkgdb message about status change). - Also store this into PDC. it will be represented by the "active" True/False flag on each branch. - Store collection/branches status - this is going to PDC as well (for modularity work, cf the video below) - https://fedorapeople.org/groups/factory2/sprint-014/mprahl-pdc-sla-apis.mp4 - Store anitya integration flag - store this in a yaml/toml file in the dist-git repo - let the consumers - do an http request to retrieve the file - listen to fedmsg to catch changes to this file (and update a local cache based on this) - Store koshei integration flag - store this in a yaml/toml file in the dist-git repo - let the consumers - do an http request to retrieve the file - listen to fedmsg to catch changes to this file (and update a local cache based on this) - Store critpath status - Store this in PDC and make it per branch So with the proposed solution above, everything would be stored in: - the git repo itself - the pagure project of the package - a pagure project for requests - PDC Now pkgdb interacts with a few known applications that will need to be adjusted for this: - the-new-hotness - Cf the point about anitya above - koshei - Cf the point about koshei above - zodbot - will have to query the pagure API to find who has access - bodhi - will have to query the pagure API to find who has access (also, maybe critpath flag?) - FMN.rules (who is maintaining the package) - will have to query the pagure API to find who has access - pkgdb-admin - mprahl is rewriting this to use a ticket queue, somewhere. - pkgdb-cli (here we list all the different command and their description and the possible solution) acl Display ACLs for a given package can be done, with no api keys give Give package(s) according to the specified criteria cannot be done yet. just return a "cannot be done yet" error. list List package according to the specified criteria can be done, with no api keys orphan Orphan package(s) according to the specified criteria cannot be done until "give" is done. this is just a "give" to the orphan user. unorphan Unorphan package(s) according to the specified criteria cannot be done until "give" is done. this is just a "give" to another user. request Request ACLs on package(s) according to the specified criteria to be bikedshed on devel about killing it. threebean says "just use email!" update Update ACLs on package(s) as desired No longer needed since we don't do per branch ACLs anymore branches List the active branches Query PDC pending List the pending ACLs requests can easily be done by querying the pagure project for your tickets with a status=Open (for package/branch requests) for ACLs request, since we would not be tracking them down anymore, it would have to be done by the packagers themselves and after a while likely end up with a non-responsive maintainer procedure. monitoring Get or Set the monitoring of a package can easily get with no api key. (just http call to dist-git). setting it no longer allowed. koschei Get or Set the koschei monitoring of a package can easily get with no api key. (just http call to dist-git). setting it no longer allowed. - bunch of links (long term) I hope these notes are sufficiently clear, if not, we're happy to take any and all questions. I think we covered all bases and this is looking pretty straight forward. What do you think? Thanks for your inputs, Pierre

13 24

Apprentice work day (tenative 2017-05-24)
by Kevin Fenzi 28 Apr '17

28 Apr '17

Greetings. I thought it might be a nice idea to run another apprentice work day. What is an apprentice work day? Well, it's where everyone tries to be around as much as they can on IRC and work on a specific goal with apprentices. I was thinking May 24th might be a good day. It's the middle of the week (wed), and it's when we are in F26 Beta freeze, so hopefully everyone is a bit less busy than normal. For topics I'd propose one of the following (please let me know which one seems best to you or add your own): * Unloved apps day - Look at some of our apps we don't work on or do many releases for anymore and fix bugs, perhaps learn how to do a release, etc. askbot and packages are two apps that leap to mind for this. * Triage day - Everyone goes through our issues on apps/infra/etc and adds labels, asks for more info, fixes dupes, etc. * PR day - like the above issues triage, but PRs. * Lint day - run ansible-lint and clean up things in our ansible repo. Perhaps decide on a standard syntax style and convert things to all use that. Remove uses of things like lineinfile, etc. Thoughts? Is the 24th good for everyone? Does one of these sound better than the others? Do you have some cooler idea? kevin

4 3

Plan for today's Fedora Infrastructure meeting 2017-04-27 18:00 UTC
by Stephen John Smoogen 27 Apr '17

27 Apr '17

The infrastructure team will be having it's weekly meeting tomorrow, 2017-04-27 at 18:00 UTC in #fedora-meeting on the freenode network. We have a gobby document (see: https://fedoraproject.org/wiki/Gobby ) fedora-infrastructure-meeting-next is the document. Please try and review and edit that document before the meeting and we will use it to have our agenda of things to discuss. A copy as of today is included in this email. If you have something to discuss, add the topic to the discussion area with your name. If you would like to teach other folks about some application or setup in our infrastructure, please add that topic and your name to the learn about section. This shared document is for the next fedora infrastructure meeting. = Introduction = We will use it over the week before the meeting to gather status and info and discussion items and so forth, then use it in the irc meeting to transfer information to the meetbot logs. = Meeting start stuff = #startmeeting Infrastructure (2017-04-27) #meetingname infrastructure #topic aloha #chair smooge relrod nirik abadger1999 dgilmore threebean pingou puiterwijk pbrobinson = Let new people say hello = #topic New folks introductions = Status / information / Trivia / Announcements = (We put things here we want others on the team to know, but don't need to discuss) (Please use #info <the thing> - your name) #topic announcements and information #info beta freeze will start 2017-05-17 #info infra hackfest in RDU 2017-05-08 to 2017-05-12 - everyone #info Fedora Infrastructure weathered large outage last Friday. Good job everyone #info mass update/reboot cycle next week (2017-05-02/03) - everyone #info bodhi 2.6.0 released. A few issues so look for 2.6.1 soon - bowlofeggs #info = Things we should discuss = We use this section to bring up discussion topics. Things we want to talk about as a group and come up with some consensus /suor decision or just brainstorm a problem or issue. If there are none of these we skip this section. (Use #topic your discussion topic - your username) #topic jenkins cleanup (https://jenkins.fedorainfracloud.org/view/infrastructure/) - cverna #topic Apprentice work day scheduling/topic - kevin #topic BDR (bidirectional Data Replication) in postgres - kevin = Apprentice office hours = #topic Apprentice Open office hours Here we will discuss any apprentice questions, try and match up people looking for things to do with things to do, progress, testing anything like that. = Learn about some application or gsetup in infrastructure = (This section, each week we get 1 person to talk about an application or setup that we have. Just going over what it is, how to contribute, ideas for improvement, etc. Whoever would like to do this, just add the i/nfo in this section. In the event we don't find someone to teach about something, we skip this section and just move on to open floor.) #topic Learn about: = Meeting end stuff = #topic Open Floor #endmeeting -- Stephen J Smoogen.

1 0

2017-03-21 PHX2 Outage thoughts
by Kevin Fenzi 25 Apr '17

25 Apr '17

Greetings. As everyong likely knows we had a long outage yesterday of our primary datacenter (PHX2). It started around 14:45 UTC and lasted until around 06:45UTC saturday morning. The outage was networking/routing related, but we don't know any further details. Happy to share what we know and can share when investigations are completed. The purpose of this email however is to talk about what we might be able to easily adjust to handle things like this moving forward. Happily these sorts of outages are very rare, but there's a few things I think we can do to make them a bit easier on us. First, lets talk about things that went well: * The mirrorlist containers worked great, so mirrorlist impact was low (but not nothing, see below). * Our distributed proxy system meant things like getfedora.org and other static websites were fine. * Our mirror network + metalinks/mirrorlists meant that people could still get updates, etc just fine. * status.fedoraproject.org was just great and even survived slashdotting (well, thats not all it used to be :) And some things could have been better: * We couldn't take proxy01/10 (The two proxies at phx2) out of dns because we didn't have the dnssec keys available anywhere to write new data out. This meant that sometimes people would hit those in dns and get a timeout and have to retry. * We couldn't reach our offside backups at RDU2. Currently we access them via a VPN that is from PHX2. We could of course have bugged someone to get us access, but they were all busy with the outage. * geoip was down, so might have affected installs. We may be able to move this to a container. * sudo anywhere depends on fas being up, but it was not, so the only way we could get root on vm's was to go to console and login directly as root. We also did a bit of brainstorming on IRC for what we could do if an outage like this happened again and was longer. Without a bunch of new hardware somewhere the buildsystem side of things wouldn't be very practical to bring up anywhere else, but we could bring some non build related services back up in other datacenters. Or at least improve the errors they give. So, IMHO, some action items we might consider: * Create a batcave02.rdu that has all our repos, ansible keys and dns keys. We can then use this to push things out if needed. It shouldn't be too hard to pull from batcave01 or otherwise keep things in sync. * Consider having a tested set of configs for a long outage like this: haproxy/apache could be adjusted to show a outage page for all the down services and we could bring things up more gradually. * Consider a ipa03.rdu freeipa replicant. This would at least allow kerberos auth (but of course with most things that use it down it doesn't matter too much). * Accelerate plans to move away from pam_url for sudo to something that will work when fas is down. kevin

5 4

Pagure on dist-git, the state of things
by Pierre-Yves Chibon 25 Apr '17

25 Apr '17

Good Morning Everyone, I figured it has been a while since I reported progress on making pagure a front-end for dist-git. So here is a small status update. What works: - Well currently pagure is working as a front-end for dist-git in stg: https://src.stg.fedoraproject.org/pagure/ - Hosting repos, browsing them, creating PR works all fine (w/ one , see below) - Features we want to exclude have been turned off (no user management on the pagure side) What does not work: - Syncing the ACLs from pkgdb to gitolite takes ~3 minutes in prod, in stg, the updated script in stg (which sync the ACLs from pkgdb to gitolite and pagure) runs in ~30/35 minutes (if it doesn't crash, which my last run just did) This is a blocker since it means it takes 30 to 35 minutes for someone to get access to their git repo or (worse) their fork - This pagure instance has the same issue as the main one, including a heisenbug we're trying to track but have had no luck reproducing so far :( What needs to be done: - Fix the sync script - Make it *way* faster than it is - Make it creates the project on pagure using the releng user rather than relying on the first contributor it finds in the list of maintainers - Make fedmsg-genacls be triggered on pagure's fork fedmsg message so that we re-generate the gitolite configuration file when someone forks a project (and thus give them access to their fork) - Once above is done: call for more testers In the future: - I think we will want to deprecate pkgdb entirely, so while the work above is important, ideally it won't be there for too long. - With pkgdb out of the loop, we'll need to figure some things out: - Where/How to store the contact info for bugzilla - Not sure relying on pagure's ACLs there is the way to go since we would loose a level of granularity in the ACLs that I know people like and ask for (having commit w/o being on the CC list in bugzilla or being on the CC list w/o being a packager) - How/when to require people be part of the packager group in FAS? - Since one of the idea of pagure is to make it easier for "drive-by" contribution to spec files, requiring to be a packager should only be there for maintainers, but pagure doesn't have this level of information/requirement, so we would need to find something or some place to add this requirement or see if that requirement still stands This is all I can think of for now, I'll update this thread if I come up with more ideas/challenges. Have a nice day, Pierre

6 13

[release] pkgdb2: 2.6.1 and 2.6.2
by Pierre-Yves Chibon 25 Apr '17

25 Apr '17

Good Morning Everyone, Yesterday I prepared and tagged a 2.6.1 release but I messed up with the git history, so I ended up creating a new release 2.6.2 today. Here are their corresponding changelogs: * Tue Apr 25 2017 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 2.6.2-1 - Update to 2.6.2 - Fix git history in 2.6.2 * Mon Apr 24 2017 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 2.6.1-1 - Update to 2.6.1 - Make the mapping in pkgdb-sync-bugzilla script easier to adjust (by placing it in the config file) So 2.6.2 is now running in stg and prod and we should be free of any hotfix :) Happy packaging, Pierre

1 0

[Fedocal] Reminder meeting : Bodhi stakeholder's meeting
by bowlofeggs＠fedoraproject.org 25 Apr '17

25 Apr '17

Dear all, You are kindly invited to the meeting: Bodhi stakeholder's meeting on 2017-04-25 from 15:00:00 to 16:00:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Source: https://apps.fedoraproject.org/calendar/meeting/4671/

1 0

Weekly Koji Infra Tag Report
by Nobody 24 Apr '17

24 Apr '17

This is a list of packages in the various infrastructure koji tags Please check and make sure there are not any that can be removed/dropped epel6-infra (no matching packages) epel7-infra Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- freeipa-ktutils epel7-infra puiterwijk compose-utils epel7-infra ausil fedmsg-beaker-repoupdate epel7-infra tflink anitya epel7-infra jcline the-new-hotness epel7-infra jcline fedocal epel7-infra pingou python-IPy epel7-infra kevin python-robosignatory epel7-infra puiterwijk pdc-updater epel7-infra ralph python-pdc epel7-infra ralph glusterfs epel7-infra kevin mirrormanager2 epel7-infra puiterwijk blockerbugs epel7-infra tflink python-django-jsonfield epel7-infra ralph f23-infra Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- libphutil, f23-infra tflink arcanist, f23-infra tflink phabricator f23-infra tflink phabricator-extension-ipsilonauth f23-infra tflink libphutil f23-infra tflink arcanist f23-infra tflink f24-infra Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- mediawiki-openid f24-infra kevin phabricator-extension-oauth f24-infra tflink python-twill f24-infra codeblock stickynotes2modernpaste f24-infra codeblock python-flask-testing f24-infra codeblock modern-paste f24-infra codeblock mediawiki-skin-fedora f24-infra puiterwijk mediawiki-FedoraBadges f24-infra kevin basset f24-infra puiterwijk phabricator f24-infra tflink mediawiki-Lockdown f24-infra kevin libphutil f24-infra tflink arcanist f24-infra tflink mediawiki-RSS f24-infra kevin mirrormanager2 f24-infra puiterwijk f25-infra Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- python-flask-testing f25-infra codeblock modern-paste f25-infra codeblock python-coveralls f25-infra codeblock mdapi f25-infra pingou basset f25-infra puiterwijk mediawiki-FedoraBadges f25-infra kevin mediawiki-Lockdown f25-infra kevin mediawiki-RSS f25-infra kevin mediawiki-openid f25-infra kevin plus-plus-service f25-infra pingou python-pdc f25-infra ralph python-django-cors-headers f25-infra ralph python-django-rest-framework-composed-permissions f25-infra ralph patternfly1 f25-infra ralph fas f25-infra kevin libphutil, f25-infra tflink arcanist, f25-infra tflink phabricator f25-infra tflink phabricator-extension-ipsilonauth f25-infra tflink libphutil f25-infra tflink arcanist f25-infra tflink python-twill f25-infra codeblock stickynotes2modernpaste f25-infra codeblock mediawiki-skin-fedora f25-infra kevin f26-infra (no matching packages) f27-infra (no matching packages)

1 0

Meeting Agenda Item: Michael O'Brien
by Michael OBrien 19 Apr '17

19 Apr '17

1 0

Bodhi 2.6.0 beta 0
by Randy Barlow 18 Apr '17

18 Apr '17

Bodhi 2.6.0b0 has been deployed to staging[0]. Release notes are available in its docs[1]. Enjoy! [0] https://bodhi.stg.fedoraproject.org/ [1] https://bodhi.stg.fedoraproject.org/docs/release_notes.html#id1

1 0

Weekly Koji Infra Tag Report
by Nobody 17 Apr '17

17 Apr '17

This is a list of packages in the various infrastructure koji tags Please check and make sure there are not any that can be removed/dropped epel6-infra (no matching packages) epel7-infra Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- freeipa-ktutils epel7-infra puiterwijk compose-utils epel7-infra ausil fedmsg-beaker-repoupdate epel7-infra tflink anitya epel7-infra jcline the-new-hotness epel7-infra jcline fedocal epel7-infra pingou python-IPy epel7-infra kevin python-robosignatory epel7-infra puiterwijk pdc-updater epel7-infra ralph python-pdc epel7-infra ralph glusterfs epel7-infra kevin mirrormanager2 epel7-infra puiterwijk blockerbugs epel7-infra tflink python-django-jsonfield epel7-infra ralph f23-infra Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- libphutil, f23-infra tflink arcanist, f23-infra tflink phabricator f23-infra tflink phabricator-extension-ipsilonauth f23-infra tflink libphutil f23-infra tflink arcanist f23-infra tflink f24-infra Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- mediawiki-openid f24-infra kevin phabricator-extension-oauth f24-infra tflink python-twill f24-infra codeblock stickynotes2modernpaste f24-infra codeblock python-flask-testing f24-infra codeblock modern-paste f24-infra codeblock mediawiki-skin-fedora f24-infra puiterwijk mediawiki-FedoraBadges f24-infra kevin basset f24-infra puiterwijk phabricator f24-infra tflink mediawiki-Lockdown f24-infra kevin libphutil f24-infra tflink arcanist f24-infra tflink mediawiki-RSS f24-infra kevin mirrormanager2 f24-infra puiterwijk f25-infra Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- python-flask-testing f25-infra codeblock modern-paste f25-infra codeblock python-coveralls f25-infra codeblock mdapi f25-infra pingou basset f25-infra puiterwijk mediawiki-FedoraBadges f25-infra kevin mediawiki-Lockdown f25-infra kevin mediawiki-RSS f25-infra kevin mediawiki-openid f25-infra kevin plus-plus-service f25-infra pingou python-pdc f25-infra ralph python-django-cors-headers f25-infra ralph python-django-rest-framework-composed-permissions f25-infra ralph patternfly1 f25-infra ralph fas f25-infra kevin libphutil, f25-infra tflink arcanist, f25-infra tflink phabricator f25-infra tflink phabricator-extension-ipsilonauth f25-infra tflink libphutil f25-infra tflink arcanist f25-infra tflink python-twill f25-infra codeblock stickynotes2modernpaste f25-infra codeblock mediawiki-skin-fedora f25-infra kevin f26-infra (no matching packages) f27-infra (no matching packages)

1 0

Re: Subscription lists
by Misha S. 16 Apr '17

16 Apr '17

It is still happening. Error 503 Backend fetch failed Backend fetch failed Guru Meditation: XID: 16073665 Varnish cache server Also, "Automatic responses" settings won't save. "An error occured: A server error occurred. Please contact the administrator.” Also, “Mailman settings” in the profile drop-down won’t open for me. Best regards, Misha https://fedoraproject.org/wiki/User:Shnurapet English / Español / Italiano / Русский

1 0

Re: April status update for Fedora Infrastructure Apprentices
by Aurelie Deromedis 14 Apr '17

14 Apr '17

Hi, I guess my reply it's a bit too late. I've started in a new role (Salesforce) last week and I've been quite busy since then. I guess my apprentice account has been disabled? I hope not tbh. I should be able to get back on track in the next few weeks. Have a good day, Aurelie On 3 April 2017 at 19:04, Kevin Fenzi <kfenzi(a)redhat.com> wrote: > You are getting this email because you are in the 'fi-apprentice' group > in the fedora account system (or are reading this on the infrastructure > list). > > Feel free to reply just directly to me, or cc the infrastructure list > for everyone to see and comment on. > > https://fedoraproject.org/wiki/Infrastructure_Apprentice > > At the first of every month(or so), I am going to be sending out an > email like this one. I would like feedback on how things are going for > you. I'd like to ask for everyone to send me a quick reply with the > following data or anything related you can think of that might help us > make the apprentice program more useful. > > 0. Whats your fedora account system login? > > > 1. Have you logged in and used your fi-apprentice membership to look at > our machines/setup in the last month? Do you plan to? > > > 2. Has it helped you decide any area you wish to focus on or contribute > to more? > > > 3. Have you looked at or been able to work on any of the fi-apprentice > 'easyfix' tickets? > https://pagure.io/fedora-infrastructure/issues?status=Open&tags=easyfix > > > 4. Do you still wish to be a member of the group? If not (for whatever > reason) could you provide any hints to help others down the road? > > > 5. Is there any help or communication or ideas you have that would help > you do any of the above? > > > 6. What do you find to be the hardest part of getting involved? Finding > things to work on? Getting attention from others to help you? Finding > tickets in your interest area? > > > 7. Have you been able to make any weekly irc meetings? Do you find them > helpful or interesting? > > > 8. Have you logged into our Gobby instance and read/seen/added to our > meeting agenda? https://fedoraproject.org/wiki/Gobby > > > 9. Have you used any containers in the last week or so? > (docker, rocket, etc) > > Any other general feedback is also quite welcome, including improvements > to this email, the wiki page, etc. Note that we recently revamped the > getting started and other pages. Please do take a minute to re-read them > and let me know if they are more clear or need further adjustments. > > Any folks I do not hear from in the next week will be removed from the > group. (Note that it's easy to be readded when you have time or whatever > and it's nothing at all personal, we just want to keep the group up to > date with active folks). > > Thanks, and looking forward to your feedback! > > kevin >

2 1

Plans for a new mirrormanager release
by Adrian Reber 14 Apr '17

14 Apr '17

I would like to update MirrorManager to the latest version before the next freeze to get the some of the fixes in git and new features running in production. The biggest change is the change in update-master-directory-list to support master server scanning based on fullfiletimelist-* Basically this remove the need of any I/O on the master mirror as it relies on the information provided in the fullfiletimelist-* files. This reduces the scan time of the master mirror from multiple hours to 20 minutes: https://github.com/fedora-infra/mirrormanager2/pull/207 The changes in this PR are probably hard to review but I tested it in staging and it seems to do the right thing. So if anyone could have a look at this PR to see if there are any obvious errors I could merge this PR and make a release to get it running in production some time before the beta freeze to make sure it works as desired before the freeze. Adrian

1 0

Fwd: reuse space on retrace02 for retrace01
by Miroslav Suchý 13 Apr '17

13 Apr '17

FYI -------- Přeposlaná zpráva -------- Předmět: reuse space on retrace02 for retrace01 Datum: Thu, 13 Apr 2017 16:07:37 +0200 Od: Miroslav Suchý <msuchy(a)redhat.com> Společnost: Red Hat Czech s.r.o. Komu: abrt-devel-list(a)redhat.com Hi, for Fedora we have two servers: retrace01.qa.fedoraproject.org retrace02.qa.fedoraproject.org retrace.fedoraproject.org is alias for retrace01. retrace02 is used for devel purposes. And according my quick search it is not used. Well the only use is for storing daily DB backups. We have daily backups since 2015 and it takes 3.8 TB! We have two other staging servers one for faf and other for retrace. I plan to delete those daily backups. Which will give us 8 TB free space on retrace02. On the retrace01 we are all the time on 99% of used storage so we need some space. I propose to install glusterFS on retrace01,02 and use the concated space on retrace01. This will gave us more space and we would be able to enable retrace for armhpf too. Unless there is some objections I will work on that next week. -- Miroslav Suchy, RHCA Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys

1 0

Fwd: fedmsg error log badges-backend01.phx2.fedoraproject.org
by Justin W. Flory 13 Apr '17

13 Apr '17

Hi all, I had about 150-200 of these emails come in over the last few minutes. Is there any kind of action we can take to quiet these down or get to the bottom of what's causing them? This also happens intermittently outside of the past 30 minutes too. Thanks! -------- Forwarded Message -------- Subject: fedmsg error log badges-backend01.phx2.fedoraproject.org Date: Mon, 10 Apr 2017 23:41:09 -0000 From: fedmsg(a)fedoraproject.org To: sysadmin-badges-members(a)fedoraproject.org Message ------- [2017-04-10 23:41:09][moksha.hub ERROR] ```python Couldn't talk to pkgdb2 for u'mbs/mbs.fedoraproject.org', 404, u'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">\n<title>404 Not Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>\n' ``` Process Details --------------- - host: badges-backend01.phx2.fedoraproject.org - PID: 31829 - name: fedmsg-hub - command: /usr/bin/python /usr/bin/fedmsg-hub - msg_id: 2017-18d2a3e1-2cb0-45a2-97d8-aba549a4e8fb Callstack that lead to the logging statement -------------------------------------------- ```python File "/usr/lib64/python2.7/threading.py", line 784 in __bootstrap self.__bootstrap_inner() File "/usr/lib64/python2.7/threading.py", line 811 in __bootstrap_inner self.run() File "/usr/lib64/python2.7/threading.py", line 764 in run self.__target(*self.__args, **self.__kwargs) File "/usr/lib64/python2.7/site-packages/twisted/python/threadpool.py", line 167 in _worker result = context.call(ctx, function, *args, **kwargs) File "/usr/lib64/python2.7/site-packages/twisted/python/context.py", line 118 in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/lib64/python2.7/site-packages/twisted/python/context.py", line 81 in callWithContext return func(*args,**kw) File "/usr/lib/python2.7/site-packages/moksha/hub/api/consumer.py", line 191 in _work self.consume(message) File "/usr/lib/python2.7/site-packages/fedbadges/consumers.py", line 237 in consume for recipient in badge_rule.matches(msg): File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 236 in matches if not self.criteria.matches(msg): File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 94 in inner return method(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 368 in matches child.matches(msg) for child in self.children File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 66 in <lambda> "not": lambda x: all([not item for item in x]) File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 368 in <genexpr> child.matches(msg) for child in self.children File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 96 in inner log.exception(e) ``` Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 94, in inner return method(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 371, in matches return self.specialization.matches(msg) File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 418, in matches user=expectation['user'], File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 145, in get_pkgdb_packages_for return _getter(user) File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 1053, in decorate should_cache_fn) File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 657, in get_or_create async_creator) as value: File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line 158, in __enter__ return self._enter() File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line 98, in _enter generated = self._enter_create(createdtime) File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line 149, in _enter_create created = self.creator() File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 625, in gen_value created_value = creator() File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 1049, in creator return fn(*arg, **kw) File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 143, in _getter return _get_pkgdb2_packages_for(config, user) File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 166, in _get_pkgdb2_packages_for data = _get_page(1) File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 159, in _get_page username, req.status_code, req.text)) IOError: Couldn't talk to pkgdb2 for u'mbs/mbs.fedoraproject.org', 404, u'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">\n<title>404 Not Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>\n'

2 2

Plan for tomorrow's Fedora Infrastructure meeting 2017-04-13
by Stephen John Smoogen 13 Apr '17

13 Apr '17

The infrastructure team will be having it's weekly meeting tomorrow, 2017-04-13 at 18:00 UTC in #fedora-meeting on the freenode network. We have a gobby document (see: https://fedoraproject.org/wiki/Gobby ) fedora-infrastructure-meeting-next is the document. Please try and review and edit that document before the meeting and we will use it to have our agenda of things to discuss. A copy as of today is included in this email. If you have something to discuss, add the topic to the discussion area with your name. If you would like to teach other folks about some application or setup in our infrastructure, please add that topic and your name to the learn about section. = Introduction = We will use it over the week before the meeting to gather status and info and discussion items and so forth, then use it in the irc meeting to transfer information to the meetbot logs. = Meeting start stuff = #startmeeting Infrastructure (2017-04-13) #meetingname infrastructure #topic aloha #chair smooge relrod nirik abadger1999 dgilmore threebean pingou puiterwijk pbrobinson = Let new people say hello = #topic New folks introductions = Status / information / Trivia / Announcements = (We put things here we want others on the team to know, but don't need to discuss) (Please use #info <the thing> - your name) #topic announcements and information #info hosted03 has had final systems removed #info beta freeze will start 2017-05-17 = Things we should discuss = We use this section to bring up discussion topics. Things we want to talk about as a group and come up with some consensus /suor decision or just brainstorm a problem or issue. If there are none of these we skip this section. (Use #topic your discussion topic - your username) #topic Upcoming Hackathon #info https://fedoraproject.org/wiki/CI_and_Infrastructure_Hackathon_2017 #info core members will be less available. #info focus on modularity #topic Clean out the backlog of tickets #info Put in a day for going through old tickets #info Triage and clean out old tickets #info Goal bring it down to 40 tickets = Apprentice office hours = #topic Apprentice Open office hours Here we will discuss any apprentice questions, try and match up people looking for things to do with things to do, progress, testing anything like that. = Learn about some application or gsetup in infrastructure = (This section, each week we get 1 person to talk about an application or setup that we have. Just going over what it is, how to contribute, ideas for improvement, etc. Whoever would like to do this, just add the i/nfo in this section. In the event we don't find someone to teach about something, we skip this section and just move on to open floor.) #topic Learn about: SSH and where its going. = Meeting end stuff = #topic Open Floor #endmeeting -- Stephen J Smoogen.

1 0

Re: SSH Key types
by Zach Villers 12 Apr '17

12 Apr '17

Thanks Patrick! -- Zach Villers zach(a)znix.site On Wed, Apr 12, 2017, at 02:28 PM, Patrick Uiterwijk wrote: > Hi all, > > As of right now, we are allowing SSH ECDSA and ed25519 keys to be used > in the Fedora Account System. > However, I would *strongly* suggest infrastructure team members to > *not* use either of those, because that would prevent you from > accessing any of the remaining RHEL6 machines[1] we still have. > > The reason this has been changed now is because we shut off hosted03, > which was the final server that non-infra folks have SSH access to. > > Regards, > Patrick > > > [1]: https://fedoraproject.org/wiki/Infrastructure/RHEL6_hosts > _______________________________________________ > infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org > To unsubscribe send an email to > infrastructure-leave(a)lists.fedoraproject.org

1 0

SSH Key types
by Patrick Uiterwijk 12 Apr '17

12 Apr '17

Hi all, As of right now, we are allowing SSH ECDSA and ed25519 keys to be used in the Fedora Account System. However, I would *strongly* suggest infrastructure team members to *not* use either of those, because that would prevent you from accessing any of the remaining RHEL6 machines[1] we still have. The reason this has been changed now is because we shut off hosted03, which was the final server that non-infra folks have SSH access to. Regards, Patrick [1]: https://fedoraproject.org/wiki/Infrastructure/RHEL6_hosts

1 0

Re: Fwd: fedmsg error log badges-backend01.phx2.fedoraproject.org
by Zach Villers 11 Apr '17

11 Apr '17

Justin, I don't know if this is the cause or not, but I will tell you what I've done in case it helps someone. My first commit to badges was figuring out that the script that ran daily to check for the X Year Badger badges wasnt running. The length of the list was so long that the connection timed out just trying to pull down the list of contributors to check. My fix was to just loop over the alphabet and pull in all of the "A" names, all the way to the "Z" names. I imagine that if the script fails to run now, for whatever reason, it does it 26 times. :) A better solution could involve ( and Im doing this from memory, so apologies if this is a wild goose chase ) - having puiterwijk or some other person smart about DBs write a query to pull out all of the badge anniversary dates and put a text file of results somewhere this script could pull in - having the loop check each time through to see if the database responds, waiting in between responses, before giving up - rewriting all of fedmsg in python3 ( maybe dont do this - I think it would be painful ) I haven't been around much lately due to work/life, but I just updated my ssh key. If I haven't been removed from the badges-admin group for being largely absent for several months. (which I believe is the primary reason people are removed) I will try to look at it this week. This would be a good task for a new contributor to look into as well. I don't remember if all the scripts for badges are in ansible, but I imagine they are. Regards, Zach -- Zach Villers zach(a)znix.site On Mon, Apr 10, 2017, at 07:44 PM, Justin W. Flory wrote: > Hi all, > > I had about 150-200 of these emails come in over the last few minutes. > Is there any kind of action we can take to quiet these down or get to > the bottom of what's causing them? This also happens intermittently > outside of the past 30 minutes too. > > Thanks! > > > -------- Forwarded Message -------- > Subject: fedmsg error log badges-backend01.phx2.fedoraproject.org > Date: Mon, 10 Apr 2017 23:41:09 -0000 > From: fedmsg(a)fedoraproject.org > To: sysadmin-badges-members(a)fedoraproject.org > > Message > ------- > [2017-04-10 23:41:09][moksha.hub ERROR] > ```python > Couldn't talk to pkgdb2 for u'mbs/mbs.fedoraproject.org', 404, > u'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">\n<title>404 > Not Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was not > found on the server. If you entered the URL manually please check your > spelling and try again.</p>\n' > ``` > > > Process Details > --------------- > - host: badges-backend01.phx2.fedoraproject.org > - PID: 31829 > - name: fedmsg-hub > - command: /usr/bin/python /usr/bin/fedmsg-hub > - msg_id: 2017-18d2a3e1-2cb0-45a2-97d8-aba549a4e8fb > > Callstack that lead to the logging statement > -------------------------------------------- > ```python > File "/usr/lib64/python2.7/threading.py", line 784 in __bootstrap > self.__bootstrap_inner() > File "/usr/lib64/python2.7/threading.py", line 811 in __bootstrap_inner > self.run() > File "/usr/lib64/python2.7/threading.py", line 764 in run > self.__target(*self.__args, **self.__kwargs) > File > "/usr/lib64/python2.7/site-packages/twisted/python/threadpool.py", line > 167 in _worker > result = context.call(ctx, function, *args, **kwargs) > File "/usr/lib64/python2.7/site-packages/twisted/python/context.py", > line 118 in callWithContext > return self.currentContext().callWithContext(ctx, func, *args, **kw) > File "/usr/lib64/python2.7/site-packages/twisted/python/context.py", > line 81 in callWithContext > return func(*args,**kw) > File "/usr/lib/python2.7/site-packages/moksha/hub/api/consumer.py", > line 191 in _work > self.consume(message) > File "/usr/lib/python2.7/site-packages/fedbadges/consumers.py", line > 237 in consume > for recipient in badge_rule.matches(msg): > File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 236 > in matches > if not self.criteria.matches(msg): > File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 94 in > inner > return method(self, *args, **kwargs) > File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 368 > in matches > child.matches(msg) for child in self.children > File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 66 in > <lambda> > "not": lambda x: all([not item for item in x]) > File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 368 > in <genexpr> > child.matches(msg) for child in self.children > File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 96 in > inner > log.exception(e) > ``` > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 94, > in inner > return method(self, *args, **kwargs) > File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 371, > in matches > return self.specialization.matches(msg) > File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 418, > in matches > user=expectation['user'], > File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 145, > in get_pkgdb_packages_for > return _getter(user) > File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line > 1053, in decorate > should_cache_fn) > File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line > 657, in get_or_create > async_creator) as value: > File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line > 158, in __enter__ > return self._enter() > File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line > 98, in _enter > generated = self._enter_create(createdtime) > File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line > 149, in _enter_create > created = self.creator() > File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line > 625, in gen_value > created_value = creator() > File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line > 1049, in creator > return fn(*arg, **kw) > File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 143, > in _getter > return _get_pkgdb2_packages_for(config, user) > File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 166, > in _get_pkgdb2_packages_for > data = _get_page(1) > File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 159, > in _get_page > username, req.status_code, req.text)) > IOError: Couldn't talk to pkgdb2 for u'mbs/mbs.fedoraproject.org', 404, > u'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">\n<title>404 > Not Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was not > found on the server. If you entered the URL manually please check your > spelling and try again.</p>\n' > > _______________________________________________ > infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org > To unsubscribe send an email to > infrastructure-leave(a)lists.fedoraproject.org > Email had 1 attachment: > + signature.asc > 1k (application/pgp-signature)

1 0

[PATCH] Fix a typo in Rawhide DNF nightly cron job
by Lubomír Sedlář 10 Apr '17

10 Apr '17

The variable needs to be called TMPDIR, not TPMDIR. Signed-off-by: Lubomír Sedlář <lsedlar(a)redhat.com> --- roles/releng/files/rawhide | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/releng/files/rawhide b/roles/releng/files/rawhide index 62f4333..38e374d 100644 --- a/roles/releng/files/rawhide +++ b/roles/releng/files/rawhide @@ -1,4 +1,4 @@ # rawhide compose MAILTO=releng-cron(a)lists.fedoraproject.org 15 5 * * * root TMPDIR=`mktemp -d /tmp/rawhide.XXXXXX` && cd $TMPDIR && git clone https://pagure.io/pungi-fedora.git && cd pungi-fedora && LANG=en_US.UTF-8 ./nightly.sh && sudo -u ftpsync /usr/local/bin/update-fullfiletimelist -l /pub/fedora-secondary/update-fullfiletimelist.lock -t /pub fedora fedora-secondary -15 17 * * * root TPMDIR=$(mktemp -d /tmp/rawhide-dnf.XXXXXX) && cd $TMPDIR && git clone https://pagure.io/pungi-fedora.git && cd pungi-fedora && LANG=en_US.UTF-8 ./nightly-dnf.sh +15 17 * * * root TMPDIR=$(mktemp -d /tmp/rawhide-dnf.XXXXXX) && cd $TMPDIR && git clone https://pagure.io/pungi-fedora.git && cd pungi-fedora && LANG=en_US.UTF-8 ./nightly-dnf.sh -- 2.9.3

2 1

Review for PR on ansible
by Paul W. Frields 10 Apr '17

10 Apr '17

I'm trying to add to the filter on the ircbot for Magazine so it doesn't output anything that concerns Council in Pagure. Is this right, or even sane? * * * diff --git a/roles/fedmsg/irc/templates/ircbot.py b/roles/fedmsg/irc/templates/ircbot.py index 2b2e0c3..0dcb0e1 100644 --- a/roles/fedmsg/irc/templates/ircbot.py +++ b/roles/fedmsg/irc/templates/ircbot.py @@ -446,12 +446,13 @@ config = dict( nickname='fm-magazine', {% endif %} channel='#fedora-magazine', - # If the word modularity appears in any message, forward it. + # If the word magazine appears in any message, forward it. filters=dict( topic=[ '^((?!(pagure|planet|badges|fas.group|mailman|meetbot\.meeting)).)*$', ], - body=['^((?!(magazine|Magazine)).)*$'], + body=['^((?!(magazine|Magazine)).)*$', + "u'namespace': u'Fedora-Council'"], ), ), -- Paul W. Frields http://paul.frields.org/ gpg fingerprint: 3DA6 A0AC 6D58 FEC4 0233 5906 ACDB C937 BD11 3717 http://redhat.com/ - - - - http://pfrields.fedorapeople.org/ The open source story continues to grow: http://opensource.com

2 1

Looking for GSoC Mentor
by Radka Janekova 10 Apr '17

10 Apr '17

Hey guys, I have two netcore projects but can officially mentor but one and my original 2nd mentor kinda bailed on me, so as the subject says... The project in question without mentor is an ASP.NET Core website that would be checking users fas groups using ipsilon - that is why I'm poking infra :] What would be expected of you - some help with ipsilon side of things (I would still cover the code and deployment, etc...) and essentially being the official primary mentor for the project. Time requirements would be a LOT smaller than with regular GSoC project, probably less than an hour a week. There are two students that could be up for this project: https://fedoraproject.org/wiki/GSOC_2017/Student_Application_stephenonnen https://fedoraproject.org/wiki/GSOC_2017/Student_Application_spytec Best regards, Radka ------------------------------ *Radka Janeková* *radka.janek(a)redhat.com <radka.janek(a)redhat.com>*

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure April 2017