#fedora-meeting: Infrastructure (2015-03-05)
Meeting started by nirik at 18:00:17 UTC. The full logs are available at
* aloha (nirik, 18:00:17)
* New folks introductions / Apprentice feedback (nirik, 18:00:17)
* Information / status updates from the last week (nirik, 18:06:15)
* Trying new meeting process this week with shared document - kevin
* Figured out why squid smp mode wasnt working and posted to list -
kevin (nirik, 18:06:26)
* Ipsilon is now in staging, please test and give me your problems -
puiterwijk (nirik, 18:06:26)
* A large number of bug/RFEs have been coming in on anitya,
the-new-hotness, and fmn. Any help appreciated. - ralph (nirik,
* reminder: we are in Fedora 22 Alpha Freeze - kevin (nirik,
* worked on getting gobby back in shape - kevin (nirik, 18:06:29)
* made progress on new openstack cloud and kept old one stumbling
along - kevin and puiterwijk (nirik, 18:06:31)
* nagios alerts this week - http://ur1.ca/jr7j4
- kevin (nirik,
* moved osu hosts around, should help nagios alerting - kevin (nirik,
* fixed issue with varnish caching old wiki pages - puiterwijk
* progit moving along but the more tester the better - pingou (nirik,
* fedmenu prototype nearing completion, feedback requested
- ralph (nirik, 18:06:41)
* New meeting process (nirik, 18:06:56)
* Fedorahosted (nirik, 18:12:00)
* will move hosted to rhel7 and keep trac and current setup for now.
* will work on progit to groom it to replace hosted someday when it
grows up. (nirik, 18:17:55)
* Learn about.... "the-new-hotness" (nirik, 18:25:09)
* LINK: http://fedoraproject.org/easyfix/
we have some on github
* Open Floor (nirik, 18:46:01)
* we should investigate doing a docker registery. (nirik, 18:52:18)
Meeting ended at 18:56:43 UTC.
Action Items, by person
People Present (lines said)
* nirik (105)
* threebean (38)
* pingou (27)
* dgilmore (26)
* puiterwijk (11)
* Cydrobolt (11)
* smooge (11)
* oddshocks (7)
* jsmith (5)
* zodbot (4)
* langdon (4)
* pbrobinson (3)
* mirek-hm (3)
* lmacken (2)
* andreasch (1)
* tflink (1)
* relrod (1)
* mclasen_ (1)
* danofsatx (1)
* roshi (1)
* mdomsch (0)
* abadger1999 (0)
18:00:17 <nirik> #startmeeting Infrastructure (2015-03-05)
18:00:17 <zodbot> Meeting started Thu Mar 5 18:00:17 2015 UTC. The chair is nirik.
Information about MeetBot at http://wiki.debian.org/MeetBot
18:00:17 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:17 <nirik> #meetingname infrastructure
18:00:17 <zodbot> The meeting name has been set to 'infrastructure'
18:00:17 <nirik> #topic aloha
18:00:17 <nirik> #chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch
threebean pingou puiterwijk
18:00:17 <zodbot> Current chairs: abadger1999 dgilmore lmacken mdomsch nirik pingou
puiterwijk relrod smooge threebean
18:00:17 <nirik> #topic New folks introductions / Apprentice feedback
18:00:25 <nirik> morning everyone.
18:00:35 <threebean> hey hey
18:00:43 * mirek-hm is here
18:00:48 <puiterwijk> (UGT) morning everyone
18:00:55 <pingou> Morning
18:00:59 <Cydrobolt> morning
18:01:02 * danofsatx is lurking about in the vicinity
18:01:03 * lmacken
18:01:24 <andreasch> hi
18:01:51 <nirik> we have a gobby document up. See
18:01:54 * tflink is here
18:01:57 * Cydrobolt is here, but may be distracted
18:02:20 <nirik> also http://gobby.fedoraproject.org/cgit/.git/
18:02:58 * oddshocks here
18:03:01 <Cydrobolt> installing gobby
18:03:01 <smooge> hello. sorry was playing with gobby/sobby/obby
18:03:04 <nirik> any new folks or apprentices with questions?
18:03:48 <nirik> In 2 minutes or so I will dump all the #infos from the gobby doc
here for meetbot to digest. ;)
18:03:58 <nirik> feel free to add any before then...
18:04:03 <Cydrobolt> nirik, first time able to catch this meeting
18:04:10 <nirik> hey Cydrobolt. welcome.
18:04:17 <Cydrobolt> :)
18:06:15 <nirik> #topic Information / status updates from the last week
18:06:23 <nirik> <flood coming>
18:06:26 <nirik> #info Trying new meeting process this week with shared document -
18:06:26 <nirik> #info Figured out why squid smp mode wasnt working and posted to
list - kevin
18:06:26 <nirik> #info Ipsilon is now in staging, please test and give me your
problems - puiterwijk
18:06:27 <nirik> #info A large number of bug/RFEs have been coming in on anitya,
the-new-hotness, and fmn. Any help appreciated. - ralph
18:06:27 <nirik> #info reminder: we are in Fedora 22 Alpha Freeze - kevin
18:06:29 <nirik> #info worked on getting gobby back in shape - kevin
18:06:31 <nirik> #info made progress on new openstack cloud and kept old one
stumbling along - kevin and puiterwijk
18:06:33 <nirik> #info nagios alerts this week - http://ur1.ca/jr7j4
18:06:35 <nirik> #info moved osu hosts around, should help nagios alerting - kevin
18:06:37 <nirik> #info fixed issue with varnish caching old wiki pages - puiterwijk
18:06:39 <nirik> #info progit moving along but the more tester the better - pingou
18:06:41 <nirik> #info fedmenu prototype nearing completion, feedback requested
18:06:56 <nirik> #topic New meeting process
18:07:06 <dgilmore> hola
18:07:12 * pbrobinson waves
18:07:15 <nirik> so this is all new, what do folks think? how can we adjust it or
should we just go back to the old way?
18:07:20 <nirik> welcome dgilmore and pbrobinson. :)
18:07:25 * roshi lurks
18:07:30 * pingou ok with trying a little longer
18:07:36 <oddshocks> +1
18:07:44 <puiterwijk> yeah, +1 on trying at least some more time
18:07:45 <pingou> I thing I was thinking is that it makes it easier for people to
not be present
18:07:52 <mirek-hm> nirik: +1 it will be faster (although maybe little bit async)
18:07:55 <nirik> It seems like folks don't remember to update the shared doc
really... but not sure how to fix that
18:07:59 <pingou> I just dump my info on gobby and then I can go do something else
18:08:05 <threebean> heh, I think I like it already.
18:08:09 <nirik> pingou: yeah. unless there's a discussion item?
18:08:14 <nirik> that you care about
18:08:26 <pingou> nirik: yes, but easily missed, especially if added at the last
18:08:38 <smooge> no no no.. he just agrees to take the work from the item he
18:08:51 <nirik> true. Perhaps those items should be only added until the day
18:08:55 <Cydrobolt> it sounds interesting
18:09:24 <nirik> gobby isn't the best thing in the world either. ;( But it's
free and we have it now...
18:09:35 * jsmith shows up late and lurks
18:09:59 <mirek-hm> ad OpenStack - all services are now available using SSL and only
SSL ports are exported via endpoints. Everything works using command line tools, but when
I start VM I get error, therefore it is still not finished
18:10:15 <pingou> arf
18:10:17 <pingou> so close
18:10:30 <nirik> mirek-hm: I was going to look at vnc later today perhaps... and I
can look at the vm error.
18:10:45 <nirik> ok, so sounds like folks are ok to keep trying and adjusting this
new format then?
18:11:15 * relrod here late, sorry!
18:11:23 <Cydrobolt> hi relrod
18:11:56 <nirik> ok, moving on
18:12:00 <nirik> #topic Fedorahosted
18:12:11 <nirik> sounds like we had some good discussion on list about this...
18:12:29 <nirik> and the concensus was mostly to just keep the current setup, but
move progit forward as we can
18:13:16 <nirik> I guess we should move it to rhel7 though...
18:13:25 <Cydrobolt> nirik, do you have a link to progit?
18:13:37 <Cydrobolt> I can only find information about the book
18:13:57 <puiterwijk> nirik: I can take a look at trac on upgrading our current
traci nstances to the new trac
18:14:09 <pingou> Cydrobolt: http://184.108.40.206/progit/issue/47
18:14:18 <puiterwijk> (so packaging requires extensions etc)
18:14:29 <pingou> btw, everybody is welcome to weight in that ticket as well ^ :)
18:14:37 <Cydrobolt> ah, we are developing it ourselves?
18:14:42 <nirik> puiterwijk: so, I looked into this. looks like 0.12.x is almost
dead... they might do one more release, but thats it.
18:15:04 <puiterwijk> nirik: yeah, I saw the summary on your email. I think we want
to look if 1.X is doable
18:15:05 <nirik> so, 1.0 seems like the best bet. It's unlikely to live for 10
18:15:10 <Cydrobolt> interesting
18:15:32 <nirik> but I think we can just package it as 'trac' in epel7 and
then when it dies we can package 1.2 as trac12 or whatever.
18:15:33 <puiterwijk> right, but I'm hoping the changes in the X-series will be
18:15:38 <nirik> ie, not worry about the parallel right now
18:15:41 <puiterwijk> yeah
18:15:48 <pingou> \ó/
18:15:56 <nirik> in which case, we don't need reviews, just branching and
building and testing
18:16:13 <puiterwijk> yeah
18:16:34 <puiterwijk> so I'd be willing to look into that next week, and see if
I can get an ansible playbook for hosted
18:16:49 <nirik> cool. I can build up the stack if you want to work on the
18:16:52 <nirik> or vice versa.
18:17:17 <puiterwijk> let's discuss that after the meeting, as it depends on
where the stack is at this moment I guess
18:17:19 <nirik> I already do have the trac maintainers go ahead to branch
18:17:23 <nirik> sure.
18:17:41 <nirik> #info will move hosted to rhel7 and keep trac and current setup for
18:17:55 <nirik> #info will work on progit to groom it to replace hosted someday
when it grows up.
18:18:18 <nirik> ok, any other discussion items before we move on to the 'learn
18:19:03 <smooge> I do have a question
18:19:17 <nirik> fire away. ;)
18:19:28 <smooge> will we be putting the trac we use on the el7 boxes in epel or
18:19:43 <nirik> I was assuming epel7...
18:19:53 <nirik> no real reason not to IMHO.
18:20:16 <smooge> OK but I saw that there were massive updates to trac from what we
are using.. I didn't know if we later updated to that trac what the effect would be
18:20:33 <smooge> eg the mediawiki doom
18:20:50 <nirik> well, not sure how massive it is really... they changed their
numbering... 0.11, then 0.12, then 1.0
18:21:09 <nirik> 1.x where x is odd is devel release, even is 'stable'
18:21:09 <smooge> or we make that some one else's problem if they want to update
it (eg pingou because he didn't check into gobby)
18:21:50 <pingou> smooge: I am actually :)
18:21:55 <nirik> well, from 0.12 to 1.0 is not much:
18:22:12 <nirik> from 1.0 to 1.2? we don't know. they were talking about
releasing 1.2 later this year sometime and try and do yearly releases.
18:23:12 <oddshocks> The devel/stable numbering system sounds odd to me :P
18:23:13 <nirik> so, I think it's ok to put 1.0 in epel7 as 'trac' and
then if 1.2 is too much pain, we can make trac12, but hopefully we can just update a
18:23:40 <smooge> ok sorry to sidetrack this. I just wanted to get an idea of me to
package it as track012
18:24:13 <nirik> smooge: yeah, I think the only reason we would do that is if they
were going to keep maintaining it... which it sounds like... they are not.
18:24:26 <nirik> oddshocks: the kernel used to do that, but then abandoned it. ;)
18:24:36 <smooge> of course not
18:24:45 * oddshocks TIL
18:25:09 <nirik> #topic Learn about.... "the-new-hotness"
18:25:14 <nirik> threebean: you're up. ;)
18:25:16 <threebean> yay
18:25:33 <threebean> So, 'the-new-hotness' is one of the three parts of our
new upstream release monitoring setup
18:25:45 <threebean> The project page is here
18:26:20 <threebean> And, in short, it's the "Fedora-specific" cohort
to the "distro-agnostic" release-monitoring.org
18:26:48 <pingou> \ó/
18:26:56 <threebean> the-new-hotness, like the fedbadges backend and the
notifications backend, is a fedmsg 'consumer' plugin that runs as a part of the
18:27:08 * oddshocks thinks threebean just likes saying "the-new-hotness"
18:27:12 <threebean> I do
18:27:17 <oddshocks> :)
18:27:20 <threebean> It is.. fun. :)
18:27:28 <pbrobinson> :-P
18:27:29 <nirik> if something else hot comes along will this be the old hotness?
18:27:40 <threebean> I can't wait..
18:27:40 <nirik> :)
18:27:43 <dgilmore> nirik: the older new hotness
18:28:08 <threebean> at its core, this thing takes care of four distinct tasks:
18:28:32 <threebean> 1) when anitya notices a new upstream release is available, we
try to file a ticket in bugzilla if pkgdb says its okay.
18:28:50 <threebean> if that succeeds, kick off a scratch build if possible and
remember the task id.
18:29:16 <threebean> 2) when koji tasks complete, if we were the one to kick them
off in the first place, then go followup on the bug we previously filed with the status of
18:29:41 <threebean> 3) when "real" koji builds complete for rawhide,
followup on open bugs if we can find them.
18:30:06 <threebean> 4) when a new package is added to pkgdb, try to add it to
anitya with some reasonable defaults.
18:30:20 * mclasen_ considers any automated process that ends in filing a bug to be
18:30:31 * jsmith saw steps 1-3 work today, and was very pleased with the results :-)
18:31:10 <jsmith> mclasen_: In many cases, there's already going to be an open
bug saying that a new release version is available -- this just adds to that ticket, if it
can find it
18:32:08 <threebean> mclasen_: sure, and we provide a flag in pkgdb to disable this
whole process. it is the 'monitoring' tab on the upper right-hand side of the
package view page.
18:32:11 <nirik> Ideally it would be nice if we could use our staging koji for the
scratch builds... but we need to make that work again and add some arm builders to it
18:33:09 <lmacken> #4, awesome.
18:33:22 <dgilmore> nirik: it would be kinda nice for it to commit to staging git
and lookaside and do a real build
18:33:33 <threebean> I'd add that there are some dev instructions in the README
and defaults that point at https://partner-bugzilla.redhat.com
, so you can hack on it
without fear of accidentally filing real bugs on the production bugzilla.
18:33:39 <dgilmore> nirik: then we will have content to test composes in stage
18:33:42 <nirik> dgilmore: yeah, even better
18:34:23 <dgilmore> the way we setup staging its easy to get builds
18:34:27 <nirik> or perhaps it could do both... stg stuff and the current prod
stuff... most people wouldn't need to care about the stg part, but we could use that
for test composes, etc
18:34:31 <dgilmore> but there is nothing for doing composing
18:35:26 <dgilmore> anyway just an idea
18:35:57 <threebean> yeah, auto-committing to dist-git (staging or not) might be
18:36:01 <pingou> threebean: did we turn off the stg instance?
18:36:06 <pingou> of the-new-hotness
18:36:19 <threebean> in that, we still rely on packagers to review the content of
new upstream releases. what if the license changes? what if new non-free content is
18:36:37 <dgilmore> threebean: sure
18:36:52 <threebean> pingou: it is technically still running, it is just not
listening to the production messages from anitya anymore.. and is therefore not doing
18:37:02 <pingou> ok
18:37:52 <threebean> re-establishing that link would just involve uncommenting this
18:38:19 * langdon just came here to say "dock-ah"
18:38:24 <nirik> ha.
18:38:55 <smooge> sorry not enough walsh in it
18:39:01 <smooge> please try again
18:39:15 <nirik> threebean: some more thoughts (might be too hard): could we have it
add new stuff to packages? instead of the packages app regening? or I guess that could be
done by revamping packages...
18:39:47 <threebean> yeah, that might start clustering too many varied
responsibilities in this one service (imo)
18:39:54 <pingou> +1 there
18:39:55 * nirik nods
18:40:03 <threebean> we could rename it from the-new-hotness to
18:40:13 <threebean> :P
18:40:18 <nirik> it's the new *
18:40:20 <pingou> but threebean were pondering how we could revamp packages and
update it via fedmsg
18:40:21 <threebean> ha
18:40:24 <pingou> 42?
18:40:24 * threebean nods
18:40:36 <threebean> yeah, revamped fedora-packages should probably work in a
18:40:36 <oddshocks> It's the Unix philosophy, after all, to create a program to
18:40:41 <pbrobinson> or all-the-shit ;-)
18:41:02 <threebean> we do have some open bugs, if people are interesting in hacking
on it: https://github.com/fedora-infra/the-new-hotness/issues/
18:41:03 <nirik> anyhow, thanks threebean. Anything else on this app?
18:41:13 <nirik> cool,
18:41:21 <threebean> this one is particularly nasty, and needs fixed sooner than
18:41:32 <pingou> http://fedoraproject.org/easyfix/
we have some on github
18:41:44 <threebean> i.e., we can report to packagers that the scratch build is all
good, when really it mistakenly is a rebuild of an old version.
18:41:59 <nirik> yeah, nasty corner case
18:42:06 <threebean> (that's all I have)
18:42:18 <pingou> threebean: we could do a fedpkg source just after getting the spec
file and see if it is different from the once we get after updating the spec
18:42:37 <pingou> or faster: check the content of the sources file
18:43:01 <nirik> well, the sources will have the old/current one always right?
18:43:01 <pingou> and see if the new sources are different from the content of the
current sources file
18:43:05 <threebean> pingou: +1. that's the best approach I've got too. a
hash of the sources.
18:43:09 <nirik> yeah.
18:43:34 <threebean> as mentioned in the ticket, github tarballs always have
different hash sums, since they're generated on demand. which.. messes all that up.
18:43:44 <nirik> ah github, ;(
18:43:57 <threebean> a corner case for the corner case.
18:44:02 <puiterwijk> threebean: it gets better... they even have different sizes,
though the content is the same! :D
18:44:06 <nirik> sharp corner. ;)
18:44:25 <nirik> reproducability? naw. don't need that
18:44:32 <pingou> threebean: ignore github ? :)
18:44:39 <nirik> I guess you could unpack it and diff -Nur ? :)
18:44:48 * threebean notes that.
18:45:44 <nirik> big hassle tho. I guess only for github
18:45:57 <nirik> ok, on to...
18:46:01 <nirik> #topic Open Floor
18:46:19 <dgilmore> nirik: I have one thing
18:46:19 <nirik> the floor is open. Just like Fedora and our default. :)
18:46:34 <nirik> go dgilmore
18:46:51 <dgilmore> I have built a docker base image for arm
18:47:16 <dgilmore> and as the upstream docker registry does not support arches
18:47:22 <nirik> :(
18:49:04 <dgilmore> I am thinking of setting up a registry
18:49:05 <nirik> you are suggesting we run a registry?
18:49:10 <dgilmore> since its all open source
18:49:20 <nirik> ok. Not at all sure whats involved there. I think it's actually
a flask app
18:49:23 <dgilmore> I think it would be nice for use to do so
18:49:42 <dgilmore> but only for arm and maybe other arches if teh secondaries get
18:49:59 <nirik> is there any plan for i686 ones?
18:50:09 <dgilmore> I probably can not commit to maintaining it. but maybe we can
get someone to
18:50:16 <dgilmore> nirik: I think we shoudl
18:50:18 <dgilmore> should
18:50:44 <dgilmore> maybe we support x86_64 also, but still push things to upstream
18:50:44 <nirik> ok. I'd say bring it up on the list and we can see if we can
find someone to investigate it and propose a RFR, etc.
18:50:50 * nirik nods. yeah.
18:50:54 <dgilmore> It needs some investigation
18:51:01 <nirik> doesn't upstream have some other weird stuff like no signing?
18:51:05 <dgilmore> just wanted to put the idea out there
18:51:20 <dgilmore> not really sure
18:51:51 <dgilmore> upstream is kinda weird
18:51:58 <dgilmore> and we should be part of it
18:52:07 <nirik> ok. cool.
18:52:15 <dgilmore> and maybe us doing something will get them to be better
18:52:18 <nirik> #info we should investigate doing a docker registery.
18:52:42 <threebean> Unrelated question: When is the alpha freeze expected to
18:52:57 <jsmith> threebean: When it's fully baked?!?
18:53:00 * jsmith ducks and hides
18:53:02 <langdon> nirik, (et al) .. envs&stacks is having much discussion about
docker-build & docker-reg .. just fyi
18:53:30 <nirik> threebean: well, next wed if we ship next tuesday, but thats still
up in the air. ;(
18:53:48 <nirik> langdon: cool. Good to know. Perhaps someone from there could tell
us whats involved in running a registery?
18:54:20 <nirik> ok, anything else open floor wise?
18:54:28 <pingou> I'll be afk next week
18:54:32 <pingou> most of the week
18:54:47 <pingou> available by phone and I'll probably check my emails once or
18:54:48 <nirik> cool. :) enjoy.
18:54:52 <threebean> pingou: enjoy! :)
18:54:57 <pingou> will do :)
18:55:08 <langdon> nirik, well.. i think there are a lot of considerations on the
'what does it do" and "what is it for" ... as well as the "running
18:55:50 <nirik> langdon: indeed. we definitely want stakeholders involved. I mean
we could set it up perhaps and let them decide other parts, but sometimes setting up feeds
into what you can or can't do after it's setup
18:56:03 <langdon> nirik, right
18:56:34 <nirik> cool.
18:56:41 <nirik> ok, thanks for coming everyone!
18:56:43 <dgilmore> langdon: well we would want to do something to support what
18:56:43 <nirik> #endmeeting