1:48 p.m.
============================================
#fedora-meeting: Infrastructure (2014-01-23)
============================================
Meeting started by nirik at 19:02:52 UTC. The full logs are available at
http://meetbot.fedoraproject.org/fedora-meeting/2014-01-23/infrastructure...
.
Meeting summary
---------------
* welcome y'all (nirik, 19:02:52)
* New folks introductions and Apprentice tasks. (nirik, 19:04:45)
* Applications status / discussion (nirik, 19:08:29)
* fedorahosted.org/reviewboard is back up with openid (thanks to
puiterwijk!). Still some issues to work out, but hopefully we can
look at it again and see if it's useful now. (nirik, 19:09:25)
* fedmsg notifications hopefully deployed by end of next week (nirik,
19:19:50)
* Sysadmin status / discussion (nirik, 19:30:11)
* lots of ansible cleanup this week (nirik, 19:30:26)
* filed 3 new sysadmin easyfix tickets this morning. :) apprentices
take a look and see if any one of them interests you. (nirik,
19:35:00)
* new machines for 4thQ still in limbo as far as I know. (nirik,
19:38:12)
* netapp has been slow lately, storage folks are working on it.
(nirik, 19:38:27)
* Smooge will be heading out to our main datacenter in mid-feb.
(nirik, 19:38:50)
* Upcoming Tasks/Items (nirik, 19:39:29)
* LINK: https://apps.fedoraproject.org/calendar/list/infrastructure/
(nirik, 19:39:30)
* Open Floor (nirik, 19:40:39)
* LINK: http://infrastructure.fedoraproject.org/cgit/ansible.git/tree/
(nirik, 19:44:50)
Meeting ended at 19:47:25 UTC.
Action Items
------------
Action Items, by person
-----------------------
* **UNASSIGNED**
* (none)
People Present (lines said)
---------------------------
* nirik (87)
* sgallagh (17)
* threebean (15)
* mariocav (14)
* adimania (12)
* lmacken (12)
* johe2 (7)
* willo (5)
* zodbot (5)
* relrod (4)
* abadger1999 (3)
* randomuser (2)
* dgilmore (2)
* pjones (1)
* oddshocks (1)
* puiterwijk (0)
* smooge (0)
* mdomsch (0)
* pingou (0)
--
19:02:52 <nirik> #startmeeting Infrastructure (2014-01-23)
19:02:52 <zodbot> Meeting started Thu Jan 23 19:02:52 2014 UTC. The chair is nirik.
Information about MeetBot at http://wiki.debian.org/MeetBot.
19:02:52 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
19:02:52 <nirik> #meetingname infrastructure
19:02:52 <nirik> #topic welcome y'all
19:02:52 <nirik> #chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch
threebean pingou puiterwijk
19:02:52 <zodbot> The meeting name has been set to 'infrastructure'
19:02:52 <zodbot> Current chairs: abadger1999 dgilmore lmacken mdomsch nirik pingou
puiterwijk relrod smooge threebean
19:03:06 * relrod waves
19:03:07 * lmacken
19:03:14 * threebean is here
19:03:16 * adimania says hi
19:03:18 * oddshocks is here but has to split in around 40 minutes
19:03:18 <mariocav> hello
19:03:25 * johe2 is here
19:03:36 * abadger1999 is here but also paying attention to the board meeting
19:04:15 * willo mubbles morning
19:04:45 <nirik> #topic New folks introductions and Apprentice tasks.
19:05:00 <nirik> any new folks like to introduce themselves? or apprentices with
questions or comments?
19:05:08 <mariocav> yep, me!
19:05:12 <mariocav> ^^
19:05:30 <threebean> mariocav: hi there :)
19:05:46 * dgilmore is here but will be heading home soon
19:05:47 <lmacken> (hah, my local fedmsg-notify popped up the meeting topic change
before it appeared in my irc client)
19:05:56 <mariocav> well, my name is Mario, I'm from Barcelona.
19:06:05 <nirik> mariocav: welcome. Are you more interested in sysadmin type things?
or applications development ? or both?
19:06:12 <nirik> lmacken: yeah, that happens to me sometimes too. ;)
19:06:14 <mariocav> I really don't know
19:06:16 <mariocav> I mean
19:06:31 <mariocav> What I'm doing these days is to read wikis and get
information about the services you cover.
19:06:52 <nirik> great.
19:06:52 <mariocav> My experience is very limited (html/css/php/js, c++, and admin
of 2 centos servers that offer basic web services - http/dns and mail)
19:07:07 <mariocav> so I don't thing I'll start contributing shortly
19:07:23 <mariocav> Anyways, if someone has any suggestion on what he/she would to
next... I'll appreciate
19:07:29 <mariocav> think*
19:07:30 <nirik> well, welcome and do ask questions as you run into them. :) Our
usual channels (#fedora-admin, #fedora-apps, #fedora-noc are good for that ongoing)
19:07:40 <mariocav> thank you
19:07:42 * randomuser is lurking around. reading infra docs at a very slow pace
19:07:57 <mariocav> well, that's it
19:07:59 <mariocav> :)
19:08:01 <nirik> randomuser: cool. :)
19:08:07 <nirik> welcome again!
19:08:22 <randomuser> thanks
19:08:29 <nirik> #topic Applications status / discussion
19:08:39 <nirik> any applications news on plans upcoming?
19:08:53 <nirik> I've one thing to sort of note:
19:09:25 <nirik> #info fedorahosted.org/reviewboard is back up with openid (thanks
to puiterwijk!). Still some issues to work out, but hopefully we can look at it again and
see if it's useful now.
19:09:37 <abadger1999> yay!
19:09:46 <sgallagh> What are the remaining issues?
19:09:51 <abadger1999> thanks puiterwijk!
19:10:03 * sgallagh thought puiterwijk had it solved yesterday
19:10:06 <nirik> sgallagh: well, I had issues trying to add a remote repo and do a
review against it. Might be operator error.
19:10:18 <nirik> it's much faster than before IMHO.
19:10:28 <nirik> I think we can make it faster yet if we want.
19:10:38 <sgallagh> nirik: 2-3s per page instead of 20-50s is a big improvement,
yes
19:10:55 <nirik> if we move the db to the same datacenter I think it would be
faster.
19:11:07 <nirik> right now it's hitting that over the vpn
19:11:33 <sgallagh> Yes. I also saw some upstream commits recently that should
hopefully reduce the DB activity as well (for RB 2.0)
19:11:55 <nirik> also, I wonder if it would make sense to move it to it's own
instance? or is having the local git repos a big advantage?
19:12:39 <sgallagh> nirik: The local git repos are no longer necessary
19:12:56 <sgallagh> I added patches upstream a while ago to support fedorahosted
cgit from remote locations
19:13:02 <nirik> yeah, didn't think so. ;) so, we might move it to it's own
instance with it's own db... which might make it faster.
19:13:14 <sgallagh> They were necessary when we originally set it up.
19:13:26 <nirik> then it could possibly be very useful to us. :)
19:13:48 <sgallagh> Well, we need to do some performance testing
19:13:56 <nirik> sure.
19:14:03 <sgallagh> i.e. Is the DB speedup more valuable than the loss of the local
git access?
19:14:20 <sgallagh> Because while cgit works pretty well, it *is* network traffic
over HTTP
19:14:38 <nirik> sure, but it could still be in the same datacenter as
fedorahosted...
19:14:46 * sgallagh nods
19:14:47 <nirik> even on the same vmhost...
19:15:14 <nirik> anyhow, we will see what we see.
19:15:18 <sgallagh> Right, but pointing it at cgit will mean more load on cgit
19:15:47 <nirik> any other application news ?
19:15:47 <sgallagh> Which is notably more heavyweight than pure git
19:15:52 <nirik> sgallagh: true
19:16:19 <nirik> can it use git:// proto ? or just cgit/http?
19:16:25 <johe2> Sorry lost my network. Will follow from mobile Phone
19:16:42 <nirik> hey johe2. :)
19:17:38 <nirik> any other applications news?
19:17:51 <sgallagh> nirik: the pure git protocol doesn't have a feature for
retrieving file blobs (major oversight)
19:17:59 <nirik> sgallagh: lame. ;(
19:18:03 <sgallagh> Hence why ReviewBoard has to rely upon cgit or gitweb
19:18:48 * pjones watches people complain about free software instead of fixing it
19:18:48 <threebean> I'm still aiming at deploying the new fmn app to production
by the end of next week
19:19:07 <nirik> threebean: cool.
19:19:10 <threebean> (fmn -> fedmsg notifications)
19:19:22 <nirik> pjones: nothing new under the sun. ;)
19:19:27 <sgallagh> pjones: I don't have the technical knowledge to solve that
problem.
19:19:40 <sgallagh> And a reasonably-ubiquitous workaround exists.
19:19:50 <nirik> #info fedmsg notifications hopefully deployed by end of next week
19:19:52 <threebean> nirik: I'll need to setup some new hosts. should I file a
ticket for that or just do it?
19:20:05 <nirik> threebean: you can just do it. Happy to help out.
19:20:11 <threebean> cool, cool.
19:20:44 <lmacken> I'd like to get the bodhi cloud vm + ansible ball rolling
very soon. If anyone is interested in helping port our puppet stuff over, let me know :)
19:21:13 <nirik> lmacken: cool. Yeah in fact I was thinking bodhi might be a good
next thing to move off the app servers...
19:21:22 <lmacken> yeah, definitely
19:21:28 <nirik> of course we likely need different bodhi and bodhi2 stuff.
19:21:29 <johe2> Lmacken i would join
19:21:31 <adimania> I am up for helping out in migration
19:21:42 <adimania> can you point out some tickets?
19:21:54 <lmacken> johe2, adimania: cool :) no tickets at the moment, but I'll
file one now
19:22:10 <nirik> should be easy to whip up a bodhi-dev.cloud... and look at porting
the bodhi2 puppet stuff to ansible.
19:22:23 <relrod> threebean: re: fmn, how are the model changes coming along?
19:22:27 <johe2> Okay. Sounds good
19:22:32 <nirik> lmacken: is there enough bodhi2 to have a web interface yet? or
thats the largely unfinished part?
19:23:37 <lmacken> nirik: not anything that people will enjoy poking at, it's
super bare-bones at the moment, but I'll try and workon that part this week so we have
something to actually see :)
19:23:54 <willo> lmacken: I'd be keen to assist in puppet2ansible as well
19:24:11 <nirik> yeah, that would be cool. sounds like adamw at least might be up
for a irc meeting on bodhi2 soon too... rehash what was wanted and such...
19:24:19 <lmacken> part of me just wants to copy/paste the existing bodhi templates,
another part wants to start fresh with bootstrap or something I wish we had a
fedora-bootstrap template :)
19:24:21 <threebean> relrod: all set
19:24:35 <threebean> relrod: I'll put a new release in staging and ping you when
its there.
19:25:12 <relrod> threebean: awesome!
19:25:27 <threebean> lmacken: https://github.com/ralphbean/bootstrap-fedora
19:25:32 <lmacken> threebean: oh snap
19:25:34 <relrod> threebean: thanks for doing those, I know it was kind of a pain
;(
19:25:37 <nirik> :)
19:25:43 <threebean> lmacken: it needs love :)
19:25:51 <lmacken> threebean: it's a good start :)
19:26:20 <lmacken> even with a modern template, I'll bet we still get plenty of
flames for changing the look-and-feel :P
19:26:39 <nirik> well, if not at the 1->2 time, when? ;)
19:27:35 <nirik> any folks looking for the board meeting, it's over in
#fedora-meeting--1
19:27:38 <nirik> any folks looking for the board meeting, it's over in
#fedora-meeting-1 even
19:27:57 <nirik> ok, any other application type news or discussion?
19:28:05 <lmacken> .ticket 4193
19:28:06 <zodbot> lmacken: #4193 (Port the bodhi module to ansible) – Fedora
Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/4193
19:28:12 <lmacken> for those interested
19:28:33 <nirik> great. I can add more details there.
19:28:49 <nirik> or perhaps I should do a blog post on migrating stuff from puppet
to ansible.
19:29:07 <adimania> nirik, that would help a lot
19:29:26 <adimania> some sort of guidelines would be a +1
19:29:47 <nirik> I'll see what I can whip up. ;)
19:30:11 <nirik> #topic Sysadmin status / discussion
19:30:18 <nirik> so, on the sysadmin side of things...
19:30:26 <nirik> #info lots of ansible cleanup this week
19:30:41 <nirik> I setup roles_path today so we don't have to have a long path
in role names...
19:31:08 <nirik> I ran --check --diff on all playbooks several times, and then ran
real playbooks to fix up things.
19:31:48 <nirik> I fixed out cloud setup task so it doesn't show failed when
growing partitions and it can't grow them....
19:32:04 <nirik> (ie, so you can rerun it multiple times and it doesnt keep showing
failed for that task)
19:32:27 <nirik> I have the commits for a daily playbook check/diff run ready. Will
push those in later today or tomorrow.
19:32:42 <nirik> and it will send a email out with things that changed or failed int
he check run
19:33:20 <nirik> oh, setup mailman01/02 prod instances, but they still need a bunch
of work. will ask abompard to work on them.
19:34:25 <nirik> In other news I see there's a new el6 kernel update, so we
might schedule mass reboots sometime in the next few weeks.
19:34:25 <threebean> for those that missed it, we got a new collectd plugin ->
http://da.gd/s0kH
19:34:34 <nirik> Oh, and...
19:34:46 <threebean> (that's roughly the # of people listening to the fedmsg
bus)
19:35:00 <nirik> #info filed 3 new sysadmin easyfix tickets this morning. :)
apprentices take a look and see if any one of them interests you.
19:35:43 <nirik> threebean: nice. Thats via everything? websocket/desktop
notification/etc?
19:35:52 <threebean> everything but websockets
19:36:00 <threebean> so, koji-shadow and stuff like that + desktop notification
19:36:17 <nirik> cool.
19:36:29 <dgilmore> threebean: you mean koji-stalk
19:36:37 <threebean> right, apologies.
19:37:02 <willo> nirik: will do
19:38:12 <nirik> #info new machines for 4thQ still in limbo as far as I know.
19:38:27 <nirik> #info netapp has been slow lately, storage folks are working on
it.
19:38:50 <nirik> #info Smooge will be heading out to our main datacenter in
mid-feb.
19:39:29 <nirik> #topic Upcoming Tasks/Items
19:39:30 <nirik> https://apps.fedoraproject.org/calendar/list/infrastructure/
19:39:39 <nirik> any upcoming items folks would like to note or schedule?
19:40:32 * nirik listens to the crickets. :)
19:40:39 <nirik> #topic Open Floor
19:40:46 <nirik> anyone have items for open floor?
19:40:58 <nirik> suggestions, comments, ideas, brands of popcorn?
19:41:03 * adimania would be stalking nirik for a guide for migrations to ansible.
19:41:40 <nirik> :) adimania: you can also look at the puppet/modules/ask and
ansible/roles/ask... I just moved ask over from puppet the other day...
19:41:52 <adimania> ah!
19:41:57 <adimania> that would certainly help
19:42:30 <nirik> but yeah, I will try and do some kind of blog post or the like
soon.
19:42:54 <adimania> awesome!
19:43:18 <johe2> Is there a Style guide?
19:43:39 <nirik> for which? our ansible repo?
19:43:46 <adimania> I found
http://infrastructure.fedoraproject.org/infra/docs/ansible.txt
19:43:57 <adimania> I think it is more or less up-to-date.
19:44:00 <adimania> right?
19:44:07 <nirik> yeah, thats a basic SOP for our ansible setup
19:44:16 <willo> nirik: Where's your blog? I couldn't find it on your wiki
page
19:44:16 <johe2> Yep for ansible
19:44:27 <nirik> there's also a README file and a CONVENTIONS file in the
ansible repo itself
19:44:50 <nirik> http://infrastructure.fedoraproject.org/cgit/ansible.git/tree/
19:44:57 <johe2> Okay, i guess that will work than
19:45:19 <nirik> willo: http://www.scrye.com/wordpress/nirik/ but it's also on
fedora planet.
19:45:39 <willo> Ah, yep, thanks
19:46:32 <nirik> cool.
19:46:48 <nirik> well, if nothing else we will move on and continue in
#fedora-admin, #fedora-apps and #fedora-noc. ;)
19:47:22 <nirik> ok, thanks for coming everyone!
19:47:25 <nirik> #endmeeting