Infrastructure IRC Log - 2008-08-21

[Nigel Jones]

For those that missed it... Today's IRC Log


08:02 -!- mmcgrath changed the topic of #fedora-meeting to:
Infrastructure -- Who's here?
08:02 < G> I iz here :)
08:02 < jeremy> G: no you're not
08:02 < fchiulli> fchiulli: is here
08:03  * MrBawb /
08:03 < G> jeremy: bleh
08:03  * nirik is sitting in the nosebleed seats in the back
08:03  * ianweller 
08:03 < G> jeremy: good point, I left my brain in the other Nigel :)
08:03 < jeremy> G: there's two nigels?  I had been wondering how you got
so much done :-)
08:04 < mmcgrath> Ok guys lets get started.
08:04 -!- mmcgrath changed the topic of #fedora-meeting to:
Infrastructure -- "Issues"
08:04 < mmcgrath> So yeah, i'd imagine lots of us here want to talk
about these crazy issues that paul has notified everyone about.
08:04 < mmcgrath> Unfortunately all I can say right now is that it won't
happen in this meeting and I'm sorry.  I really am.
08:05 < ianweller> :(
08:05 < mmcgrath> It pains me to have to not comment but I just can't so
for right now, this is just going to be a boring, normal meeting.  :)
08:05 < G> fine my me
08:05 < G> *fine by me
08:05 < fchiulli> I'll just have to cry in my beer.  :-)
08:05 < smooge> I still say its zombies
08:05  * mmcgrath loves my team.  
08:05  * lmacken rolls in late
08:06 < ianweller> can you give us an ETA or is that even way out there
08:06  * abadger1999 here
08:06 < mmcgrath> I'm not going to ever be untruthful to you guys,
really.  So all I can say.. is that I can't say.
08:06 < ianweller> mmcgrath: thanks :)
08:06 < mmcgrath> ianweller: sorry, I can't give you a quantitiative
answer like "in 2 days" or anything like that.
08:06  * dgilmore notes he is a zombie
08:06 < ianweller> damn
08:06 < mmcgrath> Ok, so.  Lets look at the tickets.
08:06  * londo is here
08:07 < G> mmcgrath: we still have tickets? wow I forgot :P
08:07 < mmcgrath> oh and I think zodbot is mostly back up :)
08:07 < mmcgrath> .tiny
https://fedorahosted.org/fedora-infrastructure/query?status=new&statu...
08:07 < zodbot> <http://tinyurl.com/47e37y> (at fedorahosted.org)
08:07 < zodbot> mmcgrath: http://tinyurl.com/47e37y
08:07 < mmcgrath> boo zodbot.
08:07 -!- ianweller changed the topic of #fedora-meeting to:
Infrastructure -- tickets
08:07 < ianweller> hah wow
08:07 < mmcgrath> :)
08:07 < mmcgrath> ok.
08:08 < mmcgrath> so lets do
08:08 < mmcgrath> .ticket 732
08:08 < mmcgrath> first
08:08 < zodbot> mmcgrath: #732 (Investigate New System Monitoring
Methods) - Fedora Infrastructure - Trac -
https://fedorahosted.org/fedora-infrastructure/ticket/732
08:08 < mmcgrath> G: want to take this one?
08:08 < G> Yeah okay
08:09 < mmcgrath> so have at it.
08:09 < mmcgrath> I still see some segfaulting :)
08:09 < G> Right so Zabbix has been working okay, but I encountered some
issues with the distributed monitoring in 1.4.x, looks like we'll need
to use the 1.5.x branch (due to be 1.6 in September)
08:10 < rsc> G: Is Fedora Infrastructures' Zabbix already somewhere
running?
08:10 < G> 1.6 allows us to set up Zabbix proxies on certain hosts to
take some of the grunt work off noc1 but still only have one source of
announcements etc
08:10 < G> (example is, VPN servers and noc2)
08:10 < mmcgrath> rsc: it is
08:11 < mmcgrath> rsc: along side our nagios install
08:11 < rsc> mmcgrath: URL?
08:11 < G> rsc: yeah, we've got an instance going, seems to be fairly
stable
08:11 < mmcgrath> https://admin.fedoraproject.org/zabbix/
08:11 < G> mmcgrath: btw segfaults? where?
08:11 < rsc> mmcgrath: btw I'm still lacking THAT infrastructure
mail/explanation for the last week ;)
08:11 < lmacken> G: i've seen some on app1
08:11 < G> lmacken: rpm -q zabbix-agent?
08:11 < mmcgrath> G: I thought I saw some on bastion as well
08:12 < mmcgrath> skvidal: we should send an email out about that.  Can
you do that?
08:12 < lmacken> G: zabbix-agent-1.4.6-1.el5
08:12 < rsc> mmcgrath: but can't login with FAS data there...
08:12 < G> argh
08:12 < skvidal> mmcgrath: what do you want an email about?
08:12 < mmcgrath> rsc: correct.
08:12 < G> lmacken: I'll talk to jeff
08:12 < mmcgrath> skvidal: about why some mail was dropping and that
some people have gotten unsubscribed from lists as a result of it.
08:12 < skvidal> mmcgrath: oh, you're just being a dick, I see
08:12 < skvidal> mmcgrath: oh - I see
08:13 < mmcgrath> skvidal: heh, no.  there was really a problem and it
affected people :)
08:13 < skvidal> mmcgrath: oh - I thought you were just screwing w/me
08:13 < mmcgrath> skvidal: sorry not right now :)
08:13 < skvidal> sorry
08:13 < skvidal> yah
08:13 < jcollie> eh, more segfaults? need to get everybody updated to
the latest RPMs :)
08:13 < mmcgrath> no no on :)
08:13 < mmcgrath> G: did we get the new new rpm in the repo?
08:13 < skvidal> I can email to f-i
08:13 < skvidal> or does it need to go to announce?
08:13 < jcollie> G: no, I don't think so
08:13 < G> anyway, the two main points are, a full zabbix deployment
will conflict with our infrastructure freeze
08:13 < mmcgrath> skvidal: I'd say devel-announce and f-i would be good.
08:14 < G> (for F10)
08:14 < rsc> mmcgrath: why? For nagios I was able to read the stuff at
least.
08:14 < G> rsc: you still can
08:14 < G> rsc: Monitoring tab
08:14 < rsc> G: I would like to just read at zabbix.
08:14 < G> https://admin.fedoraproject.org/zabbix/dashboard.php
08:15 < G> mmcgrath: whats the proceedure for that?
08:15 < mmcgrath> G: sorry?  to just read zabbix?
08:15 < skvidal> mmcgrath: what day were the bounces? All my days have
run together
08:15 < skvidal> tuesday right?
08:15 < mmcgrath> skvidal: it was likely over a longer period of time
then that.
08:15 < jcollie> i'd say that since the F10 beta is going to slip for a
while we don't have to freeze just yet
08:15 < mmcgrath> no one reported it till tuesday though.
08:15 < skvidal> so I'll say sat -> tues
08:16 < G> mmcgrath: no, Infrastructure Frezze and a full scale zabbix
deployment?
08:16 < mmcgrath> sure
08:16 < mmcgrath> G: oh, so the infrastructure freeze is still a bit out
I'll get an exact day for that.
08:16 < mmcgrath> as far as the full scale zabbix deployment, we're
lucky to have nagios up still so we can set it up in tandum.
08:17 < mmcgrath> as soon as the monitoring of nagios has been
duplicated.  I'd say we just... tell people to setup their alerts.
08:17 < jcollie> i don't think that rel-eng knows when beta will be out
since they can't do any builds or composes yet
08:17 < G> mmcgrath: roger that
08:17 < mmcgrath> jcollie: until I hear otherwise.  The schedule hasn't
shipped.  now, i'd certainly think its going to, but right now I'm
moving assuming it's still on schedule.
08:18 < mmcgrath> err schedule hasn't slipped anyway :)
08:18 < G> Anyway, to finish up, I'm working some documentation for
Zabbix and the _right_ way on doing stuff such as setting up triggers
and monitoring items so I don't have to wack people over the head and
clean up stuff every day :)
08:18 < jcollie> yeah haven't seen much of f13 lately
08:19 < G> And thats about it from me
08:19 < abadger1999> G: Excellent (documentation)
08:19 < mmcgrath> G: yeah a policy thing would be good.
08:20 < G> yeah, Nagios has got very tatty
08:21 < fchiulli> Now I can go back and work on my ticket.  Hot damn.
08:21 < mmcgrath> heheh
08:21 < mmcgrath> Ok
08:21 < mmcgrath> G: anything else on that?
08:21 < G> mmcgrath: nah, lets roll on
08:21 < mmcgrath> solid
08:21 < mmcgrath> .ticket 753
08:21 < zodbot> mmcgrath: #753 (Mini-freeze for Fedora 10 beta) - Fedora
Infrastructure - Trac -
https://fedorahosted.org/fedora-infrastructure/ticket/753
08:22  * mmcgrath isn't familiar with this one
08:22 < G> heh, I guess we kinda discussed this?
08:22 < mmcgrath> yeah I guess so
08:22 < mmcgrath> If Jesse needs a freeze we'll give him one.  I'll make
sure to discuss this next week.
08:22 < mmcgrath> .ticket 395
08:22 < zodbot> mmcgrath: #395 (Audio Streaming of Fedora Board
Conference Calls) - Fedora Infrastructure - Trac -
https://fedorahosted.org/fedora-infrastructure/ticket/395
08:22 < mmcgrath> I'm pretty sure there's no new work on this
08:23 < mmcgrath> jcollie: anything?
08:23 < jcollie> nope
08:23 < G> btw, in relation to F10 Beta slipping, I'm pretty sure I
heard on -devel-list that it'll be at least a week
08:23 < jcollie> we should try and get something done soon though
08:23 < mmcgrath> k
08:23 < mmcgrath> that sounds reasonable.
08:23 < mmcgrath> .ticket 446
08:23 < zodbot> mmcgrath: #446 (Possibility to add external links on
spins page) - Fedora Infrastructure - Trac -
https://fedorahosted.org/fedora-infrastructure/ticket/446
08:23 < mmcgrath> dgilmore: ping?
08:23 < dgilmore> mmcgrath: pong
08:24 < mmcgrath> anything about the spins stuff?  is that not needed
anymore?  if not can we get the ticket closed?
08:24 < dgilmore> mmcgrath: we have a wiki page.  need an appropriate
disclaimer
08:24 < dgilmore> im not sure if we do need it?
08:27 < mmcgrath> dgilmore: not sure, you just said you'd take this one
so I keep pinging you about it :)
08:27 < dgilmore> mmcgrath: i should get hold of the spins people and
determine if its still neeeded
08:27 < dgilmore> if it is then get the proper disclaimer
08:28 < dgilmore> page is at http://fedoraproject.org/wiki/Spins
08:28 < mmcgrath> <nod>
08:28 < dgilmore> not really anything to add
08:28 < dgilmore> so lets move on :)
08:28 < mmcgrath> hah, k
08:28 < mmcgrath> .ticket 576
08:28 < zodbot> mmcgrath: #576 (Infrastructure Contact Information) -
Fedora Infrastructure - Trac -
https://fedorahosted.org/fedora-infrastructure/ticket/576
08:28 < mmcgrath> I was actually going to try to do this this week but I
got distracted :)
08:29 < mmcgrath> so we'll say thats a next week thing
08:29 < mmcgrath> .ticket 740
08:29 < zodbot> mmcgrath: #740 (Loaning out system time to OLPC
participants) - Fedora Infrastructure - Trac -
https://fedorahosted.org/fedora-infrastructure/ticket/740
08:29 < jcollie> heh and we probably could have actually used it :)
08:29 < mmcgrath> dgilmore: you have any more thoughts on that?
08:30 < dgilmore> mmcgrath: let me look
08:30 < dgilmore> ok so what i think they are looking for is
08:30 < dgilmore> a sand box system
08:30 < dgilmore> somewhere to do mock builds.
08:31 < dgilmore> possibly also to run koji commands from
08:31  * dgilmore should find someone to package koji for debian 
08:32 < mmcgrath> hmm
08:32 < mmcgrath> dgilmore: we should probably hit this up on the list
and see if its even something we should be doing.
08:32 < G> I still say, rack up a couple of the decommissoned xen hosts
install Fedora on them, and throw them in a different part of the
network, lock them down and ...
08:32 < dgilmore> we should get clarification that is what they want
08:32 < mmcgrath> I don't see any immediate benefit to Fedora but I'm
not sure what the cost is quite yet.
08:32 < mmcgrath> yeah.
08:32 < mmcgrath> G: yeah we'd probably try to host that all somewhere
else
08:33 < dgilmore> mmcgrath: a benefit for fedora would be closer working
relationship.  which should have the benefit of being able to build
smaller fedora systems
08:34 < mmcgrath> <nod>
08:34 < mmcgrath> dgilmore: lets hold off till we find out what they
want.
08:34 < mmcgrath> gregdek posted that ticket, ther emight be a better
contact at RH
08:35 < dgilmore> mmcgrath: im guessing gdk is the best contact
08:35 < mmcgrath> err not at RH, at OLPC.
08:35 < mmcgrath> my bad.
08:35 < dgilmore> mmcgrath: but im guessing he doesnt know the
requirements
08:35 < mmcgrath> either way we should get a clear idea of what they
want to do.
08:35 < mmcgrath> <nod>
08:35 < dgilmore> m_stone: is likely the best contact
08:35 < mmcgrath> k
08:35 < mmcgrath> for now though, probably nothing new there?
08:36 < gregdek> I think best wait.
08:36 < gregdek> They'd really rather see koji back online first.  :)
08:36 < mmcgrath> heh
08:36 < mmcgrath> fair enough
08:36 < mmcgrath> so next ticket
08:36 < mmcgrath> No new ticket.
08:36 < mmcgrath> Soooo.
08:36 -!- mmcgrath changed the topic of #fedora-meeting to:
Infrastructure -- IBM tech
08:36 < mmcgrath> An IBM tech is headed to fix a bad drive in xen14
right now.  That should happen in the next hour or two.
08:37 < G> goodie :)
08:37 < mmcgrath> yeah, it'll be good to have xen14 back online.
08:37 < mmcgrath> and thats really all I have for this meeting
08:37 < G> mmcgrath: can we get him to leave a couple of extra servers
behind while he's at it? :)
08:37 -!- mmcgrath changed the topic of #fedora-meeting to:
Infrastructure -- Open Floor
08:37 < mmcgrath> G: we can but we'd have no where to put them all :)
08:38 < jcollie> DNS servers... can we get some more?
08:38 < mmcgrath> although I am going to get rid of some 2U's.
08:38 < G> spoil sport :)
08:38 < smooge> DNS-SEC should we sign our zone?
08:38 < mmcgrath> jcollie: yeah, Its not really a priority right now
though, we don't really have any dns issues right now but we can get
another dns server or two up.
08:38 < mmcgrath> smooge: we could I think jcollie was even looking at
that a few months back.
08:38 < mmcgrath> jcollie: ?
08:39 < jcollie> smooge: i've been using it for my home domain ... we
need to figure out how to make the workflow work right
08:39 < jcollie> you have to resign your domain every 30 days
08:39 < jcollie> plus some key rotations
08:39 < smooge> ouch.
08:39 < mmcgrath> really?
08:40 < jcollie> yeah the signatures on the domain have a expiration
date ... i think the default is 30 days - not sure if you can extend it
or not
08:40 < mmcgrath> eeeenteresting
08:40 < G> Oh yeah, https://admin.fedoraproject.org/fingerprints has a
short list of our Package Signing Keys and SSH Host Fingerprints :)
08:41 < jcollie> the tools are very much in their infancy.. for right
now i'd say hold off on dns-sec, at least until we get more comfortable
08:41 < mmcgrath> G: that it does.  We'll get cvs in there as soon as
its back online as well.
08:41 < mmcgrath> jcollie: sounds good to me.  If its something you feel
like tackling though some day.  i'm fine with being an early adopter
there.
08:41 < G> mmcgrath: yeah
08:41 < mmcgrath> especially if it can help mature that technology more
quickly.
08:42 < jcollie> yeah, i'm working on the procedures with my personal
domain... once i have it figured out i'll work on how to do fedora
domains
08:42 < mmcgrath> schweet.
08:42  * smooge needs to get that workflow into csi
08:42 < mmcgrath> :)
08:42 < mmcgrath> so much needs to get into csi
08:42 < G> mmcgrath: zabbix stuff maybe :)
08:42 < smooge> well hopefully I will have some time week after next to
write some rough stuff
08:43 < mmcgrath> yeah
08:43  * mmcgrath hopes he will too
08:43 < mmcgrath> Ok, so anyone have anything else they'd like to
discuss?  If not I'll close the meeting in 30
08:43 < smooge> what I need is a newer system at home and then hope the
pain meds don't make me too dopey
08:43  * G told Amazon to forget my order for the second nagios book :)
08:43 < mmcgrath> :)
08:43 < G> mmcgrath: ummm I don't think so, except...
08:44 < mmcgrath> mhmmm?
08:44 < jcollie> mmcgrath: if you want to set up a couple guests i'll
set up secondary dns services on them
08:44 < jcollie> maybe we want a sysadmin-dns group?
08:44 < mmcgrath> jcollie: <nod>  we'll get something up.
08:44 < mmcgrath> good question.  I'm not sure we'll need it.
08:44 < G> I move to say that everyone here appreciates the hard work
Mike, Toshio, Jeremy, Seth and Luke (and anyone else I've forgotten) has
put it the last week. It's much appreciated
08:44 < mmcgrath> in theory we won't even need access to that box to
admin it.
08:44 < skvidal> G: ricky, jeremy, too
08:44 < jcollie> mmcgrath: either that or approve me for
sysadmin-main :)
08:44 < skvidal> ah, you said jeremy
08:45 < abadger1999> G: And dgilmore
08:45 < mmcgrath> and G
08:45 < skvidal> nod
08:45 < G> skvidal: Dennis and Ricky
08:45 < G> yeah
08:45 < jcollie> mmcgrath: access to the box would be more for
troubleshooting
08:45 < mmcgrath> G helpped get our web services back online :)
08:45 < jcollie> mmcgrath: could lump it under sysadmin-tools maybe
08:45 < G> mmcgrath: right but you folks did the grunt work to get the
core back up
08:46 < mmcgrath> jcollie: yeah that makes sense actually.
08:46 < mmcgrath> sysadmin-tools I mean.
08:46 < mmcgrath> G: thanks for the kind words and work :)
08:46 < mmcgrath> Ok, we'll close the meeting early :)
08:46 < mmcgrath> 30
08:47 < mmcgrath> 15
08:47 < G> mmcgrath: yeah, thats all I had to say :)
08:47 < mmcgrath> 10
08:47 < mmcgrath> 5
08:47 < dgilmore> mmcgrath: i have a dns server ill gladly offer as
tertiary
08:47 -!- mmcgrath changed the topic of #fedora-meeting to:
Infrastructure -- Meeting end!
08:47 < mmcgrath> dgilmore: naw, I think if anything we'll move in to
running more of our own servers.
08:47 < mmcgrath> best to keep those internal if we can.
08:47 < mmcgrath> Thanks for coming to the meeting everyone!!!
08:48 -!- mmcgrath changed the topic of #fedora-meeting to: Channel is
used by various Fedora groups and committees for their regular meetings
| Note that meetings often get logged | For questions about using Fedora
please ask in #fedora | See
http://fedoraproject.org/wiki/Communicate/FedoraMeetingChannel for
meeting schedule
08:48  * ianweller hopes everything is fixed soonish :)
08:48 < jcollie> mmcgrath: dgilmore: esp since we can set up xen guests
in multiple geographical locations
08:50 < mmcgrath> jcollie: yeah

-- 
Nigel Jones <dev(a)nigelj.com&gt;