infrastructure March 2008

infrastructure@lists.fedoraproject.org

44 participants
48 discussions

Outages and Koji
by Mike McGrath 22 Mar '08

22 Mar '08

We just had the longest buildsys outage we've had since I started (not happy). But the good news is we have lots of options to make sure this particular issue doesn't happen again. The problem: Is sort of a 3 fold issue. 1) NFS is running on the xen dom0 (will be fixed next week) 2) nfs lock is hanging and disallowing our clients to lock files 3) When it gets in this state a restart of nslock won't fix the problem, the port stays open so we have to restart the host. There's not much we can do about 2 or 3 except rely on upstream to fix the problem. Seth suggested that when we fix 1) we make it a RHEL4 box. I think this is probably the best solution. I'm also looking at other solutions for some of our other applications. Our environment isn't in terrible shape but I think it could be better. I'm looking at some different tools that might make things easier on us. I think our environment is ok but there's some apps that don't need to be load balanced, just more HA. And sometimes those apps are getting beaten up by apps that do need to be properly load balanced. I'm going to go through and see what some of our options are and I encourage those familiar with our environment to do the same. Some of our apps certainly need work (smolt especially) but I think there are things we can do as infrastructure members to mitigate risks and ensure we're seeing fewer outages. -Mike

1 0

hello
by Roland McGrath 21 Mar '08

21 Mar '08

Hi folks. I've joined the list and will apply for membership in sysadmin and sysadmin-hosted groups. I mostly want to hack on cobbling the mtn support into shape, and it seems easier to have access than to just keep bugging Mike all the time (no relation, by the way :-). I can't promise time to attend any scheduled meetings, and I don't have much spare time to work on worthy stuff in general. But if I have access, I will tend to fix random broken things in the middle of the night or weekend when I notice them rather than complain and wait til morning or Monday, and You Can Trust Me. I claim zero facility with web hooey implementation. Other than that, I can usually help fix or whip up scripts and whatnot in whatever language, and know plenty of random sysadminy crapola and arcane unixism that is occasionally useful. Thanks, Roland

2 1

Release Day planning and notes
by Paul W. Frields 21 Mar '08

21 Mar '08

There's an overall page I've been using for capturing info about this effort. It probably deserves to be moved at some point after we solidify our efforts a little more. http://fedoraproject.org/wiki/PaulWFrields/ReleaseDay I took a snapshot of the notes we worked on in Gobby and captured them here: http://fedoraproject.org/wiki/PaulWFrields/ReleaseDay/Notes/20080319-20 I tried to be diligent about capturing action items but feel free to help scootch things around where they make sense. If we can capture all the action items for this over the next couple days, we can schedule a followup for a convenient date. -- Paul W. Frields http://paul.frields.org/ gpg fingerprint: 3DA6 A0AC 6D58 FEC4 0233 5906 ACDB C937 BD11 3717 http://redhat.com/ - - - - http://pfrields.fedorapeople.org/ irc.freenode.net: stickster @ #fedora-docs, #fedora-devel, #fredlug

1 0

I'm not here!
by Roberto.Quagliozzi＠edftrading.com 21 Mar '08

21 Mar '08

I will be out of the office starting 21/03/2008 and will not return until 07/04/2008. I will only be checking my email sporadically during this period, if you have anything urgent, please call the helpdesk on x4999. ********************************************************************* This communication contains confidential information, some or all of which may be privileged. It is for the intended recipient only and others must not disclose, distribute, copy, print or rely on this communication. If an addressing or transmission error has misdirected this communication, please notify the sender by replying to this e-mail and then delete the e-mail. E-mail sent to EDF Trading may be monitored by the company. Thank you. EDF Trading Limited 80 Victoria Street, 3rd Floor, Cardinal Place, London, SW1E 5JL A Company registered in England No. 4255974. Switchboard: 020 7061 4000 EDF Trading Markets Limited is a member of the EDF Trading Limited Group and is authorised and regulated by the Financial Services Authority. VAT number: GB 735 5479 07 *********************************************************************

1 0

Meeting Log - 2008-03-20
by Ricky Zhou 20 Mar '08

20 Mar '08

16:04 < dgilmore> Hey all Infra Meeting 16:04 -!- dgilmore changed the topic of #fedora-meeting to: Fedora Infrastructure meeting 16:05 -!- jfluhmann [n=jfluhman(a)x170y234.angelo.edu] has joined #fedora-meeting 16:05 < dgilmore> ping ricky, skvidal, f13, Wakko666, jima, iWolf, anyone else i forgot 16:05 < skvidal> pong 16:05 * iWolf is here 16:05 < gkrpan> <- here 16:05 * Wakko666 waves. 16:06 < dgilmore> https://fedorahosted.org/fedora-infrastructure/query?status=new&status=assi… 16:06 < dgilmore> meeting items 16:06 < dgilmore> .ticket 395 16:06 < zodbot> dgilmore: #395 (Audio Streaming of Fedora Board Conference Calls) - Fedora Infrastructure - Trac - https://fedorahosted.org/projects/fedora-infrastructure/ticket/395 16:07 < dgilmore> jcollie: how much work do you think is involved here? 16:07 < jcollie> hmm 16:07 < dgilmore> its not something thats critical to get done 16:07 < jcollie> once we get FAS2/Asterisk integrated, then we need to get a bot set up to control things 16:07 < dgilmore> but something we need to keep in mind 16:08 * f13 16:08 * mdomsch is here 16:08 -!- rharrison [n=rharriso@nat/cisco/x-c2500f1acaecb55c] has left #fedora-meeting ["Leaving"] 16:08 < jcollie> maybe 10-16 hours of someone's time... dunno... i'm not good at estimating that 16:08 < dgilmore> if anyone wants to help this would be a nice place to do some things 16:09 < dgilmore> jcollie: we are thinking of a module for SupyBot allowing people to book and control rooms right? 16:09 < jcollie> yes 16:09 < jcollie> i've got a prototype started 16:09 < dgilmore> so it would intrface with asterisks manager api 16:09 < dgilmore> :) 16:09 < jcollie> yes 16:10 < jcollie> it's called ursabot but it doesn't join any channels yet 16:10 -!- bpepple|lt [n=bpepple|(a)adsl-69-214-168-157.dsl.wotnoh.ameritech.net] has joined #fedora-meeting 16:10 < dgilmore> :) 16:10 < jcollie> the code is in my asterisk1 home dir if anyone wants to take a look 16:10 < dgilmore> i might try poke at it 16:10 < dgilmore> so lets move on 16:11 < dgilmore> .ticket 398 16:11 < zodbot> dgilmore: #398 (elfutils `monotone' (mtn) error) - Fedora Infrastructure - Trac - https://fedorahosted.org/projects/fedora-infrastructure/ticket/398 16:11 < dgilmore> so what to do here 16:11 -!- stickster_mtg is now known as stickster 16:11 < jcollie> i say we block any new monotone repos until the anonymous access issues get sorted out upstream 16:12 < jcollie> for now you can rsync the elfutils repo anonymously iirc 16:12 < f13> jcollie: do e really have other monotone users? 16:12 < jcollie> f13: not that i know of 16:12 < jcollie> so it may never become an issue 16:13 * jfluhmann is here 16:13 < f13> yeah 16:13 < f13> I don't expect them to be knocking down our door 16:13 < dgilmore> we probably should have insisted roland changed RCS 16:14 < dgilmore> but anyway not much eles here for now 16:14 < dgilmore> .ticket 446 16:14 < zodbot> dgilmore: #446 (Possibility to add external links on spins page) - Fedora Infrastructure - Trac - https://fedorahosted.org/projects/fedora-infrastructure/ticket/446 16:14 < dgilmore> no one owns this 16:15 < dgilmore> im somewhat against it 16:15 < dgilmore> but then again i also told mether that he should find local hosting for localised spins 16:15 < mether> unfortunately that is easier said that done 16:16 < dgilmore> I can see the want to put localised spins links somewhere 16:16 < iWolf> the concern is that we can't be certain of the validity of the spins if we post external links? i.e. it falls outside our circle of trust? 16:16 * jwb perks up 16:17 < dgilmore> but perhaps we could link to a wiki page that has all the laocalised torrents on it 16:17 < jwb> if something changes with the localized spins please let me know 16:17 < dgilmore> iWolf: somewhat yes 16:17 < dgilmore> jwb: in what way? 16:17 < jwb> as in, if they are going to get hosted at spins.fp.o 16:17 < Wakko666> i seem to remember mmcgrath mentioning the need for board involvement on this. i don't remember the details, though. 16:17 < jwb> i'm not producing anymore localized spins at the moment 16:18 < dgilmore> jwb: why not 16:18 < dgilmore> i thought we were hosting some 16:18 < mdomsch> +1 to wiki page listing them 16:18 < dgilmore> fedora-fr.org has a french spin they are self hosting 16:18 < jwb> we have 1 hosted now. that was before the spins issue was brought up to the board 16:19 < jwb> yes, i know 16:19 < mdomsch> +1 to spins.fp.o linking to the wiki page 16:19 < dgilmore> mdomsch: I think that should be ok 16:19 -!- abadger1999 [n=abadger1@conference/pycon/x-7bf2f16291e110e7] has joined #fedora-meeting 16:19 -!- jmn [n=jmn@nat/redhat/x-ca09133ba144853c] has quit "Leaving." 16:20 < dgilmore> mdomsch: then we have have a disclaimer that the spinds are generated by local community people 16:20 < mdomsch> yep 16:20 < jwb> dgilmore, essentially we need agreement from Infrastructure, Rel-Eng, and the Board on how to handle spins before it's really going to go further 16:20 < jwb> spins hosted on spins.fp.o that is 16:20 < dgilmore> jwb: fair enough 16:20 < dgilmore> ok not much else here 16:20 < dgilmore> we have the three meeting items 16:20 < dgilmore> so Floor open 16:21 < f13> I wanted ot bring some thing up 16:21 < dgilmore> f13: shoot 16:21 < f13> I want to make infrastructure aware that I'm preparing a FESCo proposal on bringin the Jpackage team into our infrastructure, a la OLPC 16:21 < jcollie> that'd be cool 16:21 < f13> they have a need for good infrastructure, and we have a need for a closer and easier relationship with jpackage. 16:22 < f13> bringing them into our infrastructure accomlishes both of these needs, and opens up the door to many other benifits 16:22 < jcollie> would they still be able to do the nosrc rpms of the non-free bits? 16:22 < dgilmore> f13: i think its a great thing 16:23 < jwb> doesn't jpackage do non-free stuff? 16:23 < dgilmore> f13: however we really need that blade center installed first i think 16:23 < f13> the need for nosrc stuff goes out the door with openjdk 16:23 < dgilmore> jcollie: they would have to go. or be done outside of fedora 16:23 < f13> and they wouldn't be allowed to bring it into our infrastrucutre, adn they know that 16:23 < jwb> who's going to sponsor them? 16:23 < f13> dgilmore: we may need a number of things first, and that will be accounted in my proposal. 16:24 < f13> we don't need to accomplish this tomorrow, it can be a gradual thing. 16:24 < jwb> sorry, that's a FESCo question i gues 16:24 < dgilmore> f13: also we need more disk for cvs-int 16:24 < jwb> s 16:24 < f13> jcollie: I want to mentor a few of jpackage folks and get them setup as sponsors 16:24 < dgilmore> f13: that seems sane 16:24 < f13> dgilmore: or we need to actually prune lookaside cache. 16:24 < dgilmore> f13: i think we have always said we cant do that 16:25 < f13> which I call BS on, but *shrug* 16:25 < jcollie> could we just move the lookaside cache to a separate system? 16:25 < f13> anywho, I just want to make you guys aware that this is coming so that you aren't blindsided 16:25 < dgilmore> f13: as long as well can still retrive the source. then i guess its ok 16:25 < f13> the proposal will have language in it that it predicates upon Infrastructure being able to fulfill the need 16:26 < f13> dgilmore: we only need the source for the things we are still offering binaries for. 16:26 < jwb> i'm still confused 16:27 < f13> we may even be able to get them to donate some time/money/material to the Infrastructure cause. 16:27 < f13> jwb: how can I clarify? 16:27 < jwb> do OLPC have some other setup where their packages don't follow the fedora review process? 16:28 < dgilmore> f13: that is completely different to what mdomsch is trying to do with being able to recreate srpms 16:28 < jwb> or are they treated as "normal" packages, with a special OLPC branch? 16:28 < f13> jwb: no, the OLPC packages in the Fedora infrastructure have to follow our guidelines. 16:28 < dgilmore> jwb: they do and its not buit in Fedora 16:28 < f13> jwb: normal packages with olpc branch. Some don't have Fedora branches. 16:28 < f13> some do. 16:28 < jwb> so why wouldn't we just get all the jpackage packages into fedora? 16:28 < dgilmore> jwb: anything in fedora cvs must meet fedora guidelines 16:28 -!- kital [n=Joerg_Si@fedora/kital] has joined #fedora-meeting 16:29 < f13> dgilmore: not extremely. For that case, we only need to be able to recreate them of a certain age. Also, we can move the cruft to an archive box and not on our production scm instance clogging up resources. 16:29 < f13> but this is talk for another time. 16:29 < dgilmore> jwb: for there noarch packages that are useable on all distros i assume 16:29 < f13> jwb: because jpackage provides cross-distro builds of the packages at a different schedule than Fedora 16:29 < mdomsch> +1 to "archive box" 16:29 < mdomsch> there's a lot of stuff we'd like to hang onto, but not necessarily keep super-live... 16:29 < f13> jwb: Fedora and RHEL aren't the only jpackage consumers. 16:30 < jwb> so now we're turning into rPath? 16:30 < f13> no, we're providing an opensource development lab and compelling infrastructure for interesting opensource projects 16:30 < f13> jwb: they would be doing builds from their branch through our buildsystem, and then releasing them however they see fit 16:30 * jwb saves the rest for FESCo 16:31 < f13> jwb: but it'll be come much easier to get their updates into Fedora builds and vice versa. 16:31 < f13> jwb: I'd be happy to chat with you about it at any time. 16:31 < jwb> yeah, i might take you up on that 16:32 < mdomsch> as infrastructure team, I want to be careful about overcommitting and under-delivering 16:32 < mdomsch> but where we can offer assistance 16:32 < jwb> what he said 16:32 < mdomsch> I'd like to see us do so 16:32 < f13> nod 16:32 < jwb> we're at odds with other goals a bit i think 16:32 < mdomsch> and if we can get some resources from JPackage to help cover their needs plus a bit, great 16:33 < jwb> hm, not goals. intiatives 16:33 < mdomsch> f13, please include in your proposal to FESCo the resource estimates 16:33 < f13> mdomsch: I will be working on those next week. 16:33 < mdomsch> presuming JPackage has some 16:34 < f13> well, right now they have a box, with mock running on it 16:34 < f13> I think that's about it (: 16:34 < f13> but a good friend of mine was working on getting them more resources and a better buildsystem. I've kind of usurped that, so we might be able to get him to direct some of those donations our way 16:34 < f13> which don't help much when we don't have any frelling place to /put/ them :/ 16:36 < mdomsch> next topic? 16:36 < mdomsch> beta release on Tuesday, bits flowing to mirrors now 16:37 < dgilmore> mdomsch: :) 16:37 < dgilmore> f13: want to give a report on how the beta bits are flowing 16:37 < dgilmore> 'and how the changes have helped 16:38 < mdomsch> dgilmore, you mean reversing the streams 16:38 < mdomsch> ? 16:39 < f13> gaining control over the netapp has been invaluable 16:39 < f13> probably saved a good 3 to 5 days worth of delay this time around 16:39 < f13> mdomsch: do we have any idea how many mirrors have gotten the bits, or are getting the bits? 16:39 < dgilmore> mdomsch: yes, and us pushing bits rather than RH IS 16:39 -!- rdieter is now known as rdieter_away 16:40 < mdomsch> f13, not yet 16:40 < mdomsch> we'll know more tomorrow 16:40 < mdomsch> MM hasn't picked up the fact that the dir is open yet 16:40 < f13> k 16:40 < f13> I don't have access to a mirror to verify that the bits are open though. 16:40 < mdomsch> i do, and they are 16:41 < dgilmore> :) 16:41 < mdomsch> receiving file list ... done 16:41 < mdomsch> drwxr-x--- 4096 2008/03/19 16:52:18 9-Beta 16:41 < mdomsch> -rw-r--r-- 718 2008/03/17 10:31:50 9-Beta/.composeinfo 16:41 < mdomsch> drwxr-xr-x 4096 2008/03/19 16:50:46 9-Beta/Fedora 16:41 < mdomsch> ... 16:41 < dgilmore> ok anything else ? 16:42 < mdomsch> f13, are you going to do the .9-Beta hidden dir trick again in the future? 16:42 < mdomsch> that confused MM a bit 16:42 < mdomsch> because it can see .foo directories just fine 16:42 < f13> mdomsch: I don't have to, I could name it the final location and just ensure permissions are right. 16:42 < mdomsch> and thought the version was literally ".9-Beta" 16:42 < mdomsch> ok, please do the latter then 16:42 < f13> I wasn't going to leave it unlocked with a . in the name 16:43 < f13> but until I could ensure the permissions were doing what I hoped they would do I didn't want it obviously available to the world. 16:43 < mdomsch> sure 16:43 < f13> (having shell access to a mirror would help with this kind of thing) 16:43 < jwb> sekret 16:43 < mdomsch> f13, np, send ftp+fedora(a)redhat.com the IP of a box you want to claim as a rel-eng mirror 16:43 < mdomsch> and just rsync -va ... but with no dest, so all you get is the file listing :-) 16:44 < f13> mdomsch: good point, I should do that. 16:44 < mdomsch> sbathe isn't online atm 16:44 < mdomsch> she'll do it no problem 16:45 < mdomsch> ok, I"m out for the weekend - goodbye and good luck 16:45 -!- mdomsch [n=mdomsch(a)70.124.62.55] has quit "Leaving" 16:45 < dgilmore> all right guys thanks very much 16:45 < f13> later. 16:45 -!- rdieter_away [n=rdieter(a)sting.unl.edu] has quit Read error: 104 (Connection reset by peer) 16:45 < dgilmore> wrapping up in 60 unless someone has something else to add 16:46 < dgilmore> 30 16:46 < dgilmore> 15 16:46 < dgilmore> done

1 0

Asterisk and Town Hall meeting
by Paul W. Frields 20 Mar '08

20 Mar '08

The Fedora Board should be doing another "town hall" style meeting on Tuesday April 1. (No fooling.) In March we postponed plans until then to use Asterisk and Gstreamer to provide some sort of listening capability for community members. (1) How is that going? (2) Can we plan to use that solution for the April 1 meeting? (3) If not, how can we schedule so as not to crunch the team around the F9 release timetable? -- Paul W. Frields http://paul.frields.org/ gpg fingerprint: 3DA6 A0AC 6D58 FEC4 0233 5906 ACDB C937 BD11 3717 http://redhat.com/ - - - - http://pfrields.fedorapeople.org/ irc.freenode.net: stickster @ #fedora-docs, #fedora-devel, #fredlug

5 8

Release day planning call
by Paul W. Frields 20 Mar '08

20 Mar '08

To f-infrastructure-l denizens: The release day planning group is going to crash this party, and bring a little traffic here, since you guys are intimately involved with the release day efforts, and also have been very good about tracking what you want to fix the next time around. Hopefully our group can take a lesson from that and improve many of the other constituent parts of the release process. There's some notes available on Gobby as "release-day-planning.txt" where we scribbled notes. I'll post more from our meetings this week after our follow up later today; hopefully we'll have a better list of action items if needed. One surprise: Some of the attendees found it really useful to be able to say, "I need *X* earlier in the process," or, "It would be good if we had a few days' more time for <particular_task>" in an open forum with people who really know their part of the release process. In that sense, this was a little like a pre-F10 meeting, but there were plenty of areas we found that we can do for this cycle to make things come together nicely. Looking forward to talking to everyone today at noon! -- Paul W. Frields http://paul.frields.org/ gpg fingerprint: 3DA6 A0AC 6D58 FEC4 0233 5906 ACDB C937 BD11 3717 http://redhat.com/ - - - - http://pfrields.fedorapeople.org/ irc.freenode.net: stickster @ #fedora-docs, #fedora-devel, #fredlug

3 4

Metalink support
by Bram Neijt 19 Mar '08

19 Mar '08

Hi everybody, My name is Bram and I'm one of the developers the metalink tools project. I've heard that somebody tried to implement metalinks as part of the mirrormanager project and failed to get an agreable patch. I'm new to mirrormanager but I would like to see if I can find a way to help you guys with hosting metalinks in a usefull way. Even if it means it has to be done outside of the mirrormanager. Are there any opinions floating around about how metalinks could be implemented/created? Greetings, Bram PS I have some ideas, but as I'm new to this list I don't want to start dictating solutions. Feel free to invite me to do so ;-)

4 9

FAS2 Home Page Feature Request
by Thomas Chung 19 Mar '08

19 Mar '08

Hey Infrastructure Team Would you please add "approve" link in the FAS2 Home page for "Todo queue"? It takes too long to view any group (specially 'ambassadors' group with over 300 members) and approve each request. Here is the screenshot http://tchung.fedorapeople.org/FAS2/FAS2_todo_queue.png Thank you. -- Thomas Chung http://fedoraproject.org/wiki/ThomasChung

4 4

Re: Asterisk and Town Hall meeting
by Seth Vidal 19 Mar '08

19 Mar '08

On Tue, 2008-03-18 at 16:11 -0600, Matt Domsch wrote: > On Tue, Mar 18, 2008 at 10:01:09PM +0000, Paul W. Frields wrote: > > I see: > > https://hosted.fedoraproject.org/fedora-infrastructure/ticket/309 > > > > That seems to be related, but not explaining the need, so I filed: > > https://hosted.fedoraproject.org/fedora-infrastructure/ticket/453 > > > > I've marked it as a F10 milestone because (1) I don't want to block > > y'all's wiki work, and (2) I think we can live with IRC until whenever > > this gets done. > > > > Cc'ing the board list FYI only. > > Agreed. As Seth noted, there's a way to do SIP -> telephony concall > dialin, so if we really really wanted a voice town hall, we could > arrange that on a telephony concall system until a fully SIP method > was available. > sip:*tollfreenumberhere@fwd.pulver.com I use it every week. -sv

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure March 2008