infrastructure June 2006

infrastructure@lists.fedoraproject.org

20 participants
24 discussions

[Fedora-infrastructure-list] Mirror Management Specification/Requirement List
by Jason Hartley 29 Jun '06

29 Jun '06

David Means and I have been working on putting together a specification/requirement list for what we need for the mirror management task. This is for two reasons. One, we might not get all of the Centos pieces in a timely matter, so we will move forward with our own code. Two, if we get all the pieces we need to make sure they are going to work as needed. In an effort to complete the list, I am asking for some guidance in understanding the process as a whole. Please look over the list below and make any necessary comments. Here is our complied list: * Cron job to check all of the mirrors for freshness. We got this piece from Seth(Centos team - makemirrorlists2-now.pl), but it is going to need some modifications. * Database dictionary - What kind of data do we need to store for each mirror. * We will need to maintain a list of mirror sites and their state (ie useable, stall, not reachable, disabled.) * Some kind of front end to redirect clients to their best mirror match(ie GeoIP.) * Admin script to manage mirror list(additions, modifications, deletions, etc.) * Once a yum session starts, we will need to direct all of the requests for that update session to the *same* mirror. That way the updates will be consistent, even if not to the most recent version. If this is true then this increases the complexity of the task, because we need to a. notice when a new session begins, and record the IP source. b. check each incoming request against the known table of ongoing sessions, and send continuing requests on to the assigned mirror. c. clean up the sessions table when the update session ends. * Handle sites that come back with a new IP, after being off the network. Has the site name been hijacked or just part of some network clean up work? David, if I missed anything, please correct me. As for the rest of you, please look over the list and reply with any amount of insight. Thanks, Jason Hartley

1 0

[Fedora-infrastructure-list] Comment on Turbo Gears
by Jason Hartley 29 Jun '06

29 Jun '06

I just now had a chance to go over the meeting notes from last week and I noticed the discussion on TG. In FC5 it is included in the extras repo, but I had problems getting through the wiki tutorial. I seem to remember it had something to with SQLObjects having some issues. After couple of hours of messing around with it I decided to use the ez_setup.py script. After using ez_setup.py script to install TG everything work as it should. The ez_setup.py uses Python eggs, which so far have worked well for me. In any case, I think TG is great! Regards, Jason Hartley

1 0

[Fedora-infrastructure-list] Self-Introduction
by David Eisenstein 29 Jun '06

29 Jun '06

Greetings, Thought I would take a moment to introduce myself. My name is David Eisenstein. I am a volunteer with the Fedora Legacy project <http://fedoraproject.org/wiki/Legacy> at this time, and have recently started also helping out the fine Fedora Unity folks <http://fedoraunity.org/>. I have an interest in helping out with the Fedora Infrastructure project. What I am most keenly interested in at this time is being here to help work with the upcoming Fedora Legacy integration that has been blessed by the Fedora Board <https://www.redhat.com/archives/fedora-advisory-board/2006-June/msg00021.ht…>, and to help in continuing maintenance of such (new?) systems/computers/ VM's/Plague instances/whatever-you-all-will-use-for-Legcay-systems. I am also interested in helping to do whatever I can to help in lowering the bar for new sub-projects to take root. And in learning stuff to become more helpful. Regarding Legacy, I like the idea of working in Fedora's CVS system, particularly if already-existing information in cvs.fedora.redhat.com about software in legacy or to-be-legacy Fedora releases can be extracted to pre-populate the area(s) that Legacy would use in cvs.fedoraproject.org for our software maintenance activities. I am not sure what planning has been formally or informally done yet other than these: * Jesse Keating's plan about this Integration: <http://fedoraproject.org/wiki/Legacy/Status#Migration_Plan>. * Elliot Lee's response on the FAB list, here <http://www.redhat.com/archives/fedora-advisory-board/2006-June/msg00054.html> Jesse, Elliot, has anything developed on Legacy Integration since these writings? Do we have any kind of sketchy timetable in mind for any of these activities? Or would we want to establish any timetables at this juncture? Warm regards, David Eisenstein

2 1

[Fedora-infrastructure-list] public lab
by Karel Zak 28 Jun '06

28 Jun '06

Hi, is there any public bunch of computers where upstream developers can test their applications on Fedora? I know about upstream developers who don't use Linux as primary OS or they use different distributions and test upstream code with things like SELinux is difficult without access to machine with useful FC. Karel -- Karel Zak <kzak(a)redhat.com>

4 5

[Fedora-infrastructure-list] Webmaster account now part of a the ticketing system
by Mike McGrath 28 Jun '06

28 Jun '06

The webmaster(a)fedoraproject.org account is now part of the ticketing system. Emails sent to webmaster(a)fedoraproject.org will automatically get created in the webmaster queue. Users who would like access to the system to answer emails on behalf of the webmaster email address please see the web address that should be printed below. This is the first account we've put in the OTRS like this. Please let us know what you guys think of it because eventually we'd like to move other accounts into the system. As a test I'd like someone in the web-members group who already has access to the ticketing system to lock it and close it. -Mike

3 2

[Fedora-infrastructure-list] Backup thread
by Mike McGrath 27 Jun '06

27 Jun '06

Hey guys, thought I'd start a backup thread. One thing I'd like to start doing is backing up the home directories, I've had my home dir blown out a couple of times that resulted in script loss. Same thing happened recently to some of the FESCo candidates who had their access revoked to ensure a fair election :-D. Refer to previous meeting notes and list emails for whats been said so far but here's the short list: bacula, amanda, rsync and scrips, rdiff-backup dgilmore is one of the guys leading the backup stuffbut participation is encouraged by everyone. I'd like to throw backuppc in the list just because I use it / like it. It might be good for our environment. I get the sense that bacula is the current front runner. One thing should be noted. Whatever system we go with needs to be in Extras or Core so we can guarantee ongoing support. But lots of good products have gotten into our repos this way (Nagios :-P). I'd prefer whatever we choose to have native ssh tunnel support. if this isn't possible we can always create the tunnels manually. For those that aren't familiar we have 2 primary sites, one in Phoenix and one at Duke. These two sites are somewhat linked but still pretty separate. Any thoughts / suggestions?

6 7

[Fedora-infrastructure-list] DB Server Upgrade
by Curt Moore 27 Jun '06

27 Jun '06

Jeffrey/All: I've been corresponding off-list with Elliot with regard to upgrading db1. At his suggestion, I'm going to run my proposed process by everyone for feedback. I went through a production DB server upgrade a while back and this process reflects what I did then and lessons learned during the process. I also had Slony in this mix, which resulted in a few more steps. From what I've read on the wiki pages regarding the current Fedora infrastructure setup, there is only 1 DB server, a SPOF. It may be worth the time to start a separate conversation about some sort of replication or clustered setup, which a beast unto itself. I have a master/slave replication setup with Slony which works quite well. You still have the SPOF with the master but at least you can load balance between the slaves and still have read access going if the master goes down. With all of the research I've done, I've not found anything which is production ready and allows a multi master scenario for Postgres, only master/slave scenario with replication, like Slony. The closest thing I found was PGCluster but it was very flaky, at least on 8.x, the last time I played with it late last year. PGCluster was also very hardware intensive, meaning it takes like 4 machines to run a setup without a SPOF. The following assumes the desire for as minimal as possible downtime and as a result is a little more involved than if more downtime were tolerable. I welcome comments and refinements to the process. Now, onto the upgrade process. *** Make a DB and filesystem backup *** Ensure that all necessary config files are archived so they can be quickly reinstalled via kickstart later 1) Setup all apps to connect to DB using a host alias in /etc/hosts 2) Setup another server as a temporary DB server Note: An option here is to copy data to the temp DB server and do an immediate cut over to the temp server after the data has been copied but you run the risk of data being out of sync between the master and temp server. Danger, Will Robinson! 3) Temporarily disable apps hitting DB 4) pg_dump DBs and template0, users, groups, etc over to temp server 5) Test to ensure DB is up and accepting connections 6) Change the alias entry in /etc/hosts on hosts running apps to the record for the temp DB server 7) Re-enable apps 8) Re-install and configure OS on old DB server via kickstart 9) Temporarily disable apps hitting DB 10) pg_dump DBs and template0, users, groups, etc over to master server 11) Test to ensure master DB is up and accepting connections 12) Change the alias entry in /etc/hosts on servers running apps back to the record for the master DB server There, my 12 step program for a DB upgrade. :-) I had a few more Slony related steps which required shuffling the app servers between Slony slaves and the master but the above it basically the process. I made a few decisions/assumptions during my process. In my case since 99% of DB hits were reads,I still had my Slony slaves accepting read requests. I was able to do this since I maintain both read and write DB handles in the SoftSwitch. Some apps allow a "degraded" mode where data is "read only" just for this sort of thing, I'm not sure if this is the case here or not. During the time I disabled the apps to copy over the DB to the temp server, users just couldn't login to the web portal to change their settings, which are written to the master, for a few minutes. I had to ask, would this really be an issue for 5 min at 3am CST on a Sunday morning? (assuming that the DBs can be copied in the span of 5 min, I have a VLAN'd GigE management network for this sort of thing) The same downtime was experienced when I switched back over to the master DB. Since I'm assuming we're upgrading from Postgres 7.x we'll definitely want to do a pg_dump/pg_restore since there are some inherent differences in the data structures on the disk. In the past I've just stopped postgres, copied over /var/lib/pgsql to the new server and started postgres there, and called it good but you can't do that when upgrading from 7.x to 8.x. Since the data itself has to be migrated from the 7.x format to the 8.x format at some point, there is probably going to have to be some measure of downtime. Otherwise you get into having to compare data in 2 different databases to see if anything changed and manually replicating those changes to the master copy. For me, the 5 min of partial downtime was _much_ cheaper. I look forward to discussion on this. Cheers, -Curt

2 1

[Fedora-infrastructure-list] Auth Failure with extras-sync
by Warren Togami 27 Jun '06

27 Jun '06

Test sync build(a)fedora.linux.duke.edu's password: It seems that pass-through ssh key authentication isn't working anymore. Was my account disabled on fedora.linux.duke.edu due the Extras election too, or is this some other issue? Warren Togami wtogami(a)redhat.com

2 1

[Fedora-infrastructure-list] Re: Introduction
by Jason Hartley 26 Jun '06

26 Jun '06

On Fri, June 23, 2006 7:30 pm, Curt Moore wrote: > Hi Jason. > > At the suggestion of Elliot Lee, I'm contacting you about the Mirror > Management project. > > I'd like to help so please let me know what I can do. > > I saw that there was talk about borrowing the CentOS mirroring product. > Has anyone over at CentOS been contacted about this? I used to work > with some of the CentOS maintainers, but haven't spoken with them in a > while, but it may be an "in". > > Please let me know where things stand. > > Thanks! > -Curt Hey Curt, Seth Vidal has been working with Lance from the Centos team on getting the Centos mirroring code. If know or would like to contact Lance about the code, I am sure Seth would not mind. Currently we have the cron job that runs to see what mirror sites are stall. Here is the break down of Centos Mirroring: """ It's broken up into 3 parts: 1. one part queries their mysql database of the mirrors and builds a mirrorlist using geoip 2. another part is a cron job that connects to the mirrors and grabs the repomd.xml off of them to determine 'freshness' 3. the last part generates the web page based on the database info """ In addition to the 3 parts, we also need the table structure of their database. Thanks, for your help! Regards, Jason Hartley

1 0

[Fedora-infrastructure-list] Account System Updates
by TomLy 24 Jun '06

24 Jun '06

As mention in today's fedora-admin meeting, I'm looking to put together the requirements for the new account system. I'll slap the requirements on a wiki page to keep track of what's going on. ~lyz

8 10

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure June 2006