infrastructure September 2007

infrastructure@lists.fedoraproject.org

40 participants
28 discussions

Architectural Changes
by Mike McGrath 10 Sep '07

10 Sep '07

As we talked about in the meeting yesterday we have a new sponsor (http://www.teliasonera.com/) There are a couple of others in the works (I don't want to officially announce until its finalized) but one thing is clear. Pretty soon we're going to have multiple proxy servers outside of PHX. The end goal here would be to use mod_geoip to re-direct people to their nearest location but we're going to take baby steps to get there. Here are the steps as I see them. 1) Finalize the caching stuff paulobanon has been working on. 2) VPN 3) Setup 1 remote proxy server and test 4) Get DNS setup properly to direct people to the proxy servers in a RR format 5) mod_geoip. 4) is still a little fuzzy in my mind. Right now we're using Bind for DNS and, AFAIK, the version we're using does not have support for geoip. So my thought is using mod_geoip to direct people to (for example) de1.fedoraproject.org or us2.fedoraproject.org. I'm still a little unclear on the best way to do this in our environment. Those keeping an eye on the commit logs will have noticed the odd commit for t.fedoraproject.org. So, for example: ping -c1 t.fedoraproject.org For me seems to do the right thing. I get basically a RR balanced IP between 3 addresses (fp.o, yahoo and google) I just picked two ip's that weren't ours to balance around. The thing, for me at least, is I get fp.o every time if I use FireFox. This is over many days on different computers. I've seen FF bring up the google ip once. So I ask those on the list to go to http://t.fedoraproject.org/ and just tell me what you get. Or, even better, explain to me what the heck is going on there, I have one theory about first requests to DNS vs named caching in FF and name caching elsewhere. But we've had different people get many different results (some get wget to RR, some with wget always get the same thing, same with curl, lynx, w3m, and HEAD) More investigation is needed. 2) is something I'm working on now. VPN will only be for external servers (not users). We've actually already had a few issues we've had to overcome in strange ways from external servers that could have been fixed by a VPN. (puppet and bacula backups immediately come to mind) We'll tightly control (iptables) what these boxes have access to on the vpn server (bastion). We'll keep the ttl on our load balanced products lower so that if something does go wrong with one of them, we can easily take it out of the mix. The reason for 2) is so we don't have to maintain multiple different proxy server types. If we use VPN we can treat each server the same, just like the ones we have now which keeps it maintainable. Questions / Comments / Suggestions? -Mike

10 10

Off on Monday
by Mike McGrath 10 Sep '07

10 Sep '07

Just letting you guys know I'll be off on Monday, going to a cubs game and generally doing everything you've seen in: http://imdb.com/title/tt0091042/ -Mike

8 13

Fedora Infrastructure IRC Meeting Log from 2007-09-06
by Ricky Zhou 06 Sep '07

06 Sep '07

16:04:35 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Role Call 16:04:37 < mmcgrath> who's here? 16:04:39 < skvidal> I am 16:04:45 < skvidal> one sec I'll be on the bridge 16:04:55 * ricky is here, but will probably have to go very shortly. 16:04:58 * glezos is here 16:05:19 * lmacken 16:05:19 -!- jeremy [i=katzj@nat/redhat/x-48f778bf0db0139e] has quit Remote closed the connection 16:05:19 -!- warren [i=warren@redhat/wombat/warren] has quit Remote closed the connection 16:05:21 * abadger1999 here 16:05:22 < lmacken> heh 16:05:24 < lmacken> clockwork 16:05:29 < mmcgrath> there they go. 16:05:41 < abadger1999> Some people just don't want to do an honest days work :-) 16:05:43 < mmcgrath> I know dgilmore is traveling a lot this week. 16:05:49 < frankc> Frank is listening. 16:05:59 < mmcgrath> hello frank 16:06:01 < mmcgrath> paulobanon: ping? 16:06:37 < mmcgrath> Ok, lets get started. 16:06:42 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Tickets 16:06:50 < mmcgrath> https://hosted.fedoraproject.org/projects/fedora-infrastructure/query?statu… 16:06:54 < mmcgrath> We really only have the one on the list 16:07:09 < mmcgrath> and jcollie isn't around right now. 16:07:25 < mmcgrath> The lack of interest in that from the community in general leads me to believe that we'll be on CVS for a long time to come. 16:07:28 < mmcgrath> just a prediction :) 16:08:12 < mmcgrath> This will probably be a short meeting, not much happened in the last week. 16:08:17 < skvidal> I think most people don't want to hear the flame war 16:08:19 < abadger1999> heh. It's not (really) broken so don't fix it. 16:08:47 < mmcgrath> heh, thats true. 16:08:52 < mmcgrath> Ok, we'll move on to the schedule next. 16:08:56 < glezos> skvidal: what if there's no flame work and just action? 16:08:57 < lennert> ... 16:09:03 < skvidal> glezos: hahahahahahaha 16:09:05 < skvidal> glezos: that's funny 16:09:12 < skvidal> glezos: are you a comedian for your day job? :) 16:09:15 < glezos> :) 16:09:19 < ricky> Darn- I must go. 16:09:25 < mmcgrath> ricky: later ricky! 16:09:29 * ricky will read the logs in a bit :) 16:09:34 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- http://fedoraproject.org/wiki/Infrastructure/Schedule/ 16:09:40 < glezos> skvidal: we're infra, we can do anything :P 16:09:49 < skvidal> right-o :) 16:09:57 < mmcgrath> I might remove the VCS from the schedule since there's been little progress in it and it is already in the tickets. 16:10:11 < mmcgrath> As far as sponsorship goes we have a new member to add to the web group - glezos 16:10:14 < mmcgrath> welcome glezos. 16:10:23 -!- tibbs|h [n=tibbs@fedora/tibbs] has quit Read error: 104 (Connection reset by peer) 16:10:24 < skvidal> yay glezos! 16:10:28 < mmcgrath> s/web group/sysadmin-web group/g 16:10:30 < glezos> mmcgrath: glad to be here. 16:10:37 -!- tibbs|h [n=tibbs@fedora/tibbs] has joined #fedora-meeting 16:10:42 < mmcgrath> glezos: have you had a chance to verify your access to bastion? 16:10:54 < glezos> mmcgrath: yes, it seems to work fine, thanks 16:11:02 -!- JSchmitt [n=s4504kr@fedora/JSchmitt] has quit "Konversation terminated!" 16:11:25 < mmcgrath> and nothing new on the SOP stuff. 16:11:39 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Corporate Sponsors 16:11:50 < mmcgrath> So there's some good news and some same old news on the sponsorship front. 16:12:02 -!- jeremy [i=katzj@nat/redhat/x-35dd6359cde2db97] has joined #fedora-meeting 16:12:06 < mmcgrath> we've been offered a half rack + BW from a colo in Germany 16:12:09 < skvidal> jeremy: you're late! 16:12:24 < skvidal> mmcgrath: cool, well done 16:12:35 < mmcgrath> The company is TeliaSonera 16:12:42 < skvidal> pretty name 16:12:45 < mmcgrath> :) 16:12:46 < mmcgrath> yeah 16:13:22 < lennert> telia is a carrier 16:13:35 < mmcgrath> and we get free remote hands from Proio.com 16:14:15 < mmcgrath> So for us its just a matter of finding some servers to put over there. 16:14:27 < mmcgrath> I'm working on that but if the IBM's or Dell's out there hear anything let me know. 16:14:33 < skvidal> mmcgrath: so we hav 21U 16:14:41 < skvidal> do we have money to buy servers w/yet? 16:14:45 < mmcgrath> skvidal: I believe so. 16:15:01 < skvidal> ie: can we just spend 9K and get a 1950 in germany? 16:15:05 < mmcgrath> We've never really had a budget so I'm working on that. We did get money for the warrantys. 16:15:09 < mmcgrath> I'm hoping so. 16:15:29 < mmcgrath> Back in July/June I created an end of the year budget that included 2 server purchases (both 1950's) 16:15:40 < mmcgrath> so, if that gets approved as I think it has, then we can purchase and ship. 16:15:54 < mmcgrath> I'd stick one 1950 in PHX and one 1950 in Germany (probably through dell.de) 16:16:01 < skvidal> sounds like a good plan 16:16:14 < mmcgrath> the req process internally isn't the most fun and people are busy and haven't been as responsive as I'd hoped. 16:16:43 < mmcgrath> So thats the latest on that, any questions? 16:17:08 < glezos> mmcgrath: any ideas on how we're going to promote this support? banners, announcement? 16:17:29 < mmcgrath> glezos: I'm glad you asked, especially since you have an eye for good looking pages and access 16:17:33 * mmcgrath grabs ticket 16:17:47 < mmcgrath> This is it here - https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/137 16:18:00 < mmcgrath> I think ricky may have done some preliminary work on this but I'm not sure how far he's gotten. 16:18:13 < mmcgrath> I've sent a few emails to the websites list (including the cattle call one). 16:18:22 < glezos> mmcgrath: I'll take a look as well 16:18:30 < mmcgrath> I asked if there were still people on the list, they said yes, and some said they weren't sure how to get started. 16:18:44 < mmcgrath> so I sent that ticket explictly out to see if anyone would make a page but I've gotten no response :( 16:18:55 < mmcgrath> glezos: I'd appreciate that. 16:18:57 < glezos> mmcgrath: what about an announcement? 16:19:08 < mmcgrath> glezos: ? what did you have in mind? 16:19:46 < glezos> mmcgrath: dunno, if we do need more support then it could act as a "thanks" and invitation for more support 16:19:55 < glezos> on FWN for example 16:20:32 < mmcgrath> we've talked about some of that stuff, I'd like to save some of the announcements for the larger donations, like if IBM gave us 200 grand in hardware or something. 16:20:46 < glezos> mmcgrath: agreed 16:20:56 < mmcgrath> so far though no ones requested it. Its hard to put a value on donation because, in theory, we might have a bunch. 16:21:13 < mmcgrath> But, since the "powered by dell" logo is the only thing we have right now I'd at least like to get the sponsorship page and such setup. 16:21:45 < glezos> mmcgrath: right 16:21:47 < mmcgrath> so anywho, thats all we've got on that for this week. 16:21:52 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Open Floor 16:22:02 < lmacken> https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/141 16:22:04 < mmcgrath> Its been a pretty quiet week, I've been making lots of little changes. 16:22:14 < lmacken> are there any account system guys that could take care of that ? 16:22:23 < skvidal> mmcgrath: is lockbox pretty much use-free now? 16:22:46 < mmcgrath> skvidal: pretty close, we still have infrastructure.fp.o to move off of there (its stored on the netapp but served through lockbox) 16:22:56 < mmcgrath> and we still deploy some apps via lockbox though that should be trivial to fix. 16:23:03 < skvidal> mmcgrath: what does infrastructure need? 16:23:17 < mmcgrath> just some place to mount that share elsewhere. 16:23:35 * f13 peeks in 16:23:36 < abadger1999> Are we mounting the full netapp on puppet1 now? 16:23:36 < skvidal> okay b/c when its free I can bump it to rhel5 and setup reposync 16:23:38 < mmcgrath> we need to find a different master server to attach to the netapp, right now lockbox is the only server that can rw to everything on the netapp as root. 16:23:46 < mmcgrath> abadger1999: we might be but it can't write to it. 16:23:52 < abadger1999> k 16:23:55 < mmcgrath> lmacken: we'll talk about that in a sec. 16:23:59 < mmcgrath> skvidal: what did you have in mind? 16:24:21 < skvidal> I don't need much - just a rhel5 instance with entitlements and disk space 16:24:37 < skvidal> so I can get it to sync down rhel5, rhel5-virt, and rhel5-updates 16:24:48 < mmcgrath> <nod> 16:24:58 < mmcgrath> I'll put in a ticket request for that today. 16:25:03 < skvidal> you suggested lockbox before 16:25:16 < skvidal> and I had other things going on so I was waiting for it to free up to do the reposync bit 16:25:37 < abadger1999> The hosted trac repositories sync via the netapp mount on lockbox as well. 16:25:40 < mmcgrath> lockbox would be good but A) its almost dead (EOL'd) and B) I want to stick the sync mechanism on a xen instance. 16:25:58 < mmcgrath> abadger1999: yeah, there's a few cron jobs we'll need to migrate. 16:26:20 < mmcgrath> lmacken: ok, lets talk about #141 16:26:28 < lmacken> ok 16:26:53 < mmcgrath> how are they obtained now? Which pages on the wiki? 16:27:00 < skvidal> mmcgrath: okay if there's a better choice of a box to stick a xen instance on where I can write to the netapp, I can do it there 16:27:02 < lmacken> No idea.. I don't know anything about those certs 16:27:45 -!- caillon [n=caillon(a)mithril.returnzero.com] has left #fedora-meeting [] 16:28:35 * jeremy looks 16:28:56 < mmcgrath> http://fedoraproject.org/wiki/PackageMaintainers/UsingPlagueClientFaq 16:29:10 < abadger1999> lmacken: Does till just want us to host the files available from the wiki on admin.fp.o? 16:29:29 * dgilmore is here 16:29:30 < mmcgrath> abadger1999: I think the concern is that someone could go in and change those files. 16:29:44 < lmacken> yeah, in "a secure way".. so I'm assuming he wants us to host it from https://admin.fp.o/accounts 16:30:02 < jeremy> yeah, that's what it looks like 16:30:07 < jeremy> and we definitely should! 16:30:12 < lmacken> definitely 16:30:25 < mmcgrath> yep, easy fix. lmacken if you don't want to do it assign the ticket to me, I'll do it after the meeting. 16:30:52 < lmacken> sounds good. I don't know my way around the account system.. So i'll reassign :) 16:31:09 < mmcgrath> lmacken: I'm just going to stick it under /accounts/ somewhere and change the links on the wiki. 16:31:21 < mmcgrath> Does anyone have anything else they'd like to discuss? 16:31:26 < glezos> mmcgrath: As a heads-up, I'll create a ticket now for what we need for transifex 16:31:32 < mmcgrath> glezos: k 16:31:35 < abadger1999> wishlist item: I'd like to have a ~/.fedora/ directory for bodhi/koji/etc config, sessioncookies, certs. 16:31:51 < jeremy> abadger1999: makes sense to me 16:31:56 < mmcgrath> abadger1999: we can do that. 16:32:48 < mmcgrath> Ok, anything else? If not I'll close the meeting in 30 16:33:16 < mmcgrath> 10 16:33:21 < skvidal> thanks mmcgrath 16:33:29 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Meeting End 16:33:46 < mmcgrath> thanks for coming guys. 16:33:48 -!- frankc [i=824c405d(a)gateway/web/cgi-irc/ircatwork.com/x-c083a3ac2d3221c1] has left #fedora-meeting [] 16:33:50 < mmcgrath> skvidal: any time :) 16:34:16 -!- mmcgrath changed the topic of #fedora-meeting to: Channel is used by various Fedora groups and committees for their regular meetings | Note that meetings often get logged | For questions about using Fedora please ask in #fedora | See http://fedoraproject.org/wiki/Communicate/FedoraMeetingChannel for meeting schedule 16:35:05 < dgilmore> mmcgrath: you will need to co-ordinate the change so that fedora-packager-setup.sh gets the certs from the new location 16:35:39 < mmcgrath> dgilmore: where is the canonical source for fedora-packager-setup.sh 16:35:43 < dgilmore> koji 16:35:50 < dgilmore> its in the koji package 16:36:38 * dgilmore has been meaning to do up a fedora-maintainer package that has it and pulls in koji/plague/mock etc so someone can easily get a environment up 16:36:40 < mmcgrath> dgilmore: can do, I'll file a bug. 16:39:47 -!- halfline_ is now known as halfline 16:39:56 < f13> dgilmore: do it as a comps group instead. 16:40:09 < f13> dgilmore: can still do 'yum groupinstall fedora-dev-kit' 16:40:16 < f13> and can select it at install time, or graphically from pirut 16:40:23 < f13> (and can be easily included in spins) 16:41:40 < dgilmore> f13: no because we want a place to put scripts etc 16:42:24 < f13> sure, so you can have a package that does that, but also a comps group that includes the koji package, your package, whatever other tools we pick and choose 16:42:26 < dgilmore> f13: fedora-packager-setup.sh does not really belong in koji. 16:42:36 < f13> and we make those defaults, but not mandatory so that at install time you can adjust. 16:42:41 < f13> dgilmore: I don't disagree 16:42:42 < dgilmore> sure that could be done 16:42:57 < f13> I just don't want you to do a 'metapackage' with unnecessary Requires. 16:43:57 < dgilmore> we can make that work 16:44:10 < dgilmore> I should set aside time and just do it

1 0

imtiaz.rahi@fedoraproject.org address not working
by Imtiaz Rahi 06 Sep '07

06 Sep '07

Hi, I am a fedora ambassador from Bangladesh and also fedora FreeMedia contributor. Just today I found out that my fp.org mail address ( imtiaz.rahi(a)fedoraproject.org) is not working. Mail sent to this address returns with an error user unknown. I understand that imtiaz(a)fp.org addresses are not working anymore. But does my imtiaz.rahi(a)fp.org address got deleted or disabled during that process. Can anyone please shed some light regarding this matter. Cheers, Imtiaz http://fedoraproject.org/wiki/ImtiazRahi

2 1

mirrormanager future features
by Matt Domsch 04 Sep '07

04 Sep '07

MirrorManager, for what I really wanted to see by the Fedora 7 release, has been a success. But there are still several gotchas I'd like to iron out before Fedora 8. * The mirrorlist mod_python applet consumes too much memory on the app servers. It basically reads in a 2MB mirrorlist_cache pickle file which is lists, by directory, of what mirrors hold what content. Handy to have, but in mod_python, that blows the RSS size out to ~27MB per process, times all the httpd processes that have run that code, each with their own private copy. Not pretty. The mirrormanager TurboGears backend isn't fast enough to handle all the client requests for mirrorlists, hence I exported the data for mod_python to use. But the mod_python trick takes too much memory. The way out? Split the mod_python applet into two pieces: 1) Yet another daemon, listening on a local UNIX socket, that has a copy of the mirrorlist cache. It calculates the answers to return. 2) The mod_python applet connects to the daemon, passes it's list of args, and gets back the answer list. It handles redirects too. In this way, the daemon can fork() itself if necessary to handle the traffic, but those forks() use copy-on-write memory, and the children will never touch the pickle, so they'll all share mostly the same memory. One copy of the mirrorlist_cache, used by all children. Since I'm saving so much RSS memory here, I can add back into the mirrorlist_cache all the directories which are being omitted now. So, we will be able to return the list for any dir or file that the public mirrors know about, not just a few as we do now. I've got a stab at this, but am still working on the details. I'll want to do some time tests against the new code, to make sure it isn't too much slower for clients, but a quick swag shows it'll be OK; 0.3sec or so per request, even in parallel, which IMHO is "good enough". * Mike's redirection stuff is included in the above already, so that'll be online as soon as the rest is. Now, to find the time before F8... Still to come, provided I find a lot of time (unlikely), or someone else steps up to help: * Designate a way for mirrors to claim themselves to be always up-to-date. Probably will require a sysadmin to set this bit, as it's somewhat dangerous. But there are cases, e.g. a local out-of-line squid proxy, where it makes sense to do it. This change will change the schema, and has repercussions throughout the code, so I haven't wanted to make it lightly. * Some people want metalink support. Conceptually it's possible, and even pretty easy once we've got the daemon above working right. But as noted on f-d-l, it's been 10 weeks since someone asked for it and even sent some code that doesn't quite integrate but was a starting point, and I haven't had time to get to it. It's not looking good for me to add that right now, but I'd be happy to review patches. * I've wanted to add the libgeoip country->continent mappings, so we can fall back netblock -> country -> continent -> global but I don't know C->Python bindings code at all, and need that exported in python-GeoIP for mm to use. * I've got pending a request to change the fedora.repo files to make yum treat the list as in priority order. I really want the continent mappings in place before doing that though... Should we let countries with <3 mirrors return their own lists? Right now if a country has <3 mirrors, the users get the global list back. Anything else people really need to see? Thanks, Matt -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux

6 8

Re: Improving availability and guaranteeing integrity in ISO downloads
by Anthony Bryan 04 Sep '07

04 Sep '07

> http://www.getright.com/seedtorrent.html > > Supported by Azureus, among others. We already have an extensive > HTTP/FTP mirror system to leverage. > > I've noticed, after the initial release rush, torrents end up being > quite a bit slower than just downloading from a mirror. Especially on a > less popular arch. (cough ppc cough...) In the past I've just stopped > the torrent, downloaded the iso from a mirror, then restarted the > torrent to help seed. > > It would be nice to just have this happen automagically. That's how Metalink works with clients that also support P2P networks. GetRight supports this w/ metalink, hopefully KGet will eventually along with aria2. Phex also supports it over gnutella. (Sorry for the late reply) > On Tue, Jun 19, 2007 at 03:40:59PM -0400, Anthony Bryan wrote: > > >On Sat, Jun 09, 2007 at 07:51:20PM +0200, Ruben Kerkhof wrote: > > > > Have you had a chance to look over Ruben's additions? Any feedback? He > > said he re-licensed it to line up w/ mirrormanager. Any ideas/comments > > for features in Metalink that could be of use to Fedora? > > Yes, I took a quick look; I'll be able to do something with this, but > not for the next 4 weeks, as I'm out of the office moving houses and > on vacation, then catching up on real work. :-) checking in after 10 wks :) Will what Ruben submitted be usable? a few more metalink apps have been released. Free Download Manager (Win) has been released under GPL3. DownThemAll 1.0b2 firefox extension is out, and displays more of the info contained in a metalink to the person downloading, like a how many mirrors are listed, logo, description, version, os/arch, and other stuff that could be useful. here are some screenshots the DTA team put up: http://code.downthemall.net/maierman/metalink-test/metalink-feisty.png http://code.downthemall.net/maierman/metalink-test/metalink-xp.png http://code.downthemall.net/maierman/metalink-test/metalink-comment.png there's also Celerius, a GTK python downloader in progress at http://celerius.tuxfamily.org/ if anyone wants to help out. -- (( Anthony Bryan ... Metalink [ http://www.metalinker.org ] )) Easier, More Reliable, Self Healing Downloads

2 1

Problem with sending e-mails to Fedora mailing list
by Bukovansky Richard 03 Sep '07

03 Sep '07

Hi, I'm not sure I asking the right group, but I need help. When I send an e-mail to any Fedora mailing list I'm subscribed to, my e-mails are not delivered back to me even I have set "Receive your own posts to the list?" to yes. My e-mail is delived to everybody in the list, it's showing in mailing list archive. I.e.: https://www.redhat.com/archives/fedora-cs-list/2007-August/msg00073.html What is suprising me more, I'm getting this kind of error e-mails back after while: This is the Postfix program at host smtp-out4.iol.cz. #################################################################### # THIS IS A WARNING ONLY. YOU DO NOT NEED TO RESEND YOUR MESSAGE. # #################################################################### Your message could not be delivered for 8.0 hours. It will be retried until it is 5.0 days old. For further assistance, please send mail to <postmaster> If you do so, please include this problem report. You can delete your own text from the attached returned message. The Postfix program <fedora-cs-list(a)rehdat.com>: connect to rehdat.com[208.73.212.12]: Connection refused Or once I got this one: This is the Postfix program at host smtp-out3.iol.cz. #################################################################### # THIS IS A WARNING ONLY. YOU DO NOT NEED TO RESEND YOUR MESSAGE. # #################################################################### Your message could not be delivered for 8.0 hours. It will be retried until it is 5.0 days old. For further assistance, please send mail to <postmaster> If you do so, please include this problem report. You can delete your own text from the attached returned message. The Postfix program <fedora-cs-list(a)rehdat.com>: connect to rehdat.com[69.25.212.153]: Connection timed out It was working normally before and I'm not sure what can be wrong. Thank you very much. -- Bukovansky Richard richard.bukovansky(a)atlas.cz

2 2

Private builds in koji
by Ignacio Vazquez-Abrams 03 Sep '07

03 Sep '07

A couple of questions that haven't been brought up yet but have recently passed through my mind: 1) Is it considered an acceptable use of resources to use koji to scratch-build private FOSS packages that will be hosted on fedorapeople.org? 2) What about pointing people to the builds hosted within koji? 3) What about building in koji but hosting elsewhere? I think it's safe to assume that building anything non-FOSS in koji is *completely* out of the question. -- Ignacio Vazquez-Abrams <ivazqueznet(a)gmail.com> PLEASE don't CC me; I'm already subscribed

4 4

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure September 2007