mirroring for alternative content

[Kevin Fenzi]

On Sat, 21 Jun 2014 07:25:47 -0700
Colin Walters <walters at verbum.org> wrote:

> Hi,
> 
> We have several forms of non-Koji internal builds:
> 
> 1) COPR (the most visible)
> 2) http://alt.fedoraproject.org/pub/alt/rawhide-kernel-nodebug/  (this
> could be a COPR)
> 3) rpm-ostree: http://rpm-ostree.cloud.fedoraproject.org/#/
> 
> Now, I recently was allocated a new atomic01.qa.fedoraproject.org
> server dedicated to rpm-ostree composes, and I'm happy to announce I
> now have it composing trees:
> 
> fedora-atomic/rawhide/x86_64/server/docker-host =>
> 175510ab77ef39ec6000db2745c70444d55957e38c5ff28f890237e1f012ea5e
> fedora-atomic/rawhide/x86_64/workstation/gnome/core =>
> b6f3f3b53ef957c71e8d8bdc36c35519888d4177bf6a582ba37e33416ff84216
> 
> But it's on the QA network, and so I can't just serve the content
> directly from it.  Nor would I want to have the compose server doing
> double duty as a static webserver anyways.

Yeah, we can actually map an external IP into it if required, but I
agree it isn't great to have the build host also serving external
content. 

> I could just rsync this content to the current
> rpm-ostree.cloud.fedoraproject.org, but there are a few issues with
> that:
> 
> 1) It's a one off instance in the fedora cloud with no
> backup/monitoring 2) No redundancy
> 3) No tuning for static webserving
> 4) Most critically, no TLS

Yep. 
 
> COPR only relatively recently gained TLS, which is fundamentally
> necessary for retrieving arbitrary executable code that runs as root.
> 
> But COPR also (from what I understand from <puiterwijk>) is just one
> machine.

Correct. 
 
> What do you think about investing in having e.g.
> altcdn.fedoraproject.org be a load-balanced static webserver farm
> protected by TLS that could be written to by a trusted subset of
> non-Koji build/compose tooling?
> 
> "altcdn" is the first thing that came to mind offhand for a name, feel
> free to pick others =)
> 
> We'd need to determine the writing process; could be rsync-over-ssh
> access to specific subdirectories gated by ssh key?  Something else?

I think this is pretty much already in place with our /pub/alt/ space
(that the kernel nodebug uses above). 

This space is on a netapp. 
It is mirrored, but only by a few larger mirrors, most don't. 
It's served by our 5 download servers in phx2 and 3 on I2 in rdu2. 

So, we could sync the ostree stuff there as well... its a large number
of small files? Or can you expand on the size and content it will have
over time and how often updated?

kevin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/infrastructure/attachments/20140621/ad15488d/attachment.sig>