2:31 a.m.
Hi, all,
Included are package for iso watcher, which use lftp to mirror dl.fp.org/pub/alt/stage and
kick off mediakit_sanity test,
package for post-iso-build event, and package for mediakit_sanity test which test the
iso's size, md5, sha256sum, repoclosure
and file conflicts.
During the first run of /iso/watcher.py, it will mirror all the isos at
dl.fp.org/pub/alt/stage and kick off mekiakit_sanity test,
after that, it will only mirror the new isos, the tree compose and deltaiso are not
included in the mirror.
This is my maiden work, thanks for your review.
Hongqing
8:19 a.m.
New subject: iso watcher, post-iso-build event, mediakit_sanity test patch for review
Hi, all,
Included are package for iso watcher, which use lftp to mirror
dl.fp.org/pub/alt/stage and kick off mediakit_sanity test,
package for post-iso-build event, and package for mediakit_sanity test
which test the iso's size, md5, sha256sum, repoclosure
and file conflicts.
During the first run of /iso/watcher.py, it will mirror all the isos
at dl.fp.org/pub/alt/stage and kick off mekiakit_sanity test,
after that, it will only mirror the new isos, the tree compose and
deltaiso are not included in the mirror.
This is my maiden work, thanks for your review.
I pushed your code into origin/mediakit, so that it's easier to have a look at it.
Let's develop it in there from now on.
8:27 a.m.
New subject: iso watcher, post-iso-build event, mediakit_sanity test patch for review
Hi,
few comments below:
1) watchers/iso/watcher.py;
* line 77: harnesscall =
['autoqa','post-iso-build',os.path.join(opts.cachedir,iso_file),'--test','mediakit_sanity']
I don't like the "--test mediakit_sanity" parameter. This is
controlled by the test's control.autoqa, and should not be force-runned by the
watcher.
2) tests/mediakit_sanity/mediakit_sanity.py;
* Overall, I'd like to see some comments, at least about the "hitmarker"
variable. Maybe comments describing the process of getting the results from the output
too.
* line 56: starts with tab instead of 8 spaces.
3) tests/mediakit_sanity/sanity.py
* Docstrings for the methods would be great ;-)
* line 190: It would be much better to create the temporary directory using
tempfile.mkdtemp() <http://docs.python.org/library/tempfile.html#tempfile.mkdtemp>;.
Also deleting the temporary directory afterwards, might be a good idea.
* def list_file_names(path) method (lines 65 - 68): what is the purpose of this
method? Plain os.listdir() does exactly the same.
* def check_iso_size(iso_path) method (lines 77 - 103):
* I'd vote for moving the "magic constants" to named constants. It
might seem like a pointless trouble, but the code is easier to understand with named
constants instead of plain numbers.
* Why exactly is the size compared to 2 (i.e. why is size "1" wrong, but
size "2" ok?). Also, this can be done using "if 2 < size < 700:"
construct ;-)
* def iso_to_repo(file_path, mount_dir) (lines 149 - 159): It would be better to
return a tuple containing the two strings (i.e. "return (repoid, mount_dir)").
You can then easily acces both "halves" without the need to split the string all
the time.
Other than that, it seems quite OK to me.
Regards
joza
----- Original Message -----
From: "Hongqing Yang" <hoyang(a)redhat.com>
To: "AutoQA development" <autoqa-devel(a)lists.fedorahosted.org>
Sent: Friday, September 9, 2011 9:31:55 AM
Subject: iso watcher, post-iso-build event, mediakit_sanity test patch for review
Hi, all,
Included are package for iso watcher, which use lftp to mirror
dl.fp.org/pub/alt/stage and kick off mediakit_sanity test,
package for post-iso-build event, and package for mediakit_sanity test
which test the iso's size, md5, sha256sum, repoclosure
and file conflicts.
During the first run of /iso/watcher.py, it will mirror all the isos
at dl.fp.org/pub/alt/stage and kick off mekiakit_sanity test,
after that, it will only mirror the new isos, the tree compose and
deltaiso are not included in the mirror.
This is my maiden work, thanks for your review.
Hongqing
_______________________________________________
autoqa-devel mailing list
autoqa-devel(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/autoqa-devel
9:40 a.m.
New subject: iso watcher, post-iso-build event, mediakit_sanity test patch for review
Hi, all,
Included are package for iso watcher, which use lftp to mirror
dl.fp.org/pub/alt/stage and kick off mediakit_sanity test,
package for post-iso-build event, and package for mediakit_sanity test
which test the iso's size, md5, sha256sum, repoclosure
and file conflicts.
During the first run of /iso/watcher.py, it will mirror all the isos
at dl.fp.org/pub/alt/stage and kick off mekiakit_sanity test,
after that, it will only mirror the new isos, the tree compose and
deltaiso are not included in the mirror.
Hello, I haven't examined the code closely, but I have some overall architecture
thoughts:
1. We need to speak to RelEng team about http://dl.fedoraproject.org/pub/alt/stage/
We can definitely monitor that URL in the worst case, but it would be so much better if
RelEng team could ensure the following for us:
a) Consistent file naming
I have a feeling that currently all the directories there are manually created (or at
least manually named). If there were some systematic approach that we can rely on, it
would help us greatly (e.g. in determining the latest release).
b) No irrelevant files there
Currently there are not just TC/RC composes, but also "deltaisos/" and
"f16-translation/" directories that we need to blacklist. I am afraid
"stage/" serves as a storage directory for anything needed and those
"clutter" directories will change over time. It would be great to know that no
other unwanted files appear there anytime.
c) Rsync possibility
Using rsync would simplify the mirroring process at least for our Red Hat offices. I
bet we could also use it in AutoQA if desired.
2. Mirroring everything seems overkill. Personally I don't have Internet bandwidth nor
hard disk space to run this test. Can we modify the watcher so that it detects just the
last compose and downloads that one?
3. Only now I realized that watcher downloading files doesn't help us. Because the
files will be accessible on the AutoQA server, but we need them on AutoQA clients. That
means that your code works as long as you run it with "--local", but won't
work with external clients. It also means we have to work on ticket #350 "Add support
for using file proxy" (I'll try to do that soon). Ideally we will set up the
proxy on AutoQA server and all clients will be using it. The first test will download the
ISO and the other tests (if there are some) will just use the cached version (assuming
there is a fast network between the server and the client, otherwise it is of no use).
And an extra code-related question at the end:
4. Do we need labels = ['virt_capable'] in control.autoqa? This test doesn't
seem to be requiring any form of virtualization capabilities.
To sum it up, I believe we will need to do some changes. But definitely thanks for your
work!
Kamil
9:39 p.m.
New subject: iso watcher, post-iso-build event, mediakit_sanity test patch for review
----- Original Message -----
From: "Kamil Paral" <kparal(a)redhat.com>
To: "AutoQA development" <autoqa-devel(a)lists.fedorahosted.org>
Sent: Tuesday, September 13, 2011 10:40:45 PM
Subject: Re: iso watcher, post-iso-build event, mediakit_sanity test patch for review
> Hi, all,
>
> Included are package for iso watcher, which use lftp to mirror
> dl.fp.org/pub/alt/stage and kick off mediakit_sanity test,
> package for post-iso-build event, and package for mediakit_sanity
> test
> which test the iso's size, md5, sha256sum, repoclosure
> and file conflicts.
> During the first run of /iso/watcher.py, it will mirror all the isos
> at dl.fp.org/pub/alt/stage and kick off mekiakit_sanity test,
> after that, it will only mirror the new isos, the tree compose and
> deltaiso are not included in the mirror.
Hello, I haven't examined the code closely, but I have some overall
architecture thoughts:
1. We need to speak to RelEng team about
http://dl.fedoraproject.org/pub/alt/stage/
We can definitely monitor that URL in the worst case, but it would be
so much better if RelEng team could ensure the following for us:
a) Consistent file naming
I have a feeling that currently all the directories there are manually
created (or at least manually named). If there were some systematic
approach that we can rely on, it would help us greatly (e.g. in
determining the latest release).
b) No irrelevant files there
Currently there are not just TC/RC composes, but also "deltaisos/" and
"f16-translation/" directories that we need to blacklist. I am afraid
"stage/" serves as a storage directory for anything needed and those
"clutter" directories will change over time. It would be great to know
that no other unwanted files appear there anytime.
c) Rsync possibility
Using rsync would simplify the mirroring process at least for our Red
Hat offices. I bet we could also use it in AutoQA if desired.
2. Mirroring everything seems overkill. Personally I don't have
Internet bandwidth nor hard disk space to run this test. Can we modify
the watcher so that it detects just the last compose and downloads
that one?
3. Only now I realized that watcher downloading files doesn't help us.
Because the files will be accessible on the AutoQA server, but we need
them on AutoQA clients. That means that your code works as long as you
run it with "--local", but won't work with external clients. It also
means we have to work on ticket #350 "Add support for using file
proxy" (I'll try to do that soon). Ideally we will set up the proxy on
AutoQA server and all clients will be using it. The first test will
download the ISO and the other tests (if there are some) will just use
the cached version (assuming there is a fast network between the
server and the client, otherwise it is of no use).
And an extra code-related question at the end:
4. Do we need labels = ['virt_capable'] in control.autoqa? This test
doesn't seem to be requiring any form of virtualization capabilities.
To sum it up, I believe we will need to do some changes. But
definitely thanks for your work!
Kamil
Kamil,
Thanks for reviewing the code, I will modify that. About dl.fp.o/alt/stage,
I will submit a ticket to relEng. I am glad to see if you can set up the proxy.
Thanks.
12:44 a.m.
Kamil Paral <kparal <at> redhat.com> writes:
b) No irrelevant files there
Currently there are not just TC/RC composes, but also "deltaisos/" and
"f16-translation/" directories
that we need to blacklist. I am afraid "stage/" serves as a
storage directory
for anything needed and those
"clutter" directories will change over time. It would be
great to know that no
other unwanted files appear
there anytime.
Install TCs/RCs should always be in
http://dl.fedoraproject.org/pub/alt/stage/*/Fedora/, and lives in
http://dl.fedoraproject.org/pub/alt/stage/*/Live/. Would it be enough to just
ignore everything else?
c) Rsync possibility
Using rsync would simplify the mirroring process at least for our Red Hat
offices. I bet we could also use it
in AutoQA if desired.
2. Mirroring everything seems overkill. Personally I don't have Internet
bandwidth nor hard disk space to
run this test. Can we modify the watcher so that it detects just the
last
compose and downloads that one?
3. Only now I realized that watcher downloading files doesn't help us. Because
the files will be accessible
on the AutoQA server, but we need them on AutoQA clients. That means
that your
code works as long as you run it
with "--local", but won't work with external clients.
It also means we have to
work on ticket #350 "Add
support for using file proxy" (I'll try to do that soon).
Ideally we will set
up the proxy on AutoQA server
and all clients will be using it. The first test will download the
ISO and the
other tests (if there are some)
will just use the cached version (assuming there is a fast network
between the
server and the client,
otherwise it is of no use).
Have you considered trying to get an account on one of the machines where the
files already exist (such as alt.fedoraproject.org) and doing the tests on that
machine directly, without downloading? I have an account on alt.fp.o (for making
delta ISOs) and can do the size and checksum tests. The repoclosure and file
conflicts tests require loop-mounting the ISO, which can be done with
fuse/fuseiso without root privileges (as I've verified on my home machine), even
though the test instructions currently say to do it as root with the mount
command. Unfortunately, the fusermount executable in the fuse package currently
requires SUID, so it may not be possible for security reasons, but you could
check if there's a workaround. (There ought to be one - the tests simply examine
the contents of the ISO, and there's no good reason root privileges should be
needed at all.)
8:58 a.m.
Install TCs/RCs should always be in
http://dl.fedoraproject.org/pub/alt/stage/*/Fedora/, and lives in
http://dl.fedoraproject.org/pub/alt/stage/*/Live/. Would it be enough
to just
ignore everything else?
It is doable, but submerging into individual directories is not optimal, because it is
accessible only through HTTP. It's not really a one-liner. And more code -> more
failures. But definitely a way to go in the worst case.
Have you considered trying to get an account on one of the machines
where the
files already exist (such as alt.fedoraproject.org) and doing the
tests on that
machine directly, without downloading? I have an account on alt.fp.o
(for making
delta ISOs) and can do the size and checksum tests. The repoclosure
and file
conflicts tests require loop-mounting the ISO, which can be done with
fuse/fuseiso without root privileges (as I've verified on my home
machine), even
though the test instructions currently say to do it as root with the
mount
command. Unfortunately, the fusermount executable in the fuse package
currently
requires SUID, so it may not be possible for security reasons, but
you could
check if there's a workaround. (There ought to be one - the tests
simply examine
the contents of the ISO, and there's no good reason root privileges
should be
needed at all.)
Well, this is an interesting proposal. The test client could SSH into the machine, execute
the commands and receive the output. This of course brings the question of authentication
(will we copy the SSH private key to every test client?) and security (infrastructure guys
wouldn't be probably enthusiastic about this). I would rather set up our machine, that
would mirror everything ISO-related and then execute the tests there. That is also doable
today I guess. Just the mirroring process is not ideal with http-only solution.
Thanks, Andre, for interesting ideas!
9:04 p.m.
----- Original Message -----
From: "Kamil Paral" <kparal(a)redhat.com>
To: "AutoQA development" <autoqa-devel(a)lists.fedorahosted.org>
Sent: Monday, September 19, 2011 9:58:11 PM
Subject: Re: iso watcher, post-iso-build event, mediakit_sanity test patch for review
> Install TCs/RCs should always be in
> http://dl.fedoraproject.org/pub/alt/stage/*/Fedora/, and lives in
> http://dl.fedoraproject.org/pub/alt/stage/*/Live/. Would it be
> enough
> to just
> ignore everything else?
It is doable, but submerging into individual directories is not
optimal, because it is accessible only through HTTP. It's not really
a one-liner. And more code -> more failures. But definitely a way to
go in the worst case.
The lftp also has a --include function, I have tried it, it does not work, I will
continue investigating it.
And James has written a script to download the iso before, I also updated it as the two
stage download script,
That uses the HTML parser libxml2 and find all the uris ends with 'iso', then
download them, That is a possible
solution, but the code is complicated.
We can also mirror all the files at dl.fp.o, but we have to write script to clarify the
categories, such as
DVD, netinst, Live, deltaiso, etc.
Hongqing
> Have you considered trying to get an account on one of the machines
> where the
> files already exist (such as alt.fedoraproject.org) and doing the
> tests on that
> machine directly, without downloading? I have an account on
> alt.fp.o
> (for making
> delta ISOs) and can do the size and checksum tests. The repoclosure
> and file
> conflicts tests require loop-mounting the ISO, which can be done
> with
> fuse/fuseiso without root privileges (as I've verified on my home
> machine), even
> though the test instructions currently say to do it as root with
> the
> mount
> command. Unfortunately, the fusermount executable in the fuse
> package
> currently
> requires SUID, so it may not be possible for security reasons, but
> you could
> check if there's a workaround. (There ought to be one - the tests
> simply examine
> the contents of the ISO, and there's no good reason root privileges
> should be
> needed at all.)
Well, this is an interesting proposal. The test client could SSH into
the machine, execute the commands and receive the output. This of
course brings the question of authentication (will we copy the SSH
private key to every test client?) and security (infrastructure guys
wouldn't be probably enthusiastic about this). I would rather set up
our machine, that would mirror everything ISO-related and then
execute the tests there. That is also doable today I guess. Just the
mirroring process is not ideal with http-only solution.
Thanks, Andre, for interesting ideas!
_______________________________________________
autoqa-devel mailing list
autoqa-devel(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/autoqa-devel
8:24 a.m.
On Mon, 2011-09-19 at 22:04 -0400, Hongqing Yang wrote:
----- Original Message -----
> From: "Kamil Paral" <kparal(a)redhat.com>
> To: "AutoQA development" <autoqa-devel(a)lists.fedorahosted.org>
> Sent: Monday, September 19, 2011 9:58:11 PM
> Subject: Re: iso watcher, post-iso-build event, mediakit_sanity test patch for
review
>
> > Install TCs/RCs should always be in
> > http://dl.fedoraproject.org/pub/alt/stage/*/Fedora/, and lives in
> > http://dl.fedoraproject.org/pub/alt/stage/*/Live/. Would it be
> > enough
> > to just
> > ignore everything else?
>
> It is doable, but submerging into individual directories is not
> optimal, because it is accessible only through HTTP. It's not really
> a one-liner. And more code -> more failures. But definitely a way to
> go in the worst case.
>
The lftp also has a --include function, I have tried it, it does not work, I will
continue investigating it.
And James has written a script to download the iso before, I also updated it as the two
stage download script,
That uses the HTML parser libxml2 and find all the uris ends with 'iso', then
download them, That is a possible
solution, but the code is complicated.
We can also mirror all the files at dl.fp.o, but we have to write script to clarify the
categories, such as
DVD, netinst, Live, deltaiso, etc.
At the time, and still today, we don't have a standard format for ISO
drops. We know the server, and the root directory where things will
land ... but the directory structure and names change each
drop/milestone/release. We can work to define some consistency there
with Fedora release engineering. However, the quick'n'dirty
proof-of-concept approach I took was to talk the directory tree (with
safe guards) to locate applicable content for download.
As Hongqing notes, it does add some complexity over a simple `curl`
command. But still seems sustainable and the quickest option IMO.
Thanks,
James
9:30 p.m.
New subject: iso watcher, post-iso-build event, mediakit_sanity test patch for review
----- Original Message -----
From: "Kamil Paral" <kparal(a)redhat.com>
To: "AutoQA development" <autoqa-devel(a)lists.fedorahosted.org>
Sent: Tuesday, September 13, 2011 10:40:45 PM
Subject: Re: iso watcher, post-iso-build event, mediakit_sanity test patch for review
> Hi, all,
>
> Included are package for iso watcher, which use lftp to mirror
> dl.fp.org/pub/alt/stage and kick off mediakit_sanity test,
> package for post-iso-build event, and package for mediakit_sanity
> test
> which test the iso's size, md5, sha256sum, repoclosure
> and file conflicts.
> During the first run of /iso/watcher.py, it will mirror all the
> isos
> at dl.fp.org/pub/alt/stage and kick off mekiakit_sanity test,
> after that, it will only mirror the new isos, the tree compose and
> deltaiso are not included in the mirror.
Hello, I haven't examined the code closely, but I have some overall
architecture thoughts:
1. We need to speak to RelEng team about
http://dl.fedoraproject.org/pub/alt/stage/
We can definitely monitor that URL in the worst case, but it would
be so much better if RelEng team could ensure the following for
us:
a) Consistent file naming
I have a feeling that currently all the directories there are
manually created (or at least manually named). If there were some
systematic approach that we can rely on, it would help us greatly
(e.g. in determining the latest release).
b) No irrelevant files there
Currently there are not just TC/RC composes, but also "deltaisos/"
and "f16-translation/" directories that we need to blacklist. I
am afraid "stage/" serves as a storage directory for anything
needed and those "clutter" directories will change over time. It
would be great to know that no other unwanted files appear there
anytime.
c) Rsync possibility
Using rsync would simplify the mirroring process at least for our
Red Hat offices. I bet we could also use it in AutoQA if desired.
2. Mirroring everything seems overkill. Personally I don't have
Internet bandwidth nor hard disk space to run this test. Can we
modify the watcher so that it detects just the last compose and
downloads that one?
3. Only now I realized that watcher downloading files doesn't help
us. Because the files will be accessible on the AutoQA server, but
we need them on AutoQA clients. That means that your code works as
long as you run it with "--local", but won't work with external
clients. It also means we have to work on ticket #350 "Add support
for using file proxy" (I'll try to do that soon). Ideally we will
set up the proxy on AutoQA server and all clients will be using it.
The first test will download the ISO and the other tests (if there
are some) will just use the cached version (assuming there is a fast
network between the server and the client, otherwise it is of no
use).
And an extra code-related question at the end:
4. Do we need labels = ['virt_capable'] in control.autoqa? This test
doesn't seem to be requiring any form of virtualization
capabilities.
To sum it up, I believe we will need to do some changes. But
definitely thanks for your work!
Kamil
_______________________________________________
autoqa-devel mailing list
autoqa-devel(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/autoqa-devel
Hi all,
I have updated all except the minimum size of the isos, I have not thought out a
good idea to do that. I think we need the minimum size, but it is difficult to say
what is the reasonable minimum size. And there is no response from rea-eng team
after I create a ticket for the name rules at dl.fp.o/alt/stage.
Hongqing
9:12 a.m.
New subject: iso watcher, post-iso-build event, mediakit_sanity test patch for review
Hongqing Yang <hoyang <at> redhat.com> writes:
Hi all,
I have updated all except the minimum size of the isos, I have not thought
out a good idea to do that. I think we need the minimum size, but it is
difficult to say what is the reasonable minimum size.
I don't see why it's necessary to have a minimum size test. The only problem
with an image that is too small would be that it would be guaranteed to fail
other tests, and having a minimum size test allows a _human_ to catch the
problem more quickly, but if some of those other tests are going to be automated
as well, the problem should be caught quickly regardless. Also, a small image is
much faster to download and run tests on. An image that was < 1 MiB, say,
probably wouldn't even have a boot menu, and certainly couldn't pass a test
such as https://fedoraproject.org/wiki/QA:Testcase_Boot_Methods_Boot_Iso .
9:52 p.m.
New subject: iso watcher, post-iso-build event, mediakit_sanity test patch for review
----- Original Message -----
From: "Andre Robatino" <robatino(a)fedoraproject.org>
To: autoqa-devel(a)lists.fedorahosted.org
Sent: Sunday, October 16, 2011 10:12:32 PM
Subject: iso watcher, post-iso-build event, mediakit_sanity test patch for review
Hongqing Yang <hoyang <at> redhat.com> writes:
> Hi all,
> I have updated all except the minimum size of the isos, I have not
> thought
> out a good idea to do that. I think we need the minimum size, but
> it is
> difficult to say what is the reasonable minimum size.
I don't see why it's necessary to have a minimum size test. The only
problem
with an image that is too small would be that it would be guaranteed
to fail
other tests, and having a minimum size test allows a _human_ to catch
the
problem more quickly, but if some of those other tests are going to
be automated
as well, the problem should be caught quickly regardless. Also, a
small image is
much faster to download and run tests on. An image that was < 1 MiB,
say,
probably wouldn't even have a boot menu, and certainly couldn't pass
a test
such as
https://fedoraproject.org/wiki/QA:Testcase_Boot_Methods_Boot_Iso .
That is reasonable, I just need check their exist.
Thanks,
Hongqing
4568
days inactive
4606
days old
autoqa-devel@lists.fedorahosted.org
11 comments
5 participants
participants (5)
-
Andre Robatino -
Hongqing Yang -
James Laska -
Josef Skladanka -
Kamil Paral