autoqa cannot work with i386 architecture

[Hongqing Yang]

Dear All,

Now I am developing the URL and DVD automatic installation test, I try to integrate them into AutoQA.
but the architecture cannot be parsed correctly.

in the master branch, we can try it as below:

autoqa post-tree-compose http://download.englab.nay.redhat.com/pub/fedora/linux/releases/14/Fedora/i3... --arch=i386 --test=rats_install --local

it always parsed the architecture as x86_64
I find the following codes are in the parse_params function of autoqa:

# Run the tests locally, or schedule them through autotest?
    run_local = (opts.local or getbool(conf['local']))

if not opts.arch or run_local:
        opts.arch = ['noarch']

can you help me make clear why we set arch as 'noarch' when we run it locally, thanks.

Hongqing

[Kamil Paral]

> Dear All,
>
> Now I am developing the URL and DVD automatic installation test, I try
> to integrate them into AutoQA.
> but the architecture cannot be parsed correctly.
>
> in the master branch, we can try it as below:
>
> autoqa post-tree-compose
> http://download.englab.nay.redhat.com/pub/fedora/linux/releases/14/Fedora/i3...
> --arch=i386 --test=rats_install --local
>
> it always parsed the architecture as x86_64
> I find the following codes are in the parse_params function of autoqa:
>
> # Run the tests locally, or schedule them through autotest?
> run_local = (opts.local or getbool(conf['local']))
>
> if not opts.arch or run_local:
> opts.arch = ['noarch']
>
> can you help me make clear why we set arch as 'noarch' when we run it
> locally, thanks.
>
> Hongqing

Hello Hongqing,

nice to see you around. You have found one of our weak spots. The logic is mixed up a little when handling architectures, because we use --arch to express two things at once:
1. the machine architecture this test should run on
2. the architecture this test should check

It is usually the same, but not always (we may want to run 32bit code on 64bit machine, we may want to test 32bit network repo on whatever-arch machine, etc), we just hadn't hit the issue before.

I think we should:
* split the single --arch option into 'test arch' and 'machine arch' options
* don't override the machine arch when running locally, but check that it matches
* ensure the test arch option is used inside the tests

Concerning your current use case (rats_install), I have looked into rats_install.py and the architecture doesn't seem to be used for test execution:
>     cmd = "./install.py -s %s -l %s" % (self.tmpdir, self.resultsdir)
>     if image_url != "":
>         cmd += " -i %s" % image_url
>     if boot_args != "":
>         cmd += " -x %s" % boot_args
>     cmd += " %s" % baseurl
>     self.result = None
>     try:
>         out = utils.system_output(cmd + " 2>&1", retain_output=True)

Why do you need AutoQA to set it correctly?
> it always parsed the architecture as x86_64

Can you provide sample output? Are you talking about output from rats_install/install.py or some other?

[Hongqing Yang]

Hi, Kamil,

Thanks for your reply. Actually the current rats_install should also be updated. in the following code
it cannot get the parameter --arch

cmd = "./install.py -s %s -l %s" % (self.tmpdir, self.resultsdir)
         if image_url != "":
             cmd += " -i %s" % image_url
         if boot_args != "":
             cmd += " -x %s" % boot_args
         cmd += " %s" % baseurl
         self.result = None
         try:
             out = utils.system_output(cmd + " 2>&1", retain_output=True)

I will add the following
        cmd += " --arch=%s" % kwargs['arch']

since the install.py also have the option --arch, the default is x86_64.

One thing I am still not very clear, do we have both x86_64 and i386 servers to run 
x86_64 and i386 Fedora respectively?  I think we will use x86_64 to run both x86_64 and
i386 Fedora during the development. So it would be better to split the server
and the Fedora tested architectures, the server architecture can be detected by default,
the architecture of Fedora tested will be passed as a parameter. What do you think?

Thanks,
Hongqing

----- Original Message -----
From: "Kamil Paral" 
To: "AutoQA development" 
Sent: Monday, May 30, 2011 3:28:19 PM
Subject: Re: autoqa cannot work with i386 architecture
> Dear All,
>
> Now I am developing the URL and DVD automatic installation test, I try
> to integrate them into AutoQA.
> but the architecture cannot be parsed correctly.
>
> in the master branch, we can try it as below:
>
> autoqa post-tree-compose
> http://download.englab.nay.redhat.com/pub/fedora/linux/releases/14/Fedora/i3...
> --arch=i386 --test=rats_install --local
>
> it always parsed the architecture as x86_64
> I find the following codes are in the parse_params function of autoqa:
>
> # Run the tests locally, or schedule them through autotest?
> run_local = (opts.local or getbool(conf['local']))
>
> if not opts.arch or run_local:
> opts.arch = ['noarch']
>
> can you help me make clear why we set arch as 'noarch' when we run it
> locally, thanks.
>
> Hongqing

Hello Hongqing,

nice to see you around. You have found one of our weak spots. The logic is mixed up a little when handling architectures, because we use --arch to express two things at once:
1. the machine architecture this test should run on
2. the architecture this test should check

It is usually the same, but not always (we may want to run 32bit code on 64bit machine, we may want to test 32bit network repo on whatever-arch machine, etc), we just hadn't hit the issue before.

I think we should:
* split the single --arch option into 'test arch' and 'machine arch' options
* don't override the machine arch when running locally, but check that it matches
* ensure the test arch option is used inside the tests

Concerning your current use case (rats_install), I have looked into rats_install.py and the architecture doesn't seem to be used for test execution:
>     cmd = "./install.py -s %s -l %s" % (self.tmpdir, self.resultsdir)
>     if image_url != "":
>         cmd += " -i %s" % image_url
>     if boot_args != "":
>         cmd += " -x %s" % boot_args
>     cmd += " %s" % baseurl
>     self.result = None
>     try:
>         out = utils.system_output(cmd + " 2>&1", retain_output=True)

Why do you need AutoQA to set it correctly?
> it always parsed the architecture as x86_64

Can you provide sample output? Are you talking about output from rats_install/install.py or some other?
_______________________________________________
autoqa-devel mailing list
[e-mail]
https://fedorahosted.org/mailman/listinfo/autoqa-devel

[Kamil Paral]

----- Original Message -----
> I will add the following
> cmd += " --arch=%s" % kwargs['arch']

OK, then it makes sense.
> since the install.py also have the option --arch, the default is
> x86_64.

The default seems to be the machine (operating system) architecture. If you run it on x86_64, it should return x86_64. If you run it on i386, it should return i386.
> One thing I am still not very clear, do we have both x86_64 and i386
> servers to run
> x86_64 and i386 Fedora respectively?

You mean "machines/clients" instead of "servers", right? We have one autotest server (autoqa.fedoraproject.org), but many test clients with different architectures.
> I think we will use x86_64 to run
> both x86_64 and
> i386 Fedora during the development. So it would be better to split the
> server
> and the Fedora tested architectures, the server architecture can be
> detected by default,
> the architecture of Fedora tested will be passed as a parameter. What
> do you think?

Exactly, that's also my view.

But it will take some time to implement it. In the meantime, you can do one of these workarounds:
1. don't run the test locally, but schedule it through autotest server (provided that you have more physical machines available, since rats_install requires a physical machine)
2. don't use kwargs['arch'] temporarily, but instead parse the compose URL (e.g. [1]) and take the test architecture from that.

[1] http://download.englab.nay.redhat.com/pub/fedora/linux/releases/14/Fedora/i3...

Does it help?

[Hongqing Yang]

----- Original Message -----
From: "Kamil Paral" 
To: "AutoQA development" 
Sent: Monday, May 30, 2011 4:30:01 PM
Subject: Re: autoqa cannot work with i386 architecture

----- Original Message -----
> I will add the following
> cmd += " --arch=%s" % kwargs['arch']

OK, then it makes sense.
> since the install.py also have the option --arch, the default is
> x86_64.

The default seems to be the machine (operating system) architecture. If you run it on x86_64, it should return x86_64. If you run it on i386, it should return i386.
> One thing I am still not very clear, do we have both x86_64 and i386
> servers to run
> x86_64 and i386 Fedora respectively?

You mean "machines/clients" instead of "servers", right? We have one autotest server (autoqa.fedoraproject.org), but many test clients with different architectures.
> I think we will use x86_64 to run
> both x86_64 and
> i386 Fedora during the development. So it would be better to split the
> server
> and the Fedora tested architectures, the server architecture can be
> detected by default,
> the architecture of Fedora tested will be passed as a parameter. What
> do you think?

Exactly, that's also my view.

But it will take some time to implement it. In the meantime, you can do one of these workarounds:
1. don't run the test locally, but schedule it through autotest server (provided that you have more physical machines available, since rats_install requires a physical machine)
2. don't use kwargs['arch'] temporarily, but instead parse the compose URL (e.g. [1]) and take the test architecture from that.

[1] http://download.englab.nay.redhat.com/pub/fedora/linux/releases/14/Fedora/i3...

Does it help?

That is fine. I will update it in my branch as the workaround. Thanks.

Hongqing
_______________________________________________
autoqa-devel mailing list
[e-mail]
https://fedorahosted.org/mailman/listinfo/autoqa-devel

[Vitezslav Humpa]

> One thing I am still not very clear, do we have both x86_64 and i386
> servers to run
> x86_64 and i386 Fedora respectively? I think we will use x86_64 to run
> both x86_64 and
> i386 Fedora during the development. So it would be better to split the
> server
> and the Fedora tested architectures, the server architecture can be
> detected by default,
> the architecture of Fedora tested will be passed as a parameter. What
> do you think?

Hi Hongqing,

until recently the arch given as a parameter to 'autoqa' was used just to
schedule the test with autotest (and it always gets scheduled as x86_64 now
when set to run locally) and the tests themselves basically used uname
based method to determine the client architecture when needed. In[1]
we've added the arch in kwargs, so over there you should always get the
architecture actually passed to 'autoqa', if that is any help.

To the testing efforts, you can temporarily modify 'autoqa' to use i386
(just change it on line 53, I believe) when noarch instead of x86_64
- or don't test it as local and let it get scheduled by autotest as in
production instead.

[1] https://fedorahosted.org/autoqa/changeset/d65c70786a76016504beeb35841c19b029...

--
Vita Humpa

[Hongqing Yang]

----- Original Message -----
From: "Vitezslav Humpa" 
To: "AutoQA development" 
Sent: Monday, May 30, 2011 4:42:59 PM
Subject: Re: autoqa cannot work with i386 architecture
> One thing I am still not very clear, do we have both x86_64 and i386
> servers to run
> x86_64 and i386 Fedora respectively? I think we will use x86_64 to run
> both x86_64 and
> i386 Fedora during the development. So it would be better to split the
> server
> and the Fedora tested architectures, the server architecture can be
> detected by default,
> the architecture of Fedora tested will be passed as a parameter. What
> do you think?

Hi Hongqing,

until recently the arch given as a parameter to 'autoqa' was used just to
schedule the test with autotest (and it always gets scheduled as x86_64 now
when set to run locally) and the tests themselves basically used uname
based method to determine the client architecture when needed. In[1]
we've added the arch in kwargs, so over there you should always get the
architecture actually passed to 'autoqa', if that is any help.

To the testing efforts, you can temporarily modify 'autoqa' to use i386
(just change it on line 53, I believe) when noarch instead of x86_64
- or don't test it as local and let it get scheduled by autotest as in
production instead.

[1] https://fedorahosted.org/autoqa/changeset/d65c70786a76016504beeb35841c19b029...

--
Vita Humpa

Hi, Vita,

Now I am confused. I know we use the 'arch' here to represent the architecture of the machine (test server or client).
The architecture I am talking about here is the Fedora system that we will test. Since I would like
autoqa can cover the test case: 
http://fedoraproject.org/wiki/QA:Installer_image_presence_test_case
I would like to check the arch listed in .treeinfo is correct or not, so we have to tell autoqa which
arch we are testing, then we can compare the real arch with that in .treeinfo.
If the arch only represent the machine arch, then the option 'arch' in parser is not useful, since the
arch can be detected by default. Am I right?

Thanks.

_______________________________________________
autoqa-devel mailing list
[e-mail]
https://fedorahosted.org/mailman/listinfo/autoqa-devel

[Vitezslav Humpa]

> Hi, Vita,
>
> Now I am confused. I know we use the 'arch' here to represent the
> architecture of the machine (test server or client).
> The architecture I am talking about here is the Fedora system that we
> will test. Since I would like
> autoqa can cover the test case:
> http://fedoraproject.org/wiki/QA:Installer_image_presence_test_case
> I would like to check the arch listed in .treeinfo is correct or not,
> so we have to tell autoqa which
> arch we are testing, then we can compare the real arch with that in
> .treeinfo.
> If the arch only represent the machine arch, then the option 'arch' in
> parser is not useful, since the
> arch can be detected by default. Am I right?
>
> Thanks.

Not completely un-useful, since the arch that we detected in the test was 
actually the arch of the system running, not necessarily of the actual HW.
So if you were running 32bit system on 64bit, you'd get 32bit there.
Actually, although that is probably exactly the behaviour you want now,
the point of the 'arch' parameter is to pass the actual HW architecture the
test was scheduled to run upon.

I am not sure when it could be needed to know the HW arch really instead
of system arch (esp. should we use 64bit HW to test 32bit Fedora) - so perhaps
it is of little significance right now.

In any way, just using util.get_basearch() should get you the arch you want
(of the system), you are right.

Thanks
Vita

[Kamil Paral]

> Hi, Vita,
>
> Now I am confused.

My previous answer is still valid. Vita wrote about one implementation detail, but I believe it is not fully relevant to your use case.
> I know we use the 'arch' here to represent the
> architecture of the machine (test server or client).

Yes, the 'arch' parameter specifies the machine architecture we asked for.
> The architecture I am talking about here is the Fedora system that we
> will test. Since I would like
> autoqa can cover the test case:
> http://fedoraproject.org/wiki/QA:Installer_image_presence_test_case

Yes, you need to specify 'test architecture'. I.e. "I want to check 32bit compose, and I don't care what type of machine I am assigned".

That must be fixed in the future, currently you can use one of those workarounds I mentioned.
> If the arch only represent the machine arch, then the option 'arch' in
> parser is not useful, since the
> arch can be detected by default. Am I right?

Good thinking, but no. Even though the test runs on i386 or x86_64, the 'arch' parameter may contain 'noarch' keyword to indicate this is a noarch test we're running. It is important when reporting results.

But this is another detail I believe you don't need to care about for your current use case.
> Thanks.

PS: Please make sure you quote the emails correctly (by using '>' marks). It hard to read them otherwise. Thanks.

[Hongqing Yang]

----- Original Message -----
From: "Kamil Paral" 
To: "AutoQA development" 
Sent: Monday, May 30, 2011 8:04:20 PM
Subject: Re: autoqa cannot work with i386 architecture
> Hi, Vita,
>
> Now I am confused.

My previous answer is still valid. Vita wrote about one implementation detail, but I believe it is not fully relevant to your use case.
> I know we use the 'arch' here to represent the
> architecture of the machine (test server or client).

Yes, the 'arch' parameter specifies the machine architecture we asked for.
> The architecture I am talking about here is the Fedora system that we
> will test. Since I would like
> autoqa can cover the test case:
> http://fedoraproject.org/wiki/QA:Installer_image_presence_test_case

Yes, you need to specify 'test architecture'. I.e. "I want to check 32bit compose, and I don't care what type of machine I am assigned".

That must be fixed in the future, currently you can use one of those workarounds I mentioned.
> If the arch only represent the machine arch, then the option 'arch' in
> parser is not useful, since the
> arch can be detected by default. Am I right?

Good thinking, but no. Even though the test runs on i386 or x86_64, the 'arch' parameter may contain 'noarch' keyword to indicate this is a noarch test we're running. It is important when reporting results.

But this is another detail I believe you don't need to care about for your current use case.
> Thanks.

PS: Please make sure you quote the emails correctly (by using '>' marks). It hard to read them otherwise. Thanks.

Kamil and Vita,

Thanks for your reply, now it is clear that we have three architectures, two environment architectures(hardware and OS) and
one Fedora product architecture.  I will use the workaround you mentioned previously, parsing the tree url  or the ISO name 
to detect the architecture for official tree compose and ISO images, or I will just comment the 'or local:' condition in autoqa
for my local test.  Maybe we can improve this later. Thanks again.

Hongqing

_______________________________________________
autoqa-devel mailing list
[e-mail]
https://fedorahosted.org/mailman/listinfo/autoqa-devel

[Hongqing Yang]

----- Original Message -----
> From: "Hongqing Yang" 
> To: "AutoQA development" 
> Sent: Tuesday, May 31, 2011 1:27:31 PM
> Subject: Re: autoqa cannot work with i386 architecture
> ----- Original Message -----
> From: "Kamil Paral" 
> To: "AutoQA development" 
> Sent: Monday, May 30, 2011 8:04:20 PM
> Subject: Re: autoqa cannot work with i386 architecture
> > Hi, Vita,
> >
> > Now I am confused.
>
> My previous answer is still valid. Vita wrote about one implementation
> detail, but I believe it is not fully relevant to your use case.
> > I know we use the 'arch' here to represent the
> > architecture of the machine (test server or client).
>
> Yes, the 'arch' parameter specifies the machine architecture we asked
> for.
> > The architecture I am talking about here is the Fedora system that
> > we
> > will test. Since I would like
> > autoqa can cover the test case:
> > http://fedoraproject.org/wiki/QA:Installer_image_presence_test_case
>
> Yes, you need to specify 'test architecture'. I.e. "I want to check
> 32bit compose, and I don't care what type of machine I am assigned".
>
> That must be fixed in the future, currently you can use one of those
> workarounds I mentioned.
> > If the arch only represent the machine arch, then the option 'arch'
> > in
> > parser is not useful, since the
> > arch can be detected by default. Am I right?
>
> Good thinking, but no. Even though the test runs on i386 or x86_64,
> the 'arch' parameter may contain 'noarch' keyword to indicate this is
> a noarch test we're running. It is important when reporting results.
>
> But this is another detail I believe you don't need to care about for
> your current use case.
> > Thanks.
>
> PS: Please make sure you quote the emails correctly (by using '>'
> marks). It hard to read them otherwise. Thanks.
>
> Kamil and Vita,
>
> Thanks for your reply, now it is clear that we have three
> architectures, two environment architectures(hardware and OS) and
> one Fedora product architecture. I will use the workaround you
> mentioned previously, parsing the tree url or the ISO name
> to detect the architecture for official tree compose and ISO images,
> or I will just comment the 'or local:' condition in autoqa
> for my local test. Maybe we can improve this later. Thanks again.
>
> Hongqing

Sorry, the prefix is not added again. It is the problem of my email setting.