Default invocation of pytest for qadevel projects

[Martin Krizek]

----- Original Message -----
> From: "Tim Flink" <tflink at redhat.com>
> To: qa-devel at lists.fedoraproject.org
> Sent: Thursday, March 6, 2014 3:29:32 PM
> Subject: Re: Default invocation of pytest for qadevel projects
> 
> On Thu, 6 Mar 2014 05:44:15 -0500 (EST)
> Kamil Paral <kparal at redhat.com> wrote:
> 
> > > For the qadevel projects I'm aware of, we've been using pytest for
> > > our unit/functional test runner.
> > > 
> > > As part of the shared configuration, tests are split up into two
> > > categories, unit and functional. Unit tests are fast, do not touch
> > > the network, database or filesystem (there are some exceptions to
> > > that last part, though). Functional tests tend to be more
> > > integration tests that can set up a database or do other actions
> > > which fall outside of the previous definition of "unit test".
> > > 
> > > When you run pytest without any extra args, only the unit tests are
> > > run. The '--functional' arg is needed to enable collection and
> > > execution of the functional tests.
> > > 
> > > Kamil recently made a request [1] to do one of two things:
> > > 
> > > [1] https://phab.qadevel.cloud.fedoraproject.org/T89
> > > 
> > > 1. Change the default such that functional tests are collected and
> > > exclude them from collection using a '--unit' arg
> > > 
> > > 2. Change the functional arg from '--functional' to something
> > > shorter, like '--func'
> > > 
> > >   * note that there are restrictions on which args we can use. For
> > >     example, '-f' is not allowed as single letter args are reserved
> > > for pytest itself
> > > 
> > > As stated in T89, I don't have a strong opinion on this as long as
> > > it's possible to exclude the functional tests from collection and
> > > we make the same change across all of our active projects. However,
> > > I wanted to put this up for a wider discussion before changing
> > > things.
> > > 
> > > Any other thoughts on this proposed change?
> > 
> > I should remember to read the mailing list before the Phab comments.
> > I already closed the task as wontfix with my comment included:
> > https://phab.qadevel.cloud.fedoraproject.org/T89#9
> > 
> > I agree that you have a point in running just the unit tests by
> > default.
> 
> For my workflow, sure. On the other hand, I'm not sure what other
> peoples' workflow looks like and if it reduces confusion, we may be
> better off changing the defaults to run functional tests. As I stated
> in the ticket, as long as I can turn off the functional tests with a
> command line arg, my workflow won't be affected.
> 
> Other thoughts on which is a better default?
> 

I don't have a strong feeling about either but I'd run all the tests by default 
if I had to choose (given that functional ones don't run "forever", which is 
not the case at this point).

> > Changing --functional to --func would be nice, though. It
> > even matches the file prefix for func tests, 'functest_'.
> 
> Sure, I'm game for changing that. The two args that come to mind are:
> 
>  * --long (more generic than functional)
>  * --func (short for functional)
> 
> Any thoughts on which of those (if either) would be better?
> 

--func sounds better to me.

Martin