Default invocation of pytest for qadevel projects

[Tim Flink]

On Thu, 6 Mar 2014 05:44:15 -0500 (EST)
Kamil Paral <kparal at redhat.com> wrote:

> > For the qadevel projects I'm aware of, we've been using pytest for
> > our unit/functional test runner.
> > 
> > As part of the shared configuration, tests are split up into two
> > categories, unit and functional. Unit tests are fast, do not touch
> > the network, database or filesystem (there are some exceptions to
> > that last part, though). Functional tests tend to be more
> > integration tests that can set up a database or do other actions
> > which fall outside of the previous definition of "unit test".
> > 
> > When you run pytest without any extra args, only the unit tests are
> > run. The '--functional' arg is needed to enable collection and
> > execution of the functional tests.
> > 
> > Kamil recently made a request [1] to do one of two things:
> > 
> > [1] https://phab.qadevel.cloud.fedoraproject.org/T89
> > 
> > 1. Change the default such that functional tests are collected and
> > exclude them from collection using a '--unit' arg
> > 
> > 2. Change the functional arg from '--functional' to something
> > shorter, like '--func'
> > 
> >   * note that there are restrictions on which args we can use. For
> >     example, '-f' is not allowed as single letter args are reserved
> > for pytest itself
> > 
> > As stated in T89, I don't have a strong opinion on this as long as
> > it's possible to exclude the functional tests from collection and
> > we make the same change across all of our active projects. However,
> > I wanted to put this up for a wider discussion before changing
> > things.
> > 
> > Any other thoughts on this proposed change?
> 
> I should remember to read the mailing list before the Phab comments.
> I already closed the task as wontfix with my comment included:
> https://phab.qadevel.cloud.fedoraproject.org/T89#9
> 
> I agree that you have a point in running just the unit tests by
> default.

For my workflow, sure. On the other hand, I'm not sure what other
peoples' workflow looks like and if it reduces confusion, we may be
better off changing the defaults to run functional tests. As I stated
in the ticket, as long as I can turn off the functional tests with a
command line arg, my workflow won't be affected.

Other thoughts on which is a better default?

> Changing --functional to --func would be nice, though. It
> even matches the file prefix for func tests, 'functest_'.

Sure, I'm game for changing that. The two args that come to mind are:

 * --long (more generic than functional)
 * --func (short for functional)

Any thoughts on which of those (if either) would be better?

Tim
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/qa-devel/attachments/20140306/4113540f/attachment.sig>