Fedora Audio Test Cases for Test Day (a proposal)

[Lukas Ruzicka]

Hello friends of Fedora,

the following is a proposal for the test scope of PipeWire audio in Fedora,
which could be tested as part of Fedora Audio Test Days. Please let me know
what you think about it.
Particular test cases have not been developed yet, but as soon as I have
the idea what is actually desired for testing, I will develop them and
prepare them as a material for test days.

Thanks for cooperation,
Lukas


====
Test Cases for Test Days (proposal) Description

The following is a proposal for test cases that could be tested as a part
of a Fedora Audio Test Day.
Possible test cases

   1.

   *PipeWire is used by default.* This test case should test that audio is
   controlled by PipeWire by default.
   2.

   *There are GUI and CLI tools to control sound available to the user*.
   This test case tests that sound production can be controlled using
   different tools.
   3.

   *Sound can be played back from all applications* – this test case tests
   that any audio application installable from Fedora repositories with a
   proper setup must produce audible sounds of expected quality using the
   selected hardware. This will cover the majority of JACK or ALSA based
   applications.
   4.

   *Sound can be recorded in all applications* – this test case tests that
   any audio application installable from Fedora repositories is able to
   record sound from the external sources in expected quality. This will also
   affect default applications, such as Firefox, that is used for conferencing
   and the ability to receive sound is crucial for this use case.
   5.

   *Dedicated hardware can be used to produce or record audio* – any
   dedicated audio hardware, especially soundcards, must play audible sounds
   of expected quality, as well as record them, if they have been successfully
   recognized by the operating system.
   6.

   *MIDI capable hardware communicates with the MIDI capable applications*
   – if the MIDI capable hardware is properly recognized by the system, it
   must be able to communicate with an application. However, it should not
   ever be required that the application understands that communication fully
   and without issues.


-- 

Lukáš Růžička

FEDORA QE, RHCE

Red Hat

<https://www.redhat.com>

Purkyňova 115

612 45 Brno - Královo Pole

lruzicka(a)redhat.com
TRIED AND PERSONALLY TESTED, ERGO TRUSTED. <https://redhat.com/trusted>

[Kamil Paral]

On Tue, Jan 19, 2021 at 2:26 PM Lukas Ruzicka <lruzicka(a)redhat.com&gt; wrote:

>  Hello friends of Fedora,
>
>  the following is a proposal for the test scope of PipeWire audio in
>  Fedora, which could be tested as part of Fedora Audio Test Days. Please let
>  me know what you think about it.
>  Particular test cases have not been developed yet, but as soon as I have
>  the idea what is actually desired for testing, I will develop them and
>  prepare them as a material for test days.
>
>  Thanks for cooperation,
>  Lukas
>
>
>  ====
>  Test Cases for Test Days (proposal) Description
>
>  The following is a proposal for test cases that could be tested as a part
>  of a Fedora Audio Test Day.
>  Possible test cases
>
>     1.
>
>     *PipeWire is used by default.* This test case should test that audio
>     is controlled by PipeWire by default.
>     2.
>
>     *There are GUI and CLI tools to control sound available to the user*.
>     This test case tests that sound production can be controlled using
>     different tools.
>     3.
>
>     *Sound can be played back from all applications* – this test case
>     tests that any audio application installable from Fedora repositories with
>     a proper setup must produce audible sounds of expected quality using the
>     selected hardware. This will cover the majority of JACK or ALSA based
>     applications.
>     4.
>
>     *Sound can be recorded in all applications* – this test case tests
>     that any audio application installable from Fedora repositories is able to
>     record sound from the external sources in expected quality. This will also
>     affect default applications, such as Firefox, that is used for conferencing
>     and the ability to receive sound is crucial for this use case.
>     5.
>
>     *Dedicated hardware can be used to produce or record audio* – any
>     dedicated audio hardware, especially soundcards, must play audible sounds
>     of expected quality, as well as record them, if they have been successfully
>     recognized by the operating system.
>     6.
>
>     *MIDI capable hardware communicates with the MIDI capable applications*
>     – if the MIDI capable hardware is properly recognized by the system, it
>     must be able to communicate with an application. However, it should not
>     ever be required that the application understands that communication fully
>     and without issues.
>
>
>  
I'm not a sound expert, but the list looks reasonable.

[Brandon Nielsen]

On 1/20/21 6:56 AM, Kamil Paral wrote:
>  On Tue, Jan 19, 2021 at 2:26 PM Lukas Ruzicka <lruzicka(a)redhat.com
>
>  <mailto:[e-mail]>> wrote:
>
>      Hello friends of Fedora,
>
>      the following is a proposal for the test scope of PipeWire audio
>      in Fedora, which could be tested as part of Fedora Audio Test
>      Days. Please let me know what you think about it.
>      Particular test cases have not been developed yet, but as soon as
>      I have the idea what is actually desired for testing, I will
>      develop them and prepare them as a material for test days.
>
>      Thanks for cooperation,
>      Lukas
>
>
>      ====
>
>
>        Test Cases for Test Days (proposal)
>
>
>          Description
>
>      The following is a proposal for test cases that could be tested as
>      a part of a Fedora Audio Test Day.
>
>
>          Possible test cases
>
>      1.
>
>          *PipeWire is used by default.* This test case should test that
>          audio is controlled by PipeWire by default.
>
>      2.
>
>          *There are GUI and CLI tools to control sound available to the
>          user*. This test case tests that sound production can be
>          controlled using different tools.
>
>      3.
>
>          *Sound can be played back from all applications* – this test
>          case tests that any audio application installable from Fedora
>          repositories with a proper setup must produce audible sounds
>          of expected quality using the selected hardware. This will
>          cover the majority of JACK or ALSA based applications.
>
>      4.
>
>          *Sound can be recorded in all applications* – this test case
>          tests that any audio application installable from Fedora
>          repositories is able to record sound from the external sources
>          in expected quality. This will also affect default
>          applications, such as Firefox, that is used for conferencing
>          and the ability to receive sound is crucial for this use case.
>
>      5.
>
>          *Dedicated hardware can be used to produce or record audio* –
>          any dedicated audio hardware, especially soundcards, must play
>          audible sounds of expected quality, as well as record them, if
>          they have been successfully recognized by the operating system.
>
>      6.
>
>          *MIDI capable hardware communicates with the MIDI capable
>          applications* – if the MIDI capable hardware is properly
>          recognized by the system, it must be able to communicate with
>          an application. However, it should not ever be required that
>          the application understands that communication fully and
>          without issues.
>
>
>
>  I'm not a sound expert, but the list looks reasonable.
>
>  
Would some kind of device switching test case be feasible? In my 
experience that's where a lot of weird behavior creeps in. For example, 
plugging in a USB interface, using it, and unplugging it. Or switching 
back and forth between Bluetooth and wired hardware (or even changing 
Bluetooth headset mode back and forth between A2DP / HFP).

[Kamil Paral]

On Wed, Jan 20, 2021 at 5:02 PM Brandon Nielsen <nielsenb(a)jetfuse.net&gt;
wrote:

>  Would some kind of device switching test case be feasible? In my
>  experience that's where a lot of weird behavior creeps in. For example,
>  plugging in a USB interface, using it, and unplugging it. Or switching back
>  and forth between Bluetooth and wired hardware (or even changing Bluetooth
>  headset mode back and forth between A2DP / HFP).
>  
This is a great idea, I love it! Most of us now have at least two output
devices - the usual jack output and a display connected over DP/HDMI. Not
all monitors have integrated speakers, but some have a jack connector you
can connect your headphones in. And that's exactly how I use it - speakers
connected to my desktop's output jack, and headphones connected to my
monitor, which allows me to switch between speakers and headphones in a
pure software way, without replugging connectors all the time. Sometimes it
causes some issues. This would be a great test case with Pipewire.

And as suggested by Brandon, other devices like USB headphones or Bluetooth
headphones are probably even more common than my own use case. Let's cover
this.

[Lukas Ruzicka]

On Wed, Jan 20, 2021 at 5:21 PM Kamil Paral <kparal(a)redhat.com&gt; wrote:

>  On Wed, Jan 20, 2021 at 5:02 PM Brandon Nielsen
> <nielsenb(a)jetfuse.net&gt;
>  wrote:
>
> > Would some kind of device switching test case be feasible? In my
> > experience that's where a lot of weird behavior creeps in. For example,
> > plugging in a USB interface, using it, and unplugging it. Or switching back
> > and forth between Bluetooth and wired hardware (or even changing Bluetooth
> > headset mode back and forth between A2DP / HFP).
> >
>
>  This is a great idea, I love it! Most of us now have at least two output
>  devices - the usual jack output and a display connected over DP/HDMI. Not
>  all monitors have integrated speakers, but some have a jack connector you
>  can connect your headphones in. And that's exactly how I use it - speakers
>  connected to my desktop's output jack, and headphones connected to my
>  monitor, which allows me to switch between speakers and headphones in a
>  pure software way, without replugging connectors all the time. Sometimes it
>  causes some issues. This would be a great test case with Pipewire.
>
>  And as suggested by Brandon, other devices like USB headphones or
>  Bluetooth headphones are probably even more common than my own use case.
>  Let's cover this.
>  
Ack




>  _______________________________________________
>  test mailing list -- test(a)lists.fedoraproject.org
>  To unsubscribe send an email to test-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  

-- 

Lukáš Růžička

FEDORA QE, RHCE

Red Hat

<https://www.redhat.com>

Purkyňova 115

612 45 Brno - Královo Pole

lruzicka(a)redhat.com
TRIED AND PERSONALLY TESTED, ERGO TRUSTED. <https://redhat.com/trusted>

[Garry T. Williams]

On Wednesday, January 20, 2021 11:01:59 AM EST Brandon Nielsen wrote:
>  Would some kind of device switching test case be feasible? In my 
>  experience that's where a lot of weird behavior creeps in. For example, 
>  plugging in a USB interface, using it, and unplugging it. 
https://bugzilla.redhat.com/show_bug.cgi?id=1912150

-- 
Garry T. Williams