STI tests.yml: Custom defined variables

[Miro Hrončok]

Hello,

please bare with me because my Ansible skills are close to none. I only use it 
for Fedora CI.

I've read 
https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.ht...

I have this (simplified) tests.yml:

     ---
     - hosts: localhost
       roles:
       - role: standard-test-basic
         tests:
         - smoke:
             dir: python/smoke
             run: VERSION=3.11 ./venv.sh
         required_packages:
         - python3.11
         - python3.11-devel

Full version in 
https://src.fedoraproject.org/rpms/python3.11/blob/rawhide/f/tests/tests.yml

I would like to replace all occurrences of 3.11 with a variable:


     ---
     - hosts: localhost
       roles:
       - role: standard-test-basic
         tests:
         - smoke:
             dir: python/smoke
             run: VERSION={{ pybasever }} ./venv.sh
         required_packages:
         - python{{ pybasever }}
         - python{{ pybasever }}-devel


But where do I put pybasever: '3.11' exactly?


Bonus question: Can I read the tested package name and regex it into the variable?

     pybasever:  {{ xxx_package_name|pex_regex_replace('^python', '') }}

Thanks,
-- 
Miro Hrončok
-- 
Phone: [phone]
IRC: mhroncok

[Petr Šplíchal]

Hi Miro,

I cannot help with the ansible magic but if you do not have STI as a hard
requirement I think there might be a relatively straightforward way to
cover your use case using tmt <https://docs.fedoraproject.org/en-US/ci/tmt/>
as it allows to set an environment
<https://tmt.readthedocs.io/en/latest/spec/plans.html#environment> variable
for the whole plan. Your rpms/python3.11 config could look like this:

discover:
    how: fmf
    url: https://src.fedoraproject.org/tests/python
    filter: component:python311
prepare:
    how: install
    package:
      - python3.11
      - python3.11-test
      - python3.11-tkinter
environment:
    VERSION: "3.11"
execute:
    how: tmt

In this way you would specify the python version and the proper package
list only once. The test metadata could be shared from the tests namespace,
no need to repeat the list again and again in individual rpms namespace
branches. I've outlined a simple setup in the following pull request:

https://src.fedoraproject.org/tests/python/pull-request/47

If this sounds like a viable solution feel free to ping me or anybody on
the #tmt channel for details. We'll be glad to help you with the config.

psss...

On Wed, 22 Jun 2022 at 12:56, Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:

>  Hello,
>
>  please bare with me because my Ansible skills are close to none. I only
>  use it
>  for Fedora CI.
>
>  I've read
>
>
> https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.ht...
>
>  I have this (simplified) tests.yml:
>
>       ---
>       - hosts: localhost
>         roles:
>         - role: standard-test-basic
>           tests:
>           - smoke:
>               dir: python/smoke
>               run: VERSION=3.11 ./venv.sh
>           required_packages:
>           - python3.11
>           - python3.11-devel
>
>  Full version in
>
>  https://src.fedoraproject.org/rpms/python3.11/blob/rawhide/f/tests/tests.yml
>
>  I would like to replace all occurrences of 3.11 with a variable:
>
>
>       ---
>       - hosts: localhost
>         roles:
>         - role: standard-test-basic
>           tests:
>           - smoke:
>               dir: python/smoke
>               run: VERSION={{ pybasever }} ./venv.sh
>           required_packages:
>           - python{{ pybasever }}
>           - python{{ pybasever }}-devel
>
>
>  But where do I put pybasever: '3.11' exactly?
>
>
>  Bonus question: Can I read the tested package name and regex it into the
>  variable?
>
>       pybasever:  {{ xxx_package_name|pex_regex_replace('^python', '') }}
>
>  Thanks,
>  --
>  Miro Hrončok
>  --
>  Phone: [phone]
>  IRC: mhroncok
>  _______________________________________________
>  CI mailing list -- ci(a)lists.fedoraproject.org
>  To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam on the list, report it:
>  https://pagure.io/fedora-infrastructure
>

[Aleksandra Fedorova]

Hi,

On Wed, Jun 22, 2022 at 12:56 PM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:
>  Hello,
>
>  please bare with me because my Ansible skills are close to none. I only use it
>  for Fedora CI.
>
>  I've read
>
> https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.ht...
>
>  I have this (simplified) tests.yml:
>
>       ---
>       - hosts: localhost
>         roles:
>         - role: standard-test-basic
>           tests:
>           - smoke:
>               dir: python/smoke
>               run: VERSION=3.11 ./venv.sh
>           required_packages:
>           - python3.11
>           - python3.11-devel
>
>  Full version in
>  https://src.fedoraproject.org/rpms/python3.11/blob/rawhide/f/tests/tests.yml
>
>  I would like to replace all occurrences of 3.11 with a variable:
>
>
>       ---
>       - hosts: localhost
>         roles:
>         - role: standard-test-basic
>           tests:
>           - smoke:
>               dir: python/smoke
>               run: VERSION={{ pybasever }} ./venv.sh
>           required_packages:
>           - python{{ pybasever }}
>           - python{{ pybasever }}-devel
>
>
>  But where do I put pybasever: '3.11' exactly? 
Vars can be defined on the same level as roles and hosts:

- hosts: localhost
  vars:
    pybasever: 3.11
  roles:
  - role: standard-test-basic
  ...

>  Bonus question: Can I read the tested package name and regex it into
> the variable?
>
>       pybasever:  {{ xxx_package_name|pex_regex_replace('^python', '') }}
I would need to dig more for this one.

The easiest way is to make a test run with a test just printing all
ENV variables, and checking if you can find the variable with the
right data in the log.

My uneducated guess would be that

    "{{ lookup('env', 'TEST_SUBJECTS') }}"

will provide the data. And then some more Ansible filter magic needed
to get the substring from it.

For example, with split
https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html...
Or with regex_search,
https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html...

>  Thanks,
>  --
>  Miro Hrončok
>  --
>  Phone: [phone]
>  IRC: mhroncok
>  _______________________________________________
>  CI mailing list -- ci(a)lists.fedoraproject.org
>  To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

-- 
Aleksandra Fedorova
bookwar

[Maxwell G]

On Wednesday, June 22, 2022 8:59:12 AM CDT Aleksandra Fedorova wrote:
>  On Wed, Jun 22, 2022 at 12:56 PM Miro Hrončok
> <mhroncok(a)redhat.com&gt; wrote: 
>  > Hello,
>  > 
>  > please bare with me because my Ansible skills are close to none. I only
>  > use
>  > it for Fedora CI. 
No problem. I'm happy to help :).

>  Vars can be defined on the same level as roles and hosts:
>  
>  - hosts: localhost
>    vars:
>      pybasever: 3.11
>    roles:
>    - role: standard-test-basic
>    ... 
You can also add variables in the `roles:` section.

```
  roles:
    - role: role_name_here
      var1: ...
      var2: ...
```

In your existing tests.yml, all of the key-value pairs in `roles:` besides 
`tags` and `role` are variables that are passed to the role. So, you can do

```
- hosts: localhost
  roles:
    - role: standard-test-basic
      tags: ...
      repositories: ...
      [...]
      pybasever: "3.11"
      required_packages:
        - "python3{{ pybasever }}"
        - "python3{{ pybasever }}-devel"

# Or equivalently:

- hosts: localhost
  roles:
    - role: standard-test-basic
      tags: ...
      vars:
        repositories: ...
        pybasever: "3.11"
        required_packages:
          - "python3{{ pybasever }}"
          - "python3{{ pybasever }}-devel"
```

In this case, it doesn't really matter at which level or in what order you 
define the variables, as variables are (re)evaluated each time they're used in 
a task. I just wanted to provide you some more context :).

Also, keep in mind that, if you're using jinja2 expressions
(e.g. {{ variable | filter }}), that you need to quote the whole expression so 
that it's valid yaml. See https://docs.ansible.com/ansible/latest/user_guide/
playbooks_variables.html#when-to-quote-variables-a-yaml-gotcha for a better 
explanation than I can give ;). 

>      "{{ lookup('env', 'TEST_SUBJECTS') }}"
Yes, this is the correct way to look up an env variable. I'm not familiar with 
Fedora CI, so I wouldn't know if that's the best way to get the package name.


-- 
Thanks,

Maxwell G (@gotmax23)
Pronouns: He/Him/His

[Miro Hrončok]

On 22. 06. 22 21:07, Maxwell G wrote:
>  On Wednesday, June 22, 2022 8:59:12 AM CDT Aleksandra Fedorova
> wrote:
> > On Wed, Jun 22, 2022 at 12:56 PM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:
>  
> >> Hello,
> >>
> >> please bare with me because my Ansible skills are close to none. I only
> >> use
> >> it for Fedora CI.
>  
>  No problem. I'm happy to help :).
>  
> > Vars can be defined on the same level as roles and hosts:
> >
> > - hosts: localhost
> >    vars:
> >      pybasever: 3.11
> >    roles:
> >    - role: standard-test-basic
> >    ...
>  
>  You can also add variables in the `roles:` section.
>  
>  ```
>     roles:
>       - role: role_name_here
>         var1: ...
>         var2: ...
>  ```
>  
>  In your existing tests.yml, all of the key-value pairs in `roles:` besides
>  `tags` and `role` are variables that are passed to the role. So, you can do
>  
>  ```
>  - hosts: localhost
>     roles:
>       - role: standard-test-basic
>         tags: ...
>         repositories: ...
>         [...]
>         pybasever: "3.11"
>         required_packages:
>           - "python3{{ pybasever }}"
>           - "python3{{ pybasever }}-devel"
>  
>  # Or equivalently:
>  
>  - hosts: localhost
>     roles:
>       - role: standard-test-basic
>         tags: ...
>         vars:
>           repositories: ...
>           pybasever: "3.11"
>           required_packages:
>             - "python3{{ pybasever }}"
>             - "python3{{ pybasever }}-devel"
>  ```
>  
>  In this case, it doesn't really matter at which level or in what order you
>  define the variables, as variables are (re)evaluated each time they're used in
>  a task. I just wanted to provide you some more context :).
>  
>  Also, keep in mind that, if you're using jinja2 expressions
>  (e.g. {{ variable | filter }}), that you need to quote the whole expression so
>  that it's valid yaml. See https://docs.ansible.com/ansible/latest/user_guide/
>  playbooks_variables.html#when-to-quote-variables-a-yaml-gotcha for a better
>  explanation than I can give ;).
>  
> >      "{{ lookup('env', 'TEST_SUBJECTS') }}"
>  
>  Yes, this is the correct way to look up an env variable. I'm not familiar with
>  Fedora CI, so I wouldn't know if that's the best way to get the package name.
Thank you both for the pointers. I'll dig.

-- 
Miro Hrončok
-- 
Phone: [phone]
IRC: mhroncok

[Miro Hrončok]

On 22. 06. 22 15:40, Petr Šplíchal wrote:
>  I've outlined a simple setup in the following pull request:
>  
>  https://src.fedoraproject.org/tests/python/pull-request/47 
>  <https://src.fedoraproject.org/tests/python/pull-request/47>
>  
>  If this sounds like a viable solution feel free to ping me or anybody on the 
>  #tmt channel for details. We'll be glad to help you with the config. 
Will do. Thanks.

-- 
Miro Hrončok
-- 
Phone: [phone]
IRC: mhroncok