Copr in 2020 and outlook for 2021

[Miroslav Suchý]

Let me sum up what we - the Copr team - did in 2020:

* We enabled CDN for repos. https://fedora-copr.github.io/posts/copr-cdn

* We enabled runtime dependecies on repositories https://fedora-copr.github.io/posts/runtime-dependencies

* We migrated all our servers from PHX datacenter to AWS. With zero downtime for your repos.

* Pavel Raiskup became the new Mock project leader, and he released seven new versions of Mock during 2020.
https://github.com/rpm-software-management/mock/wiki

* Our small team has been renamed to Community Packaging Team (CPT) because beside Copr, we maintain other tools as
well: Mock, dist-git, Tito...

* We drove changes in createrepo_c, which allowed us better throughput. Now, Copr can build thousands of builds per day.

* We worked on Fedora packages website, which was outdated and do not work on recent Fedora. Unfortunately, it will not
be likely used, and static pages will be used as it will require less maintenance.

* We introduced Project scoring for projects in Copr (up/down vote) http://frostyx.cz/posts/upvoting-projects-in-copr

* We allowed you to delete multiple builds at once. https://fedora-copr.github.io/posts/deleting-list-of-builds

* You can use Github Actions now https://pavel.raiskup.cz/blog/github-push-actions-and-copr.html

* We created modulemd-tools for low-level handling of repositories and modules. And tool for generating
module-build-macros. https://github.com/rpm-software-management/modulemd-tools

* We worked hard on better EOL handling of Copr repositories.

* We parallelized copr-dist-git imports.

* We wrote code for Ansible DNF Copr module. This still needs to land in a proper place like Ansible Galaxy.
https://pagure.io/copr/copr/blob/master/f/ansible

* We provided --isolation=* and --bootstrap=* knob in Copr

* We started an experimental "external dependencies" feature in Mock.
https://github.com/rpm-software-management/mock/wiki/Feature-external-deps

* We created a new python module "templated-dictionary" as a spin-off from Mock's code.
https://github.com/xsuchy/templated-dictionary

* We wrote four articles "4 cool new projects to try in COPR" https://fedoramagazine.org/tag/copr/

* Users now have the ability to change the build timeout up to a maximum of 48 hours

* Users can group builds in batches now. https://pavel.raiskup.cz/blog/build-ordering-by-batches-in-copr.html

We already have plans to do:

* Verify GPG checks in DNF using DNSSEC. https://pagure.io/copr/copr/issue/406

* Improve hooks (in cooperation with Packit team)

* We are going to EOL APIv1 and APIv2 https://fedora-copr.github.io/posts/EOL-APIv1-APIv2

What will we do in 2021? We have some ideas. Some of them are yours, some of them are ours. At the end of this email is
a link to Google Form where you can vote what you would like to get. We will consider your vote. (listed in no specific
order)

* When you create a project, you may specify that it will be for Package Review. We do some settings for you, and
`rpminspect`  will be run automatically at the end of the build. We can guide users on how to file Package Review BZ. Or
do that automatically on their behalf.

* Contribute to fedpkg/Koji that tagged commits will be automatically built in Koji and may be automatically sent to Bodhi.

* Help with automatic changelog entries.

* Native support for Fedora in Travis.

* Continue on external dependencies. https://github.com/rpm-software-management/mock/wiki/Feature-external-deps

* Finish rpm-spec-wizard https://github.com/xsuchy/rpm-spec-wizard

* Better integration with Koshei.

* Automatic rebuilds in Copr when dependency change (likely with the help of Koshei)

* Enhance release-monitoring and rebase-helper to automatically open PR in Pagure when new version is available.

* Enhance `Mock --chain` to try to set %bootstrap when standard loop fails. When the set succeeds, rebuild the
bootstrapped package again without %bootstrap macro.
https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping

* Improve the process of finding a sponsor for the first package.

* Contribute to fedpkg/koji to have machine-readable output.

Please vote here: https://forms.gle/UuH86ECzfZNHJgEr8
The results are intentionally publicly available. You may check them anytime.

CPT team consist of Pavel Raiskup, Silvie Chlupova, Jakub Kadlcik and this year we also had in the team Dominik Turecek
and Tomas Hrnciar.

[Alexander Scheel]

Congrats! I can say I've used several of these features and they work
well, thanks for your team's work!

One query inline... :)

On Mon, Jan 4, 2021 at 11:53 AM Miroslav Suchý  wrote:
> Let me sum up what we - the Copr team - did in 2020:
>
> We enabled CDN for repos. https://fedora-copr.github.io/posts/copr-cdn
>
> We enabled runtime dependecies on repositories https://fedora-copr.github.io/posts/runtime-dependencies
>
> We migrated all our servers from PHX datacenter to AWS. With zero downtime for your repos.
>
> Pavel Raiskup became the new Mock project leader, and he released seven new versions of Mock during 2020.
>
> https://github.com/rpm-software-management/mock/wiki
> Our small team has been renamed to Community Packaging Team (CPT) because beside Copr, we maintain other tools as
>
> well: Mock, dist-git, Tito...
>
> We drove changes in createrepo_c, which allowed us better throughput. Now, Copr can build thousands of builds per day.
>
> We worked on Fedora packages website, which was outdated and do not work on recent Fedora. Unfortunately, it will not
>
> be likely used, and static pages will be used as it will require less maintenance.
>
> We introduced Project scoring for projects in Copr (up/down vote) http://frostyx.cz/posts/upvoting-projects-in-copr
>
> We allowed you to delete multiple builds at once. https://fedora-copr.github.io/posts/deleting-list-of-builds
>
> You can use Github Actions now https://pavel.raiskup.cz/blog/github-push-actions-and-copr.html
>
> We created modulemd-tools for low-level handling of repositories and modules. And tool for generating
>
> module-build-macros. https://github.com/rpm-software-management/modulemd-tools
>
> We worked hard on better EOL handling of Copr repositories.
>
> We parallelized copr-dist-git imports.
>
> We wrote code for Ansible DNF Copr module. This still needs to land in a proper place like Ansible Galaxy.
>
> https://pagure.io/copr/copr/blob/master/f/ansible
>
> We provided --isolation=* and --bootstrap=* knob in Copr
>
> We started an experimental "external dependencies" feature in Mock.
>
> https://github.com/rpm-software-management/mock/wiki/Feature-external-deps
> We created a new python module "templated-dictionary" as a spin-off from Mock's code.
>
> https://github.com/xsuchy/templated-dictionary
>
> We wrote four articles "4 cool new projects to try in COPR" https://fedoramagazine.org/tag/copr/
>
> Users now have the ability to change the build timeout up to a maximum of 48 hours
>
> Users can group builds in batches now. https://pavel.raiskup.cz/blog/build-ordering-by-batches-in-copr.html
>
> We already have plans to do:
>
> Verify GPG checks in DNF using DNSSEC. https://pagure.io/copr/copr/issue/406
>
> Improve hooks (in cooperation with Packit team)
>
> We are going to EOL APIv1 and APIv2 https://fedora-copr.github.io/posts/EOL-APIv1-APIv2
>
> What will we do in 2021? We have some ideas. Some of them are yours, some of them are ours. At the end of this email is
> a link to Google Form where you can vote what you would like to get. We will consider your vote. (listed in no specific
> order)
> When you create a project, you may specify that it will be for Package Review. We do some settings for you, and
>
> `rpminspect`  will be run automatically at the end of the build. We can guide users on how to file Package Review BZ. Or
> do that automatically on their behalf.
>
> Contribute to fedpkg/Koji that tagged commits will be automatically built in Koji and may be automatically sent to Bodhi.
>
> Help with automatic changelog entries.
>
> Native support for Fedora in Travis.

Travis has made a lot of changes to how OSS projects can use it, and
(IMO) burnt a lot of good will in the community. All of our upstream
projects ended up moving off it and onto GitHub Actions. Is there any
chance we could vote for GitHub Actions enablement instead of Travis?
Currently we run a Fedora Docker image on top of the Ubuntu host,
which is less than ideal in some cases (e.g., filesystem package
upgrades...).

Thanks,

Alex
> Continue on external dependencies. https://github.com/rpm-software-management/mock/wiki/Feature-external-deps
>
> Finish rpm-spec-wizard https://github.com/xsuchy/rpm-spec-wizard
>
> Better integration with Koshei.
>
> Automatic rebuilds in Copr when dependency change (likely with the help of Koshei)
>
> Enhance release-monitoring and rebase-helper to automatically open PR in Pagure when new version is available.
>
> Enhance `Mock --chain` to try to set %bootstrap when standard loop fails. When the set succeeds, rebuild the
>
> bootstrapped package again without %bootstrap macro.
> https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping
>
> Improve the process of finding a sponsor for the first package.
>
> Contribute to fedpkg/koji to have machine-readable output.
>
> Please vote here: https://forms.gle/UuH86ECzfZNHJgEr8
> The results are intentionally publicly available. You may check them anytime.
>
> CPT team consist of Pavel Raiskup, Silvie Chlupova, Jakub Kadlcik and this year we also had in the team Dominik Turecek
> and Tomas Hrnciar.
>
> --
> Miroslav Suchy, RHCA
> Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys
> _______________________________________________
> devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]

[Miro Hrončok]

On 04. 01. 21 18:40, Alexander Scheel wrote:
> > Native support for Fedora in Travis.
>
> Travis has made a lot of changes to how OSS projects can use it, and
> (IMO) burnt a lot of good will in the community. All of our upstream
> projects ended up moving off it and onto GitHub Actions. Is there any
> chance we could vote for GitHub Actions enablement instead of Travis?
> Currently we run a Fedora Docker image on top of the Ubuntu host,
> which is less than ideal in some cases (e.g., filesystem package
> upgrades...).

I wanted to ask the same thing.

And also to say: Community Packaging Team, you rock!

[Dan Čermák]

Hi,

Miroslav Suchý  writes:
> Let me sum up what we - the Copr team - did in 2020:
>
> We enabled CDN for repos. https://fedora-copr.github.io/posts/copr-cdn
>
> We enabled runtime dependecies on repositories https://fedora-copr.github.io/posts/runtime-dependencies
>
> We migrated all our servers from PHX datacenter to AWS. With zero downtime for your repos.
>
> Pavel Raiskup became the new Mock project leader, and he released seven new versions of Mock during 2020.
>
> https://github.com/rpm-software-management/mock/wiki
> Our small team has been renamed to Community Packaging Team (CPT) because beside Copr, we maintain other tools as
>
> well: Mock, dist-git, Tito...
>
> We drove changes in createrepo_c, which allowed us better throughput. Now, Copr can build thousands of builds per day.
>
> We worked on Fedora packages website, which was outdated and do not work on recent Fedora. Unfortunately, it will not
>
> be likely used, and static pages will be used as it will require less maintenance.
>
> We introduced Project scoring for projects in Copr (up/down vote) http://frostyx.cz/posts/upvoting-projects-in-copr
>
> We allowed you to delete multiple builds at once. https://fedora-copr.github.io/posts/deleting-list-of-builds
>
> You can use Github Actions now https://pavel.raiskup.cz/blog/github-push-actions-and-copr.html
>
> We created modulemd-tools for low-level handling of repositories and modules. And tool for generating
>
> module-build-macros. https://github.com/rpm-software-management/modulemd-tools
>
> We worked hard on better EOL handling of Copr repositories.
>
> We parallelized copr-dist-git imports.
>
> We wrote code for Ansible DNF Copr module. This still needs to land in a proper place like Ansible Galaxy.
>
> https://pagure.io/copr/copr/blob/master/f/ansible
>
> We provided --isolation=* and --bootstrap=* knob in Copr
>
> We started an experimental "external dependencies" feature in Mock.
>
> https://github.com/rpm-software-management/mock/wiki/Feature-external-deps
> We created a new python module "templated-dictionary" as a spin-off from Mock's code.
>
> https://github.com/xsuchy/templated-dictionary
>
> We wrote four articles "4 cool new projects to try in COPR" https://fedoramagazine.org/tag/copr/
>
> Users now have the ability to change the build timeout up to a maximum of 48 hours
>
> Users can group builds in batches now. https://pavel.raiskup.cz/blog/build-ordering-by-batches-in-copr.html

That's quite an impressive list!
> We already have plans to do:
>
> Verify GPG checks in DNF using DNSSEC. https://pagure.io/copr/copr/issue/406
>
> Improve hooks (in cooperation with Packit team)
>
> We are going to EOL APIv1 and APIv2 https://fedora-copr.github.io/posts/EOL-APIv1-APIv2
>
> What will we do in 2021? We have some ideas. Some of them are yours, some of them are ours. At the end of this email is
> a link to Google Form where you can vote what you would like to get. We will consider your vote. (listed in no specific
> order)
> When you create a project, you may specify that it will be for Package Review. We do some settings for you, and
>
> `rpminspect`  will be run automatically at the end of the build. We can guide users on how to file Package Review BZ. Or
> do that automatically on their behalf.
>
> Contribute to fedpkg/Koji that tagged commits will be automatically built in Koji and may be automatically sent to Bodhi.
>
> Help with automatic changelog entries.
>
> Native support for Fedora in Travis.

I'd say don't bother with Travis, they are becoming increasingly
unusable for non-paying users and I've seen many projects migrate to
github actions.
> Continue on external dependencies. https://github.com/rpm-software-management/mock/wiki/Feature-external-deps
>
> Finish rpm-spec-wizard https://github.com/xsuchy/rpm-spec-wizard
>
> Better integration with Koshei.
>
> Automatic rebuilds in Copr when dependency change (likely with the help of Koshei)

YES! Please, please, please make this possible. And an additional box of
cookies for you all if it will work with dynamic buildrequires ;-)
> Enhance release-monitoring and rebase-helper to automatically open PR in Pagure when new version is available.
>
> Enhance `Mock --chain` to try to set %bootstrap when standard loop fails. When the set succeeds, rebuild the
>
> bootstrapped package again without %bootstrap macro.
> https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping
>
> Improve the process of finding a sponsor for the first package.
>
> Contribute to fedpkg/koji to have machine-readable output.
>
> Please vote here: https://forms.gle/UuH86ECzfZNHJgEr8
> The results are intentionally publicly available. You may check them anytime.

Is it intentional that google asks me to sign in? I don't actually have
a google account.

Cheers,

Dan

[Miroslav Suchý]

Dne 04. 01. 21 v 20:36 Dan Čermák napsal(a):
> Is it intentional that google asks me to sign in? I don't actually have
> a google account.

Yes. It is intentional. The motivation is that you can change your vote later.

If you do not have google account then just write me an email. I will add it to spreadsheet manually.

[Miroslav Suchý]

Dne 04. 01. 21 v 20:14 Miro Hrončok napsal(a):
> > Is there any
> > chance we could vote for GitHub Actions enablement instead of Travis?
> > Currently we run a Fedora Docker image on top of the Ubuntu host,
> > which is less than ideal in some cases (e.g., filesystem package
> > upgrades...).
>
> I wanted to ask the same thing.

https://pavel.raiskup.cz/blog/github-push-actions-and-copr.html

This is very basic example. Can you give me example of workflows you would like to set up? So we can document them (or
check whether there is some blocker).

[Miro Hrončok]

On 04. 01. 21 20:48, Miroslav Suchý wrote:
> Dne 04. 01. 21 v 20:14 Miro Hrončok napsal(a):
> > > Is there any
> > > chance we could vote for GitHub Actions enablement instead of Travis?
> > > Currently we run a Fedora Docker image on top of the Ubuntu host,
> > > which is less than ideal in some cases (e.g., filesystem package
> > > upgrades...).
> >
> > I wanted to ask the same thing.
>
> https://pavel.raiskup.cz/blog/github-push-actions-and-copr.html
>
> This is very basic example. Can you give me example of workflows you would like to set up? So we can document them (or
> check whether there is some blocker).

I am talking about running upstream tests via GitHub actions on Fedora directly 
instead of dockerized Fedora on Ubuntu. E.g. the "Native support for Fedora in 
Travis" point in the original email (the line I forgot to quote). Not about Copr.

[Miro Hrončok]

On 04. 01. 21 20:53, Miro Hrončok wrote:
> On 04. 01. 21 20:48, Miroslav Suchý wrote:
> > Dne 04. 01. 21 v 20:14 Miro Hrončok napsal(a):
> > > > Is there any
> > > > chance we could vote for GitHub Actions enablement instead of Travis?
> > > > Currently we run a Fedora Docker image on top of the Ubuntu host,
> > > > which is less than ideal in some cases (e.g., filesystem package
> > > > upgrades...).
> > >
> > > I wanted to ask the same thing.
> >
> > https://pavel.raiskup.cz/blog/github-push-actions-and-copr.html
> >
> > This is very basic example. Can you give me example of workflows you would 
> > like to set up? So we can document them (or
> > check whether there is some blocker).
>
> I am talking about running upstream tests via GitHub actions on Fedora directly 
> instead of dockerized Fedora on Ubuntu. E.g. the "Native support for Fedora in 
> Travis" point in the original email (the line I forgot to quote). Not about Copr.

As a specific example of a GH Actions workflow, I would like to do this:

jobs:
   build:
     runs-on: fedora-latest
     steps:
     - ...

[Ken Dreyer]

On Mon, Jan 4, 2021 at 9:52 AM Miroslav Suchý  wrote:
> Contribute to fedpkg/koji to have machine-readable output.

There is a "--json" argument to "koji call", and that produces
machine-readable output for individual Koji RPCs.

It would be really nice if rpkg had something similar, or even an
easier-to-extend Python API. There was an attempt a while back, with
https://pagure.io/rpkg2 .

- Ken

[Ken Dreyer]

On Sat, Feb 13, 2021 at 3:06 PM Ken Dreyer  wrote:
> On Mon, Jan 4, 2021 at 9:52 AM Miroslav Suchý  wrote:
> > Contribute to fedpkg/koji to have machine-readable output.
>
> There is a "--json" argument to "koji call", and that produces
> machine-readable output for individual Koji RPCs.
>
> It would be really nice if rpkg had something similar, or even an
> easier-to-extend Python API. There was an attempt a while back, with
> https://pagure.io/rpkg2 .

Incidentally I ran into this on another project with a coworker
recently, so I opened a ticket to track it:
https://pagure.io/rpkg/issue/542

- Ken