koji-devel September 2023

koji-devel@lists.fedorahosted.org

4 participants
3 discussions

Custom output build types (btype) for CG best practices
by Tomáš Hozza 13 Sep '23

13 Sep '23

Hello. I'm from the Image Builder team at Red Hat. For those who don't know it, our team works on tooling for building various OS images and integrates with Koji as a CG (https://github.com/osbuild/koji-osbuild) Our current integration works in the following way: 1. Koji builder plugin makes a REST API call to our service. 2. Our service builds the image, uploads the image file to Koji and then imports it using CG API ( https://github.com/osbuild/osbuild-composer/blob/eb8b63f3ca166ecad13e10a21b… ). 3. Koji builder plugin waits for the image build to finish and then downloads logs and osbuild manifest (a kind of recipe allowing to rebuild the image) and uploads them to the build task. As a result, valuable files (such as logs and the osbuild manifest) are not attached to the build in Koji, but to the build task. Files attached to the task usually get garbage collected after some time (I guess depending on the Koji instance configuration), so as a result we are losing these files after some time. I'm currently reworking the way osbuild-composer imports content to Koji, so that we attach build logs and osbuild manifests to the Koji build as well. When we import outputs to Koji, we need to specify the output build type. For the image, we use "image", for logs we'll use "log", but the osbuild manifest would probably benefit from its own build type (e.g. "osbuild-manifest"). From what I checked, some content generators use their own build type (e.g. when I run "koji --noauth call listBTypes" with Fedora Koji, there's a "module" build type for MBS). I would like to ask what is the common or usual way for a Content Generator to mandate that a custom build type is available in Koji when using it? My understanding is that a new build type can be added to the Koji instance by an admin running the appropriate command. Is there some other way? I'm trying to determine what's the best way to handle the situation that our CG will start using a custom build type, while being already used by multiple Koji instances. Is the only way to document this requirement and work with responsible admins of Koji instances, which integrate with our CG, to adjust the configuration before we start using the new build type? Thanks in advance for any guidance. Regards, Tomas -- Tomáš Hozza Image Builder, Linux Engineering PGP: 908AF740C0F898A1079968CDC5887AD51D9F3C2D UTC+2 (CEST) / UTC+1 (CET) Red Hat Czech http://cz.redhat.com

3 4

Re: Inquiry Regarding CMake-Based Software Packaging for Koji
by Tomas Kopecek 06 Sep '23

06 Sep '23

On Mon, Sep 4, 2023 at 4:38 PM Joe (Jun-Yan) Chen <jychena(a)ambarella.com> wrote: > Dear Koji Development Team, > > > I hope this email finds you well. I have a couple of questions regarding > the packaging of CMake-based open-source software. > > Question 1: Can Koji's buildroot be deployed to access the Internet? > > For some open-source software projects, dependencies are downloaded during > the CMake configuration phase. However, Koji's buildroot access towards > internet(GitHub) is restricted. I attempted to pre-download these > dependencies by cmake or set them as 'BuildRequires', but it appears that > the intricate CMake system are inflexible and do not provide room for such > customization. > > In the context of packaging such software for compilation on Koji, aside > from modifying the CMake scripts, I would like to know if there are any > other more straightforward methods that have been recommended based on the > experience of previous packagers. > > Nope, currently not. It is a part of koji's philosophy for reproducible builds. Anything downloaded outside of curated git repos could change in the future and rebuild of such thing will not be the same. It is also a part of Fedora's packaging guidelines ( https://docs.fedoraproject.org/en-US/packaging-guidelines/#_build_time_netw… ) Normal solution is really to alter the package to not access the internet during the build and package all dependencies. > Question 2: Handling Hardcoded Paths in CMake > > When packaging certain open-source software using CMake, I noticed that > during the CMake configuration phase, if I set > "-DCMAKE_INSTALL_PREFIX=$RPM_BUILD_ROOT/opt/oss/", CMake tends to hardcode > this path with $RPM_BUILD_ROOT into various places, such as the RPATH of > binary libraries or CMake configuration files. Clearly, this is incorrect > for the files that end up in /opt/oss/. > > As a workaround, I've experimented with running the command "find > $RPM_BUILD_ROOT -type f -exec sed -i 's/${RPM_BUILD_ROOT}//g' {} \;" at the > end of the %install section, which successfully passes the > /usr/lib/rpm/check-buildroot check. However, I'm curious to know if this > approach will introduce potential issues or concerns? > There are some predefined macros in Fedora which you can use to handle this: https://docs.fedoraproject.org/en-US/packaging-guidelines/CMake/ > Your expertise and guidance on these matters would be highly valuable to > me. Thank you for your time and assistance. > > > Best Regards, > Junyan CHEN > > ------------------------------ > This email and attachments contain Ambarella Proprietary and/or > Confidential Information and is intended solely for the use of the > individual(s) to whom it is addressed. Any unauthorized review, use, > disclosure, distribute, copy, or print is prohibited. If you are not an > intended recipient, please contact the sender by reply email and destroy > all copies of the original message. Thank you. > _______________________________________________ > koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org > To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedorahosted.org/archives/list/koji-devel@lists.fedorahosted.… > Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue > -- Tomas Kopecek <tkopecek(a)redhat.com> RHEL Build Development, RedHat

2 1

Inquiry Regarding CMake-Based Software Packaging for Koji
by Joe (Jun-Yan) Chen 04 Sep '23

04 Sep '23

Dear Koji Development Team, I hope this email finds you well. I have a couple of questions regarding the packaging of CMake-based open-source software. Question 1: Can Koji's buildroot be deployed to access the Internet? For some open-source software projects, dependencies are downloaded during the CMake configuration phase. However, Koji's buildroot access towards internet(GitHub) is restricted. I attempted to pre-download these dependencies by cmake or set them as 'BuildRequires', but it appears that the intricate CMake system are inflexible and do not provide room for such customization. In the context of packaging such software for compilation on Koji, aside from modifying the CMake scripts, I would like to know if there are any other more straightforward methods that have been recommended based on the experience of previous packagers. Question 2: Handling Hardcoded Paths in CMake When packaging certain open-source software using CMake, I noticed that during the CMake configuration phase, if I set "-DCMAKE_INSTALL_PREFIX=$RPM_BUILD_ROOT/opt/oss/", CMake tends to hardcode this path with $RPM_BUILD_ROOT into various places, such as the RPATH of binary libraries or CMake configuration files. Clearly, this is incorrect for the files that end up in /opt/oss/. As a workaround, I've experimented with running the command "find $RPM_BUILD_ROOT -type f -exec sed -i 's/${RPM_BUILD_ROOT}//g' {} \;" at the end of the %install section, which successfully passes the /usr/lib/rpm/check-buildroot check. However, I'm curious to know if this approach will introduce potential issues or concerns? Your expertise and guidance on these matters would be highly valuable to me. Thank you for your time and assistance. Best Regards, Junyan CHEN ********************************************************************** This email and attachments contain Ambarella Proprietary and/or Confidential Information and is intended solely for the use of the individual(s) to whom it is addressed. Any unauthorized review, use, disclosure, distribute, copy, or print is prohibited. If you are not an intended recipient, please contact the sender by reply email and destroy all copies of the original message. Thank you.

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

koji-devel September 2023