Announcing start of DNF 5 development

[Daniel Mach]

Hello everyone,
I'm pleased to announce start of DNF 5 development. We are planning to 
deliver a module stream or a COPR repo during Fedora 33 development for 
early adopters and tool developers and we're hoping in getting a stable 
version into Fedora 34.

More details follow.

We've managed to drop a lot of redundant code across the whole DNF stack 
in the past years, but we have reached a point when it's nearly 
impossible to consolidate the code any further without breaking the 
API/ABI. Especially with PackageKit being dead[1], we can't move with 
the old "libhif" API in libdnf, because making any bigger changes to 
PackageKit is clearly out of scope.

[1] 
https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-well...

That's why we decided to start working on a new version of the DNF 
stack: DNF 5. And this is the plan:

Priorities
----------
1. Consistency, documentation and user experience is the top priority.
2. Compatibility on the command line level.
3. Compatibility on the API level.

Maintenance
-----------
The existing DNF 4 stack stays in the current Fedoras and Red Hat 
Enterprise Linux 8. We'll keep maintaining it in dnf-4-master branches 
on GitHub. PackageKit and rpm-ostree will stay on libdnf from the DNF 4 
stack.

The existing Python API in DNF
------------------------------
The Python API in DNF stays. We'll do our best to keep it working. If 
there is an incompatible change, we'll communicate and document it properly.

The new API in libdnf
---------------------
All business logic will move from DNF (Python) to libdnf (C++). This is 
the only way to ensure that package managers work identically across the 
whole distribution. We'll start with C++ API and auto-generated Python 
bindings via SWIG. We'll focus on the Python bindings which are required 
by DNF and we will do our best to provide bindings for Go, Perl5 and 
Ruby as well. C API will be created later when the C++ API is stable. At 
that moment rpm-ostree will be ported to the new C API.

hawkey
------
Hawkey Python API is going away and will be replaced with libdnf Python API.

DNF
---
DNF stays as the primary command-line package manager. The overall 
functionality remains. We don't anticipate any negative impact of the 
API rewrite on the end-users. We have built an extensive test suite 
(over 1400 scenarios) that will help us to ensure that. The argument 
parser and outputs may slightly change in some cases to provide a more 
consistent user-experience. All such cases will be properly documented.

microdnf
--------
Microdnf is becoming important because it's part of many containers
due to its small footprint. We're getting feedback that users would
appreciate something closer to DNF. We'll focus on implementing a subset 
of DNF's functionality and improving the user experience. 100% feature 
parity with DNF is currently out of scope.

DBus service
------------
DNF team has decided to create a new DBus service replacing PackageKit 
to provide an interface to GUI applications. It's probably going to take 
a while because we're planning to start from scratch.

Roadmap (tentative)
-------------------
* Mar 2020 - making the bigger API changes, upstream code barely compiles
* May 2020 - COPR repo with first development snapshots
* Jun 2020 - F33 module available for early adopters and tool developers
* Oct 2020 - DNF 5 landing in F34 Rawhide
* Feb 2021 - DNF 5 replacing DNF 4 in stable Fedora

[Kamil Paral]

On Wed, Mar 4, 2020 at 7:04 PM Daniel Mach  wrote:
> Hello everyone,
> I'm pleased to announce start of DNF 5 development. We are planning to
> deliver a module stream or a COPR repo during Fedora 33 development for
> early adopters and tool developers and we're hoping in getting a stable
> version into Fedora 34.

Thanks for the announcement. I definitely prefer a COPR repo to a module
stream :-)

[Kevin Kofler]

Kamil Paral wrote:
> On Wed, Mar 4, 2020 at 7:04 PM Daniel Mach  wrote:
> > Hello everyone,
> > I'm pleased to announce start of DNF 5 development. We are planning to
> > deliver a module stream or a COPR repo during Fedora 33 development for
> > early adopters and tool developers and we're hoping in getting a stable
> > version into Fedora 34.
>
> Thanks for the announcement. I definitely prefer a COPR repo to a module
> stream :-)

+1

Kevin Kofler

[Zbigniew Jędrzejewski-Szmek]

On Wed, Mar 04, 2020 at 07:03:01PM +0100, Daniel Mach wrote:
> Hello everyone,
> I'm pleased to announce start of DNF 5 development. We are planning
> to deliver a module stream or a COPR repo during Fedora 33
> development for early adopters and tool developers and we're hoping
> in getting a stable version into Fedora 34.
>
> More details follow.
>
> We've managed to drop a lot of redundant code across the whole DNF
> stack in the past years, but we have reached a point when it's
> nearly impossible to consolidate the code any further without
> breaking the API/ABI. Especially with PackageKit being dead[1], we
> can't move with the old "libhif" API in libdnf, because making any
> bigger changes to PackageKit is clearly out of scope.
>
> [1] https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-well...
>
> That's why we decided to start working on a new version of the DNF
> stack: DNF 5. And this is the plan:
> Priorities
> Consistency, documentation and user experience is the top priority.
> Compatibility on the command line level.
> Compatibility on the API level.
> Maintenance
>
> The existing DNF 4 stack stays in the current Fedoras and Red Hat
> Enterprise Linux 8. We'll keep maintaining it in dnf-4-master
> branches on GitHub. PackageKit and rpm-ostree will stay on libdnf
> from the DNF 4 stack.
> The existing Python API in DNF
>
> The Python API in DNF stays. We'll do our best to keep it working.
> If there is an incompatible change, we'll communicate and document
> it properly.
> The new API in libdnf
>
> All business logic will move from DNF (Python) to libdnf (C++). This
> is the only way to ensure that package managers work identically
> across the whole distribution. We'll start with C++ API and
> auto-generated Python bindings via SWIG. We'll focus on the Python
> bindings which are required by DNF and we will do our best to
> provide bindings for Go, Perl5 and Ruby as well. C API will be
> created later when the C++ API is stable. At that moment rpm-ostree
> will be ported to the new C API.
> hawkey
>
> Hawkey Python API is going away and will be replaced with libdnf Python API.
> DNF
>
> DNF stays as the primary command-line package manager. The overall
> functionality remains. We don't anticipate any negative impact of
> the API rewrite on the end-users. We have built an extensive test
> suite (over 1400 scenarios) that will help us to ensure that. The
> argument parser and outputs may slightly change in some cases to
> provide a more consistent user-experience. All such cases will be
> properly documented.
> microdnf
>
> Microdnf is becoming important because it's part of many containers
> due to its small footprint. We're getting feedback that users would
> appreciate something closer to DNF. We'll focus on implementing a
> subset of DNF's functionality and improving the user experience.
> 100% feature parity with DNF is currently out of scope.

Hi,

the roadmap is ambitious, but not impossible. Good luck!
> Roadmap (tentative)
> Mar 2020 - making the bigger API changes, upstream code barely compiles
> May 2020 - COPR repo with first development snapshots
> Jun 2020 - F33 module available for early adopters and tool developers
> Oct 2020 - DNF 5 landing in F34 Rawhide
> Feb 2021 - DNF 5 replacing DNF 4 in stable Fedora
> DBus service
>
> DNF team has decided to create a new DBus service replacing
> PackageKit to provide an interface to GUI applications. It's
> probably going to take a while because we're planning to start from
> scratch.

Do you plan to make normal 'dnf' calls go through the dbus api?
(And e.g. provide a single cache and privilege escalation through
packagekit)?

Apart from the dbus api, do you plan to provide some graphical
application that uses this api?

Are you going to use sd-bus for the dbus library?

Zbyszek

[Neal Gompa]

On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
 wrote:
> On Wed, Mar 04, 2020 at 07:03:01PM +0100, Daniel Mach wrote:
> > Hello everyone,
> > I'm pleased to announce start of DNF 5 development. We are planning
> > to deliver a module stream or a COPR repo during Fedora 33
> > development for early adopters and tool developers and we're hoping
> > in getting a stable version into Fedora 34.
> >
> > More details follow.
> >
> > We've managed to drop a lot of redundant code across the whole DNF
> > stack in the past years, but we have reached a point when it's
> > nearly impossible to consolidate the code any further without
> > breaking the API/ABI. Especially with PackageKit being dead[1], we
> > can't move with the old "libhif" API in libdnf, because making any
> > bigger changes to PackageKit is clearly out of scope.
> >
> > [1] https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-well...
> >
> > That's why we decided to start working on a new version of the DNF
> > stack: DNF 5. And this is the plan:
> > Priorities
> > Consistency, documentation and user experience is the top priority.
> > Compatibility on the command line level.
> > Compatibility on the API level.
> > Maintenance
> >
> > The existing DNF 4 stack stays in the current Fedoras and Red Hat
> > Enterprise Linux 8. We'll keep maintaining it in dnf-4-master
> > branches on GitHub. PackageKit and rpm-ostree will stay on libdnf
> > from the DNF 4 stack.
> > The existing Python API in DNF
> >
> > The Python API in DNF stays. We'll do our best to keep it working.
> > If there is an incompatible change, we'll communicate and document
> > it properly.
> > The new API in libdnf
> >
> > All business logic will move from DNF (Python) to libdnf (C++). This
> > is the only way to ensure that package managers work identically
> > across the whole distribution. We'll start with C++ API and
> > auto-generated Python bindings via SWIG. We'll focus on the Python
> > bindings which are required by DNF and we will do our best to
> > provide bindings for Go, Perl5 and Ruby as well. C API will be
> > created later when the C++ API is stable. At that moment rpm-ostree
> > will be ported to the new C API.
> > hawkey
> >
> > Hawkey Python API is going away and will be replaced with libdnf Python API.
> > DNF
> >
> > DNF stays as the primary command-line package manager. The overall
> > functionality remains. We don't anticipate any negative impact of
> > the API rewrite on the end-users. We have built an extensive test
> > suite (over 1400 scenarios) that will help us to ensure that. The
> > argument parser and outputs may slightly change in some cases to
> > provide a more consistent user-experience. All such cases will be
> > properly documented.
> > microdnf
> >
> > Microdnf is becoming important because it's part of many containers
> > due to its small footprint. We're getting feedback that users would
> > appreciate something closer to DNF. We'll focus on implementing a
> > subset of DNF's functionality and improving the user experience.
> > 100% feature parity with DNF is currently out of scope.
>
> Hi,
>
> the roadmap is ambitious, but not impossible. Good luck!
> > Roadmap (tentative)
> > Mar 2020 - making the bigger API changes, upstream code barely compiles
> > May 2020 - COPR repo with first development snapshots
> > Jun 2020 - F33 module available for early adopters and tool developers
> > Oct 2020 - DNF 5 landing in F34 Rawhide
> > Feb 2021 - DNF 5 replacing DNF 4 in stable Fedora
> > DBus service
> >
> > DNF team has decided to create a new DBus service replacing
> > PackageKit to provide an interface to GUI applications. It's
> > probably going to take a while because we're planning to start from
> > scratch.
>
> Do you plan to make normal 'dnf' calls go through the dbus api?

This would be interesting, but wouldn't that make using DNF in rescue
situations impossible?
> (And e.g. provide a single cache and privilege escalation through
> packagekit)?

We can do the single cache thing *today* for PackageKit. The APIs
exist in libdnf _now_, it's just that they're not used
PackageKit-side.
> Apart from the dbus api, do you plan to provide some graphical
> application that uses this api?

I would expect that dnfdragora will be the first consumer of this new
API, since this plan would essentially involve taking over the role of
my dnfdaemon.
> Are you going to use sd-bus for the dbus library?

I'd hope not, given that we have cross-distro usage of DNF now, and a
couple of them don't have systemd.

[Martin Kolman]

----- Original Message -----
> From: "Neal Gompa" 
> To: "Development discussions related to Fedora" 
> Sent: Wednesday, March 4, 2020 11:01:43 PM
> Subject: Re: Announcing start of DNF 5 development
>
> On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
>  wrote:
> > On Wed, Mar 04, 2020 at 07:03:01PM +0100, Daniel Mach wrote:
> > > Hello everyone,
> > > I'm pleased to announce start of DNF 5 development. We are planning
> > > to deliver a module stream or a COPR repo during Fedora 33
> > > development for early adopters and tool developers and we're hoping
> > > in getting a stable version into Fedora 34.
> > >
> > > More details follow.
> > >
> > > We've managed to drop a lot of redundant code across the whole DNF
> > > stack in the past years, but we have reached a point when it's
> > > nearly impossible to consolidate the code any further without
> > > breaking the API/ABI. Especially with PackageKit being dead[1], we
> > > can't move with the old "libhif" API in libdnf, because making any
> > > bigger changes to PackageKit is clearly out of scope.
> > >
> > > [1]
> > > https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-well...
> > >
> > > That's why we decided to start working on a new version of the DNF
> > > stack: DNF 5. And this is the plan:
> > > Priorities
> > > Consistency, documentation and user experience is the top priority.
> > > Compatibility on the command line level.
> > > Compatibility on the API level.
> > > Maintenance
> > >
> > > The existing DNF 4 stack stays in the current Fedoras and Red Hat
> > > Enterprise Linux 8. We'll keep maintaining it in dnf-4-master
> > > branches on GitHub. PackageKit and rpm-ostree will stay on libdnf
> > > from the DNF 4 stack.
> > > The existing Python API in DNF
> > >
> > > The Python API in DNF stays. We'll do our best to keep it working.
> > > If there is an incompatible change, we'll communicate and document
> > > it properly.
> > > The new API in libdnf
> > >
> > > All business logic will move from DNF (Python) to libdnf (C++). This
> > > is the only way to ensure that package managers work identically
> > > across the whole distribution. We'll start with C++ API and
> > > auto-generated Python bindings via SWIG. We'll focus on the Python
> > > bindings which are required by DNF and we will do our best to
> > > provide bindings for Go, Perl5 and Ruby as well. C API will be
> > > created later when the C++ API is stable. At that moment rpm-ostree
> > > will be ported to the new C API.
> > > hawkey
> > >
> > > Hawkey Python API is going away and will be replaced with libdnf Python
> > > API.
> > > DNF
> > >
> > > DNF stays as the primary command-line package manager. The overall
> > > functionality remains. We don't anticipate any negative impact of
> > > the API rewrite on the end-users. We have built an extensive test
> > > suite (over 1400 scenarios) that will help us to ensure that. The
> > > argument parser and outputs may slightly change in some cases to
> > > provide a more consistent user-experience. All such cases will be
> > > properly documented.
> > > microdnf
> > >
> > > Microdnf is becoming important because it's part of many containers
> > > due to its small footprint. We're getting feedback that users would
> > > appreciate something closer to DNF. We'll focus on implementing a
> > > subset of DNF's functionality and improving the user experience.
> > > 100% feature parity with DNF is currently out of scope.
> >
> > Hi,
> >
> > the roadmap is ambitious, but not impossible. Good luck!
> > > Roadmap (tentative)
> > > Mar 2020 - making the bigger API changes, upstream code barely compiles
> > > May 2020 - COPR repo with first development snapshots
> > > Jun 2020 - F33 module available for early adopters and tool developers
> > > Oct 2020 - DNF 5 landing in F34 Rawhide
> > > Feb 2021 - DNF 5 replacing DNF 4 in stable Fedora
> > > DBus service
> > >
> > > DNF team has decided to create a new DBus service replacing
> > > PackageKit to provide an interface to GUI applications. It's
> > > probably going to take a while because we're planning to start from
> > > scratch.
> >
> > Do you plan to make normal 'dnf' calls go through the dbus api?
>
> This would be interesting, but wouldn't that make using DNF in rescue
> situations impossible?

You mean due to the regular DBus daemon & system + session bus not running
in some system rescue scenarios, right ?

While possibly a bit tricky I could imagine some fallback mechanism where
invocation of the CLI tool starts it's own DBus session & instance of the DNF
service when it detects that the regular system & session buses are not available.

Anaconda does something similar when it starts in Mock during some phases of the
compose process & finds no system bus is available - it starts it's own DBus daemon
process and uses that.
> > (And e.g. provide a single cache and privilege escalation through
> > packagekit)?
>
> We can do the single cache thing *today* for PackageKit. The APIs
> exist in libdnf _now_, it's just that they're not used
> PackageKit-side.
> > Apart from the dbus api, do you plan to provide some graphical
> > application that uses this api?
>
> I would expect that dnfdragora will be the first consumer of this new
> API, since this plan would essentially involve taking over the role of
> my dnfdaemon.
> > Are you going to use sd-bus for the dbus library?
>
> I'd hope not, given that we have cross-distro usage of DNF now, and a
> couple of them don't have systemd.
>
> --
> 真実はいつも一つ！/ Always, there's only one truth!
> _______________________________________________
> devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Fabio Valentini]

On Thu, Mar 5, 2020, 00:55 Martin Kolman  wrote:
> ----- Original Message -----
> > From: "Neal Gompa" 
> > To: "Development discussions related to Fedora" <
>
> [e-mail]>
> > Sent: Wednesday, March 4, 2020 11:01:43 PM
> > Subject: Re: Announcing start of DNF 5 development
> >
> > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> >  wrote:
> > > On Wed, Mar 04, 2020 at 07:03:01PM +0100, Daniel Mach wrote:
> > > > Hello everyone,
> > > > I'm pleased to announce start of DNF 5 development. We are planning
> > > > to deliver a module stream or a COPR repo during Fedora 33
> > > > development for early adopters and tool developers and we're hoping
> > > > in getting a stable version into Fedora 34.
> > > >
> > > > More details follow.
> > > >
> > > > We've managed to drop a lot of redundant code across the whole DNF
> > > > stack in the past years, but we have reached a point when it's
> > > > nearly impossible to consolidate the code any further without
> > > > breaking the API/ABI. Especially with PackageKit being dead[1], we
> > > > can't move with the old "libhif" API in libdnf, because making any
> > > > bigger changes to PackageKit is clearly out of scope.
> > > >
> > > > [1]
>
> https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-well...
> > > > That's why we decided to start working on a new version of the DNF
> > > > stack: DNF 5. And this is the plan:
> > > > Priorities
> > > > Consistency, documentation and user experience is the top
>
> priority.
> > > > Compatibility on the command line level.
> > > > Compatibility on the API level.
> > > > Maintenance
> > > >
> > > > The existing DNF 4 stack stays in the current Fedoras and Red Hat
> > > > Enterprise Linux 8. We'll keep maintaining it in dnf-4-master
> > > > branches on GitHub. PackageKit and rpm-ostree will stay on libdnf
> > > > from the DNF 4 stack.
> > > > The existing Python API in DNF
> > > >
> > > > The Python API in DNF stays. We'll do our best to keep it working.
> > > > If there is an incompatible change, we'll communicate and document
> > > > it properly.
> > > > The new API in libdnf
> > > >
> > > > All business logic will move from DNF (Python) to libdnf (C++). This
> > > > is the only way to ensure that package managers work identically
> > > > across the whole distribution. We'll start with C++ API and
> > > > auto-generated Python bindings via SWIG. We'll focus on the Python
> > > > bindings which are required by DNF and we will do our best to
> > > > provide bindings for Go, Perl5 and Ruby as well. C API will be
> > > > created later when the C++ API is stable. At that moment rpm-ostree
> > > > will be ported to the new C API.
> > > > hawkey
> > > >
> > > > Hawkey Python API is going away and will be replaced with libdnf
>
> Python
> > > > API.
> > > > DNF
> > > >
> > > > DNF stays as the primary command-line package manager. The overall
> > > > functionality remains. We don't anticipate any negative impact of
> > > > the API rewrite on the end-users. We have built an extensive test
> > > > suite (over 1400 scenarios) that will help us to ensure that. The
> > > > argument parser and outputs may slightly change in some cases to
> > > > provide a more consistent user-experience. All such cases will be
> > > > properly documented.
> > > > microdnf
> > > >
> > > > Microdnf is becoming important because it's part of many containers
> > > > due to its small footprint. We're getting feedback that users would
> > > > appreciate something closer to DNF. We'll focus on implementing a
> > > > subset of DNF's functionality and improving the user experience.
> > > > 100% feature parity with DNF is currently out of scope.
> > >
> > > Hi,
> > >
> > > the roadmap is ambitious, but not impossible. Good luck!
> > > > Roadmap (tentative)
> > > > Mar 2020 - making the bigger API changes, upstream code barely
>
> compiles
> > > > May 2020 - COPR repo with first development snapshots
> > > > Jun 2020 - F33 module available for early adopters and tool
>
> developers
> > > > Oct 2020 - DNF 5 landing in F34 Rawhide
> > > > Feb 2021 - DNF 5 replacing DNF 4 in stable Fedora
> > > > DBus service
> > > >
> > > > DNF team has decided to create a new DBus service replacing
> > > > PackageKit to provide an interface to GUI applications. It's
> > > > probably going to take a while because we're planning to start from
> > > > scratch.
> > >
> > > Do you plan to make normal 'dnf' calls go through the dbus api?
> >
> > This would be interesting, but wouldn't that make using DNF in rescue
> > situations impossible?
>
> You mean due to the regular DBus daemon & system + session bus not running
> in some system rescue scenarios, right ?
>
> While possibly a bit tricky I could imagine some fallback mechanism where
> invocation of the CLI tool starts it's own DBus session & instance of the
> DNF
> service when it detects that the regular system & session buses are not
> available.
>
> Anaconda does something similar when it starts in Mock during some phases
> of the
> compose process & finds no system bus is available - it starts it's own
> DBus daemon
> process and uses that.

Wow, using dbus to communicate between CLI binary and the shared library
sounds like an awful idea. Why not do a simple shim around the shared
library instead? (And not introduce another 2 moving parts into a critical
system component: dnf dbus service and dnf dbus client)

Fabio
> > > (And e.g. provide a single cache and privilege escalation through
> > > packagekit)?
> >
> > We can do the single cache thing *today* for PackageKit. The APIs
> > exist in libdnf _now_, it's just that they're not used
> > PackageKit-side.
> > > Apart from the dbus api, do you plan to provide some graphical
> > > application that uses this api?
> >
> > I would expect that dnfdragora will be the first consumer of this new
> > API, since this plan would essentially involve taking over the role of
> > my dnfdaemon.
> > > Are you going to use sd-bus for the dbus library?
> >
> > I'd hope not, given that we have cross-distro usage of DNF now, and a
> > couple of them don't have systemd.
> >
> > --
> > 真実はいつも一つ！/ Always, there's only one truth!
> > _______________________________________________
> > devel mailing list -- [e-mail]
> > To unsubscribe send an email to [e-mail]
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
>
> https://lists.fedoraproject.org/archives/list/[e-mail]
> >
>
> devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

On Thu, 2020-03-05 at 07:44 +0100, Fabio Valentini wrote:
> On Thu, Mar 5, 2020, 00:55 Martin Kolman  wrote:
> > ----- Original Message -----
> > > From: "Neal Gompa"
> > > To: "Development discussions related to Fedora" <
> >
> > [e-mail]>
> > > Sent: Wednesday, March 4, 2020 11:01:43 PM
> > > Subject: Re: Announcing start of DNF 5 development
> > >
> > > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> > >  wrote:
> > > >
> > > > On Wed, Mar 04, 2020 at 07:03:01PM +0100, Daniel Mach wrote:
> > > > > Hello everyone,
> > > > > I'm pleased to announce start of DNF 5 development. We are
> >
> > planning
> > > > > to deliver a module stream or a COPR repo during Fedora 33
> > > > > development for early adopters and tool developers and we're
> >
> > hoping
> > > > > in getting a stable version into Fedora 34.
> > > > >
> > > > >
> > > > > More details follow.
> > > > >
> > > > >
> > > > > We've managed to drop a lot of redundant code across the
> >
> > whole DNF
> > > > > stack in the past years, but we have reached a point when
> >
> > it's
> > > > > nearly impossible to consolidate the code any further without
> > > > > breaking the API/ABI. Especially with PackageKit being
> >
> > dead[1], we
> > > > > can't move with the old "libhif" API in libdnf, because
> >
> > making any
> > > > > bigger changes to PackageKit is clearly out of scope.
> > > > >
> > > > > [1]
> > > > >
> >
> > https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-well...
> > > > >
> > > > >
> > > > > That's why we decided to start working on a new version of
> >
> > the DNF
> > > > > stack: DNF 5. And this is the plan:
> > > > >
> > > > >
> > > > > Priorities
> > > > >
> > > > > Consistency, documentation and user experience is the top
> >
> > priority.
> > > > > Compatibility on the command line level.
> > > > > Compatibility on the API level.
> > > > >
> > > > >
> > > > > Maintenance
> > > > >
> > > > > The existing DNF 4 stack stays in the current Fedoras and Red
> >
> > Hat
> > > > > Enterprise Linux 8. We'll keep maintaining it in dnf-4-master
> > > > > branches on GitHub. PackageKit and rpm-ostree will stay on
> >
> > libdnf
> > > > > from the DNF 4 stack.
> > > > >
> > > > >
> > > > > The existing Python API in DNF
> > > > >
> > > > > The Python API in DNF stays. We'll do our best to keep it
> >
> > working.
> > > > > If there is an incompatible change, we'll communicate and
> >
> > document
> > > > > it properly.
> > > > >
> > > > >
> > > > > The new API in libdnf
> > > > >
> > > > > All business logic will move from DNF (Python) to libdnf
> >
> > (C++). This
> > > > > is the only way to ensure that package managers work
> >
> > identically
> > > > > across the whole distribution. We'll start with C++ API and
> > > > > auto-generated Python bindings via SWIG. We'll focus on the
> >
> > Python
> > > > > bindings which are required by DNF and we will do our best to
> > > > > provide bindings for Go, Perl5 and Ruby as well. C API will
> >
> > be
> > > > > created later when the C++ API is stable. At that moment rpm-
> >
> > ostree
> > > > > will be ported to the new C API.
> > > > >
> > > > >
> > > > > hawkey
> > > > >
> > > > > Hawkey Python API is going away and will be replaced with
> >
> > libdnf Python
> > > > > API.
> > > > >
> > > > >
> > > > > DNF
> > > > >
> > > > > DNF stays as the primary command-line package manager. The
> >
> > overall
> > > > > functionality remains. We don't anticipate any negative
> >
> > impact of
> > > > > the API rewrite on the end-users. We have built an extensive
> >
> > test
> > > > > suite (over 1400 scenarios) that will help us to ensure that.
> >
> > The
> > > > > argument parser and outputs may slightly change in some cases
> >
> > to
> > > > > provide a more consistent user-experience. All such cases
> >
> > will be
> > > > > properly documented.
> > > > >
> > > > >
> > > > > microdnf
> > > > >
> > > > > Microdnf is becoming important because it's part of many
> >
> > containers
> > > > > due to its small footprint. We're getting feedback that users
> >
> > would
> > > > > appreciate something closer to DNF. We'll focus on
> >
> > implementing a
> > > > > subset of DNF's functionality and improving the user
> >
> > experience.
> > > > > 100% feature parity with DNF is currently out of scope.
> > > > >
> > > > >
> > > > Hi,
> > > >
> > > > the roadmap is ambitious, but not impossible. Good luck!
> > > >
> > > > > Roadmap (tentative)
> > > > >
> > > > > Mar 2020 - making the bigger API changes, upstream code
> >
> > barely compiles
> > > > > May 2020 - COPR repo with first development snapshots
> > > > > Jun 2020 - F33 module available for early adopters and tool
> >
> > developers
> > > > > Oct 2020 - DNF 5 landing in F34 Rawhide
> > > > > Feb 2021 - DNF 5 replacing DNF 4 in stable Fedora
> > > >
> > > > > DBus service
> > > > >
> > > > > DNF team has decided to create a new DBus service replacing
> > > > > PackageKit to provide an interface to GUI applications. It's
> > > > > probably going to take a while because we're planning to
> >
> > start from
> > > > > scratch.
> > > >
> > > > Do you plan to make normal 'dnf' calls go through the dbus api?
> > >
> > > This would be interesting, but wouldn't that make using DNF in
> >
> > rescue
> > > situations impossible?
> >
> > You mean due to the regular DBus daemon & system + session bus not
> > running
> >
> > in some system rescue scenarios, right ?
> >
> > While possibly a bit tricky I could imagine some fallback mechanism
> > where
> >
> > invocation of the CLI tool starts it's own DBus session & instance
> > of the DNF
> >
> > service when it detects that the regular system & session buses are
> > not available.
> >
> > Anaconda does something similar when it starts in Mock during some
> > phases of the
> >
> > compose process & finds no system bus is available - it starts it's
> > own DBus daemon
> >
> > process and uses that.
>
> Wow, using dbus to communicate between CLI binary and the shared
> library sounds like an awful idea. Why not do a simple shim around
> the shared library instead? (And not introduce another 2 moving parts
> into a critical system component: dnf dbus service and dnf dbus
> client)
>
> Fabio

Aside of others, it will help to solve a dead locks when DNF is
accessing system resources and DB. Right now, Anaconda is launching
DNF/YUM in a separate process because otherwise we will be locked by
Python GIL (Global Interpreter Lock) on every C call in the DNF library
from python. Another reason is to replace PackageKit by something which
is not dead or have a possibility to replace it.
Second benefit is DBus is language agnostic so you could use any
programming language you want to use DNF.
Jirka
> > >
> > > > (And e.g. provide a single cache and privilege escalation
> >
> > through
> > > > packagekit)?
> > > >
> > >
> > > We can do the single cache thing *today* for PackageKit. The APIs
> > > exist in libdnf _now_, it's just that they're not used
> > > PackageKit-side.
> > >
> > > > Apart from the dbus api, do you plan to provide some graphical
> > > > application that uses this api?
> > > >
> > >
> > > I would expect that dnfdragora will be the first consumer of this
> >
> > new
> > > API, since this plan would essentially involve taking over the
> >
> > role of
> > > my dnfdaemon.
> > >
> > > > Are you going to use sd-bus for the dbus library?
> > > >
> > >
> > > I'd hope not, given that we have cross-distro usage of DNF now,
> >
> > and a
> > > couple of them don't have systemd.
> > >
> > >
> > >
> > > --
> > > 真実はいつも一つ！/ Always, there's only one truth!
> > >
> > > devel mailing list -- [e-mail]
> > > To unsubscribe send an email to
> >
> > [e-mail]
> > > Fedora Code of Conduct:
> > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > > List Guidelines:
> >
> > https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > List Archives:
> > >
> >
> > https://lists.fedoraproject.org/archives/list/[e-mail]
> > >
> >
> > devel mailing list -- [e-mail]
> >
> > To unsubscribe send an email to [e-mail]
> >
> > Fedora Code of Conduct: 
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> >
> > List Guidelines: 
> > https://fedoraproject.org/wiki/Mailing_list_guidelines
> >
> > List Archives: 
> > https://lists.fedoraproject.org/archives/list/[e-mail]
>
> _______________________________________________devel mailing list -- 
> [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: 
> https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Fabio Valentini]

On Thu, Mar 5, 2020, 08:56  wrote:
> On Thu, 2020-03-05 at 07:44 +0100, Fabio Valentini wrote:
>
> On Thu, Mar 5, 2020, 00:55 Martin Kolman  wrote:
>
> ----- Original Message -----
> > From: "Neal Gompa" 
> > To: "Development discussions related to Fedora" <
>
> [e-mail]>
> > Sent: Wednesday, March 4, 2020 11:01:43 PM
> > Subject: Re: Announcing start of DNF 5 development
> >
> > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> >  wrote:
> > > On Wed, Mar 04, 2020 at 07:03:01PM +0100, Daniel Mach wrote:
> > > > Hello everyone,
> > > > I'm pleased to announce start of DNF 5 development. We are planning
> > > > to deliver a module stream or a COPR repo during Fedora 33
> > > > development for early adopters and tool developers and we're hoping
> > > > in getting a stable version into Fedora 34.
> > > >
> > > > More details follow.
> > > >
> > > > We've managed to drop a lot of redundant code across the whole DNF
> > > > stack in the past years, but we have reached a point when it's
> > > > nearly impossible to consolidate the code any further without
> > > > breaking the API/ABI. Especially with PackageKit being dead[1], we
> > > > can't move with the old "libhif" API in libdnf, because making any
> > > > bigger changes to PackageKit is clearly out of scope.
> > > >
> > > > [1]
>
> https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-well...
> > > > That's why we decided to start working on a new version of the DNF
> > > > stack: DNF 5. And this is the plan:
> > > > Priorities
> > > > Consistency, documentation and user experience is the top
>
> priority.
> > > > Compatibility on the command line level.
> > > > Compatibility on the API level.
> > > > Maintenance
> > > >
> > > > The existing DNF 4 stack stays in the current Fedoras and Red Hat
> > > > Enterprise Linux 8. We'll keep maintaining it in dnf-4-master
> > > > branches on GitHub. PackageKit and rpm-ostree will stay on libdnf
> > > > from the DNF 4 stack.
> > > > The existing Python API in DNF
> > > >
> > > > The Python API in DNF stays. We'll do our best to keep it working.
> > > > If there is an incompatible change, we'll communicate and document
> > > > it properly.
> > > > The new API in libdnf
> > > >
> > > > All business logic will move from DNF (Python) to libdnf (C++). This
> > > > is the only way to ensure that package managers work identically
> > > > across the whole distribution. We'll start with C++ API and
> > > > auto-generated Python bindings via SWIG. We'll focus on the Python
> > > > bindings which are required by DNF and we will do our best to
> > > > provide bindings for Go, Perl5 and Ruby as well. C API will be
> > > > created later when the C++ API is stable. At that moment rpm-ostree
> > > > will be ported to the new C API.
> > > > hawkey
> > > >
> > > > Hawkey Python API is going away and will be replaced with libdnf
>
> Python
> > > > API.
> > > > DNF
> > > >
> > > > DNF stays as the primary command-line package manager. The overall
> > > > functionality remains. We don't anticipate any negative impact of
> > > > the API rewrite on the end-users. We have built an extensive test
> > > > suite (over 1400 scenarios) that will help us to ensure that. The
> > > > argument parser and outputs may slightly change in some cases to
> > > > provide a more consistent user-experience. All such cases will be
> > > > properly documented.
> > > > microdnf
> > > >
> > > > Microdnf is becoming important because it's part of many containers
> > > > due to its small footprint. We're getting feedback that users would
> > > > appreciate something closer to DNF. We'll focus on implementing a
> > > > subset of DNF's functionality and improving the user experience.
> > > > 100% feature parity with DNF is currently out of scope.
> > >
> > > Hi,
> > >
> > > the roadmap is ambitious, but not impossible. Good luck!
> > > > Roadmap (tentative)
> > > > Mar 2020 - making the bigger API changes, upstream code barely
>
> compiles
> > > > May 2020 - COPR repo with first development snapshots
> > > > Jun 2020 - F33 module available for early adopters and tool
>
> developers
> > > > Oct 2020 - DNF 5 landing in F34 Rawhide
> > > > Feb 2021 - DNF 5 replacing DNF 4 in stable Fedora
> > > > DBus service
> > > >
> > > > DNF team has decided to create a new DBus service replacing
> > > > PackageKit to provide an interface to GUI applications. It's
> > > > probably going to take a while because we're planning to start from
> > > > scratch.
> > >
> > > Do you plan to make normal 'dnf' calls go through the dbus api?
> >
> > This would be interesting, but wouldn't that make using DNF in rescue
> > situations impossible?
>
> You mean due to the regular DBus daemon & system + session bus not running
> in some system rescue scenarios, right ?
>
> While possibly a bit tricky I could imagine some fallback mechanism where
> invocation of the CLI tool starts it's own DBus session & instance of the
> DNF
> service when it detects that the regular system & session buses are not
> available.
>
> Anaconda does something similar when it starts in Mock during some phases
> of the
> compose process & finds no system bus is available - it starts it's own
> DBus daemon
> process and uses that.
>
> Wow, using dbus to communicate between CLI binary and the shared library
> sounds like an awful idea. Why not do a simple shim around the shared
> library instead? (And not introduce another 2 moving parts into a critical
> system component: dnf dbus service and dnf dbus client)
>
> Fabio
>
> Aside of others, it will help to solve a dead locks when DNF is accessing
> system resources and DB. Right now, Anaconda is launching DNF/YUM in a
> separate process because otherwise we will be locked by Python GIL (Global
> Interpreter Lock) on every C call in the DNF library from python. Another
> reason is to replace PackageKit by something which is not dead or have a
> possibility to replace it.
>
> Second benefit is DBus is language agnostic so you could use any
> programming language you want to use DNF.
>
> Jirka

I think my message was a bit ambiguous. I meant to say:

I hope you don't use the dbus daemon to communicate between /usr/bin/dnf
and /usr/lib64/libdnf.so.666, which would only introduce 3 more moving
parts.

As for providing a dbus-based API in addition to the shared library, of
course I'm all for that, for all the reasons you stated in your response.

And in another message in this thread, somebody already clarified that the
dnf binary won't use libdnf over dbus, but link to the shared library
directly. So my concerns seem to be invalid. :)

Fabio
> > > (And e.g. provide a single cache and privilege escalation through
> > > packagekit)?
> >
> > We can do the single cache thing *today* for PackageKit. The APIs
> > exist in libdnf _now_, it's just that they're not used
> > PackageKit-side.
> > > Apart from the dbus api, do you plan to provide some graphical
> > > application that uses this api?
> >
> > I would expect that dnfdragora will be the first consumer of this new
> > API, since this plan would essentially involve taking over the role of
> > my dnfdaemon.
> > > Are you going to use sd-bus for the dbus library?
> >
> > I'd hope not, given that we have cross-distro usage of DNF now, and a
> > couple of them don't have systemd.
> >
> > --
> > 真実はいつも一つ！/ Always, there's only one truth!
> > _______________________________________________
> > devel mailing list -- [e-mail]
> > To unsubscribe send an email to [e-mail]
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
>
> https://lists.fedoraproject.org/archives/list/[e-mail]
> >
>
> devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>
> devel mailing list --
>
> [e-mail]
>
> To unsubscribe send an email to
>
> [e-mail]
>
> Fedora Code of Conduct:
>
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>
> List Guidelines:
>
> https://fedoraproject.org/wiki/Mailing_list_guidelines
>
> List Archives:
>
> https://lists.fedoraproject.org/archives/list/[e-mail]
>
> devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Martin Kolman]

On Thu, 2020-03-05 at 09:11 +0100, Fabio Valentini wrote:
> On Thu, Mar 5, 2020, 08:56   wrote:
> > On Thu, 2020-03-05 at 07:44 +0100, Fabio Valentini wrote:
> > > On Thu, Mar 5, 2020, 00:55 Martin Kolman  wrote:
> > > > ----- Original Message -----
> > > > > From: "Neal Gompa"
> > > > > To: "Development discussions related to Fedora"
> > > > > Sent: Wednesday, March 4, 2020 11:01:43 PM
> > > > > Subject: Re: Announcing start of DNF 5 development
> > > > >
> > > > > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> > > > >  wrote:
> > > > > >
> > > > > > On Wed, Mar 04, 2020 at 07:03:01PM +0100, Daniel Mach wrote:
> > > > > > > Hello everyone,
> > > > > > > I'm pleased to announce start of DNF 5 development. We are planning
> > > > > > > to deliver a module stream or a COPR repo during Fedora 33
> > > > > > > development for early adopters and tool developers and we're hoping
> > > > > > > in getting a stable version into Fedora 34.
> > > > > > >
> > > > > > >
> > > > > > > More details follow.
> > > > > > >
> > > > > > >
> > > > > > > We've managed to drop a lot of redundant code across the whole DNF
> > > > > > > stack in the past years, but we have reached a point when it's
> > > > > > > nearly impossible to consolidate the code any further without
> > > > > > > breaking the API/ABI. Especially with PackageKit being dead[1], we
> > > > > > > can't move with the old "libhif" API in libdnf, because making any
> > > > > > > bigger changes to PackageKit is clearly out of scope.
> > > > > > >
> > > > > > > [1]
> > > > > > > https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-well...
> > > > > > >
> > > > > > >
> > > > > > > That's why we decided to start working on a new version of the DNF
> > > > > > > stack: DNF 5. And this is the plan:
> > > > > > >
> > > > > > >
> > > > > > > Priorities
> > > > > > > ----------
> > > > > > > 1. Consistency, documentation and user experience is the top priority.
> > > > > > > 2. Compatibility on the command line level.
> > > > > > > 3. Compatibility on the API level.
> > > > > > >
> > > > > > >
> > > > > > > Maintenance
> > > > > > > -----------
> > > > > > > The existing DNF 4 stack stays in the current Fedoras and Red Hat
> > > > > > > Enterprise Linux 8. We'll keep maintaining it in dnf-4-master
> > > > > > > branches on GitHub. PackageKit and rpm-ostree will stay on libdnf
> > > > > > > from the DNF 4 stack.
> > > > > > >
> > > > > > >
> > > > > > > The existing Python API in DNF
> > > > > > > ------------------------------
> > > > > > > The Python API in DNF stays. We'll do our best to keep it working.
> > > > > > > If there is an incompatible change, we'll communicate and document
> > > > > > > it properly.
> > > > > > >
> > > > > > >
> > > > > > > The new API in libdnf
> > > > > > > ---------------------
> > > > > > > All business logic will move from DNF (Python) to libdnf (C++). This
> > > > > > > is the only way to ensure that package managers work identically
> > > > > > > across the whole distribution. We'll start with C++ API and
> > > > > > > auto-generated Python bindings via SWIG. We'll focus on the Python
> > > > > > > bindings which are required by DNF and we will do our best to
> > > > > > > provide bindings for Go, Perl5 and Ruby as well. C API will be
> > > > > > > created later when the C++ API is stable. At that moment rpm-ostree
> > > > > > > will be ported to the new C API.
> > > > > > >
> > > > > > >
> > > > > > > hawkey
> > > > > > > ------
> > > > > > > Hawkey Python API is going away and will be replaced with libdnf Python
> > > > > > > API.
> > > > > > >
> > > > > > >
> > > > > > > DNF
> > > > > > > ---
> > > > > > > DNF stays as the primary command-line package manager. The overall
> > > > > > > functionality remains. We don't anticipate any negative impact of
> > > > > > > the API rewrite on the end-users. We have built an extensive test
> > > > > > > suite (over 1400 scenarios) that will help us to ensure that. The
> > > > > > > argument parser and outputs may slightly change in some cases to
> > > > > > > provide a more consistent user-experience. All such cases will be
> > > > > > > properly documented.
> > > > > > >
> > > > > > >
> > > > > > > microdnf
> > > > > > > --------
> > > > > > > Microdnf is becoming important because it's part of many containers
> > > > > > > due to its small footprint. We're getting feedback that users would
> > > > > > > appreciate something closer to DNF. We'll focus on implementing a
> > > > > > > subset of DNF's functionality and improving the user experience.
> > > > > > > 100% feature parity with DNF is currently out of scope.
> > > > > > >
> > > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > the roadmap is ambitious, but not impossible. Good luck!
> > > > > >
> > > > > > > Roadmap (tentative)
> > > > > > > -------------------
> > > > > > > * Mar 2020 - making the bigger API changes, upstream code barely compiles
> > > > > > > * May 2020 - COPR repo with first development snapshots
> > > > > > > * Jun 2020 - F33 module available for early adopters and tool developers
> > > > > > > * Oct 2020 - DNF 5 landing in F34 Rawhide
> > > > > > > * Feb 2021 - DNF 5 replacing DNF 4 in stable Fedora
> > > > > >
> > > > > > > DBus service
> > > > > > > ------------
> > > > > > > DNF team has decided to create a new DBus service replacing
> > > > > > > PackageKit to provide an interface to GUI applications. It's
> > > > > > > probably going to take a while because we're planning to start from
> > > > > > > scratch.
> > > > > >
> > > > > > Do you plan to make normal 'dnf' calls go through the dbus api?
> > > > >
> > > > > This would be interesting, but wouldn't that make using DNF in rescue
> > > > > situations impossible?
> > > >
> > > > You mean due to the regular DBus daemon & system + session bus not running
> > > >
> > > > in some system rescue scenarios, right ?
> > > >
> > > > While possibly a bit tricky I could imagine some fallback mechanism where
> > > >
> > > > invocation of the CLI tool starts it's own DBus session & instance of the DNF
> > > >
> > > > service when it detects that the regular system & session buses are not available.
> > > >
> > > > Anaconda does something similar when it starts in Mock during some phases of the
> > > >
> > > > compose process & finds no system bus is available - it starts it's own DBus daemon
> > > >
> > > > process and uses that.
> > >
> > > Wow, using dbus to communicate between CLI binary and the shared library sounds like an awful idea. Why not do a
> > > simple shim around the shared library instead? (And not introduce another 2 moving parts into a critical system
> > > component: dnf dbus service and dnf dbus client)
> > >
> > > Fabio
> >
> > Aside of others, it will help to solve a dead locks when DNF is accessing system resources and DB. Right now,
> > Anaconda is launching DNF/YUM in a separate process because otherwise we will be locked by Python GIL (Global
> > Interpreter Lock) on every C call in the DNF library from python. Another reason is to replace PackageKit by
> > something which is not dead or have a possibility to replace it.
> >
> > Second benefit is DBus is language agnostic so you could use any programming language you want to use DNF.
> >
> > Jirka
>
> I think my message was a bit ambiguous. I meant to say:
>
> I hope you don't use the dbus daemon to communicate between /usr/bin/dnf and /usr/lib64/libdnf.so.666, which would
> only introduce 3 more moving parts.
>
> As for providing a dbus-based API in addition to the shared library, of course I'm all for that, for all the reasons
> you stated in your response.
>
> And in another message in this thread, somebody already clarified that the dnf binary won't use libdnf over dbus, but
> link to the shared library directly. So my concerns seem to be invalid. :)

Sure, as long as you make sure that operations initiated over DBus do not clash with operations initiated from CLI you
really don't need to have the CLI tool use DBus internally.
> Fabio
> > > > >
> > > > > > (And e.g. provide a single cache and privilege escalation through
> > > > > > packagekit)?
> > > > > >
> > > > >
> > > > > We can do the single cache thing *today* for PackageKit. The APIs
> > > > > exist in libdnf _now_, it's just that they're not used
> > > > > PackageKit-side.
> > > > >
> > > > > > Apart from the dbus api, do you plan to provide some graphical
> > > > > > application that uses this api?
> > > > > >
> > > > >
> > > > > I would expect that dnfdragora will be the first consumer of this new
> > > > > API, since this plan would essentially involve taking over the role of
> > > > > my dnfdaemon.
> > > > >
> > > > > > Are you going to use sd-bus for the dbus library?
> > > > > >
> > > > >
> > > > > I'd hope not, given that we have cross-distro usage of DNF now, and a
> > > > > couple of them don't have systemd.
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > 真実はいつも一つ！/ Always, there's only one truth!
> > > > >
> > > > > devel mailing list -- [e-mail]
> > > > > To unsubscribe send an email to [e-mail]
> > > > > Fedora Code of Conduct:
> > > > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > > > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > > > List Archives:
> > > > > https://lists.fedoraproject.org/archives/list/[e-mail]
> > > > >
> > > >
> > > > devel mailing list -- [e-mail]
> > > >
> > > > To unsubscribe send an email to [e-mail]
> > > >
> > > > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > > >
> > > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > >
> > > > List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]
> > >
> > > _______________________________________________devel mailing list -- [e-mail]
> > > To unsubscribe send an email to [e-mail]
> > > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]
> >
> > devel mailing list -- [e-mail]
> >
> > To unsubscribe send an email to [e-mail]
> >
> > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> >
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> >
> > List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]
>
> _______________________________________________devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]

[Martin Kolman]

On Thu, 2020-03-05 at 08:56 +0100, [e-mail] wrote:
> On Thu, 2020-03-05 at 07:44 +0100, Fabio Valentini wrote:
> > On Thu, Mar 5, 2020, 00:55 Martin Kolman  wrote:
> > > ----- Original Message -----
> > > > From: "Neal Gompa"
> > > > To: "Development discussions related to Fedora"
> > > > Sent: Wednesday, March 4, 2020 11:01:43 PM
> > > > Subject: Re: Announcing start of DNF 5 development
> > > >
> > > > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> > > >  wrote:
> > > > >
> > > > > On Wed, Mar 04, 2020 at 07:03:01PM +0100, Daniel Mach wrote:
> > > > > > Hello everyone,
> > > > > > I'm pleased to announce start of DNF 5 development. We are planning
> > > > > > to deliver a module stream or a COPR repo during Fedora 33
> > > > > > development for early adopters and tool developers and we're hoping
> > > > > > in getting a stable version into Fedora 34.
> > > > > >
> > > > > >
> > > > > > More details follow.
> > > > > >
> > > > > >
> > > > > > We've managed to drop a lot of redundant code across the whole DNF
> > > > > > stack in the past years, but we have reached a point when it's
> > > > > > nearly impossible to consolidate the code any further without
> > > > > > breaking the API/ABI. Especially with PackageKit being dead[1], we
> > > > > > can't move with the old "libhif" API in libdnf, because making any
> > > > > > bigger changes to PackageKit is clearly out of scope.
> > > > > >
> > > > > > [1]
> > > > > > https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-well...
> > > > > >
> > > > > >
> > > > > > That's why we decided to start working on a new version of the DNF
> > > > > > stack: DNF 5. And this is the plan:
> > > > > >
> > > > > >
> > > > > > Priorities
> > > > > >
> > > > > > Consistency, documentation and user experience is the top priority.
> > > > > > Compatibility on the command line level.
> > > > > > Compatibility on the API level.
> > > > > >
> > > > > >
> > > > > > Maintenance
> > > > > >
> > > > > > The existing DNF 4 stack stays in the current Fedoras and Red Hat
> > > > > > Enterprise Linux 8. We'll keep maintaining it in dnf-4-master
> > > > > > branches on GitHub. PackageKit and rpm-ostree will stay on libdnf
> > > > > > from the DNF 4 stack.
> > > > > >
> > > > > >
> > > > > > The existing Python API in DNF
> > > > > >
> > > > > > The Python API in DNF stays. We'll do our best to keep it working.
> > > > > > If there is an incompatible change, we'll communicate and document
> > > > > > it properly.
> > > > > >
> > > > > >
> > > > > > The new API in libdnf
> > > > > >
> > > > > > All business logic will move from DNF (Python) to libdnf (C++). This
> > > > > > is the only way to ensure that package managers work identically
> > > > > > across the whole distribution. We'll start with C++ API and
> > > > > > auto-generated Python bindings via SWIG. We'll focus on the Python
> > > > > > bindings which are required by DNF and we will do our best to
> > > > > > provide bindings for Go, Perl5 and Ruby as well. C API will be
> > > > > > created later when the C++ API is stable. At that moment rpm-ostree
> > > > > > will be ported to the new C API.
> > > > > >
> > > > > >
> > > > > > hawkey
> > > > > >
> > > > > > Hawkey Python API is going away and will be replaced with libdnf Python
> > > > > > API.
> > > > > >
> > > > > >
> > > > > > DNF
> > > > > >
> > > > > > DNF stays as the primary command-line package manager. The overall
> > > > > > functionality remains. We don't anticipate any negative impact of
> > > > > > the API rewrite on the end-users. We have built an extensive test
> > > > > > suite (over 1400 scenarios) that will help us to ensure that. The
> > > > > > argument parser and outputs may slightly change in some cases to
> > > > > > provide a more consistent user-experience. All such cases will be
> > > > > > properly documented.
> > > > > >
> > > > > >
> > > > > > microdnf
> > > > > >
> > > > > > Microdnf is becoming important because it's part of many containers
> > > > > > due to its small footprint. We're getting feedback that users would
> > > > > > appreciate something closer to DNF. We'll focus on implementing a
> > > > > > subset of DNF's functionality and improving the user experience.
> > > > > > 100% feature parity with DNF is currently out of scope.
> > > > > >
> > > > > >
> > > > > Hi,
> > > > >
> > > > > the roadmap is ambitious, but not impossible. Good luck!
> > > > >
> > > > > > Roadmap (tentative)
> > > > > >
> > > > > > Mar 2020 - making the bigger API changes, upstream code barely compiles
> > > > > > May 2020 - COPR repo with first development snapshots
> > > > > > Jun 2020 - F33 module available for early adopters and tool developers
> > > > > > Oct 2020 - DNF 5 landing in F34 Rawhide
> > > > > > Feb 2021 - DNF 5 replacing DNF 4 in stable Fedora
> > > > >
> > > > > > DBus service
> > > > > >
> > > > > > DNF team has decided to create a new DBus service replacing
> > > > > > PackageKit to provide an interface to GUI applications. It's
> > > > > > probably going to take a while because we're planning to start from
> > > > > > scratch.
> > > > >
> > > > > Do you plan to make normal 'dnf' calls go through the dbus api?
> > > >
> > > > This would be interesting, but wouldn't that make using DNF in rescue
> > > > situations impossible?
> > >
> > > You mean due to the regular DBus daemon & system + session bus not running
> > >
> > > in some system rescue scenarios, right ?
> > >
> > > While possibly a bit tricky I could imagine some fallback mechanism where
> > >
> > > invocation of the CLI tool starts it's own DBus session & instance of the DNF
> > >
> > > service when it detects that the regular system & session buses are not available.
> > >
> > > Anaconda does something similar when it starts in Mock during some phases of the
> > >
> > > compose process & finds no system bus is available - it starts it's own DBus daemon
> > >
> > > process and uses that.
> >
> > Wow, using dbus to communicate between CLI binary and the shared library sounds like an awful idea. Why not do a
> > simple shim around the shared library instead? (And not introduce another 2 moving parts into a critical system
> > component: dnf dbus service and dnf dbus client)
> >
> > Fabio
>
> Aside of others, it will help to solve a dead locks when DNF is accessing system resources and DB. Right now, Anaconda
> is launching DNF/YUM in a separate process because otherwise we will be locked by Python GIL (Global Interpreter Lock)
> on every C call in the DNF library from python.

I'll add one more reason why we are running DNF in a separate process during the installation - RPM apparently chroots
its process intermittently when it installs packages. 
This is not really a problem when you use DNF over CLI - it already running in a separate process. But when you use the
DNF Python API & RPM does it's chroot dance, it would effectively chroot the Anaconda process into the generally almost
empty at that time target system chroot. 
Let's say GTK and other libraries do not play well with the files they want to read or write suddenly vanishing in split
second only to show up back a second later, repeatedly. This effectively demonstrated as pretty strange deadlocks,
randomly missing icons or even crashes before we tracked it down to the likely unlikely culprit of RPM.
So at the moment we use the DNF Python API from a Python sub-process, but that's really hard from ideal, even just
because I'm pretty sure we are likely the only users of the DNF Python API doing so. 
Really looking forward to the planed DNF DBus API we could use instead, like we already do with other system-level
services and tooling, such as Network Manager.
> Another reason is to replace PackageKit by something which is not dead or have a possibility to replace it.
>
> Second benefit is DBus is language agnostic so you could use any programming language you want to use DNF.
>
> Jirka
> > > >
> > > > > (And e.g. provide a single cache and privilege escalation through
> > > > > packagekit)?
> > > > >
> > > >
> > > > We can do the single cache thing *today* for PackageKit. The APIs
> > > > exist in libdnf _now_, it's just that they're not used
> > > > PackageKit-side.
> > > >
> > > > > Apart from the dbus api, do you plan to provide some graphical
> > > > > application that uses this api?
> > > > >
> > > >
> > > > I would expect that dnfdragora will be the first consumer of this new
> > > > API, since this plan would essentially involve taking over the role of
> > > > my dnfdaemon.
> > > >
> > > > > Are you going to use sd-bus for the dbus library?
> > > > >
> > > >
> > > > I'd hope not, given that we have cross-distro usage of DNF now, and a
> > > > couple of them don't have systemd.
> > > >
> > > >
> > > >
> > > > --
> > > > 真実はいつも一つ！/ Always, there's only one truth!
> > > >
> > > > devel mailing list -- [e-mail]
> > > > To unsubscribe send an email to [e-mail]
> > > > Fedora Code of Conduct:
> > > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > > List Archives:
> > > > https://lists.fedoraproject.org/archives/list/[e-mail]
> > > >
> > >
> > > devel mailing list -- [e-mail]
> > >
> > > To unsubscribe send an email to [e-mail]
> > >
> > > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > >
> > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > >
> > > List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]
> >
> > _______________________________________________devel mailing list -- [e-mail]
> > To unsubscribe send an email to [e-mail]
> > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]
>
> _______________________________________________devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]

[Neal Gompa]

On Thu, Mar 5, 2020 at 5:58 AM Martin Kolman  wrote:
> On Thu, 2020-03-05 at 08:56 +0100, [e-mail] wrote:
>
> On Thu, 2020-03-05 at 07:44 +0100, Fabio Valentini wrote:
>
> On Thu, Mar 5, 2020, 00:55 Martin Kolman  wrote:
>
> ----- Original Message -----
> > From: "Neal Gompa" 
> > To: "Development discussions related to Fedora" 
> > Sent: Wednesday, March 4, 2020 11:01:43 PM
> > Subject: Re: Announcing start of DNF 5 development
> >
> > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> >  wrote:
> > > On Wed, Mar 04, 2020 at 07:03:01PM +0100, Daniel Mach wrote:
> > > > Hello everyone,
> > > > I'm pleased to announce start of DNF 5 development. We are planning
> > > > to deliver a module stream or a COPR repo during Fedora 33
> > > > development for early adopters and tool developers and we're hoping
> > > > in getting a stable version into Fedora 34.
> > > >
> > > > More details follow.
> > > >
> > > > We've managed to drop a lot of redundant code across the whole DNF
> > > > stack in the past years, but we have reached a point when it's
> > > > nearly impossible to consolidate the code any further without
> > > > breaking the API/ABI. Especially with PackageKit being dead[1], we
> > > > can't move with the old "libhif" API in libdnf, because making any
> > > > bigger changes to PackageKit is clearly out of scope.
> > > >
> > > > [1]
> > > > https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-well...
> > > >
> > > > That's why we decided to start working on a new version of the DNF
> > > > stack: DNF 5. And this is the plan:
> > > > Priorities
> > > > Consistency, documentation and user experience is the top priority.
> > > > Compatibility on the command line level.
> > > > Compatibility on the API level.
> > > > Maintenance
> > > >
> > > > The existing DNF 4 stack stays in the current Fedoras and Red Hat
> > > > Enterprise Linux 8. We'll keep maintaining it in dnf-4-master
> > > > branches on GitHub. PackageKit and rpm-ostree will stay on libdnf
> > > > from the DNF 4 stack.
> > > > The existing Python API in DNF
> > > >
> > > > The Python API in DNF stays. We'll do our best to keep it working.
> > > > If there is an incompatible change, we'll communicate and document
> > > > it properly.
> > > > The new API in libdnf
> > > >
> > > > All business logic will move from DNF (Python) to libdnf (C++). This
> > > > is the only way to ensure that package managers work identically
> > > > across the whole distribution. We'll start with C++ API and
> > > > auto-generated Python bindings via SWIG. We'll focus on the Python
> > > > bindings which are required by DNF and we will do our best to
> > > > provide bindings for Go, Perl5 and Ruby as well. C API will be
> > > > created later when the C++ API is stable. At that moment rpm-ostree
> > > > will be ported to the new C API.
> > > > hawkey
> > > >
> > > > Hawkey Python API is going away and will be replaced with libdnf Python
> > > > API.
> > > > DNF
> > > >
> > > > DNF stays as the primary command-line package manager. The overall
> > > > functionality remains. We don't anticipate any negative impact of
> > > > the API rewrite on the end-users. We have built an extensive test
> > > > suite (over 1400 scenarios) that will help us to ensure that. The
> > > > argument parser and outputs may slightly change in some cases to
> > > > provide a more consistent user-experience. All such cases will be
> > > > properly documented.
> > > > microdnf
> > > >
> > > > Microdnf is becoming important because it's part of many containers
> > > > due to its small footprint. We're getting feedback that users would
> > > > appreciate something closer to DNF. We'll focus on implementing a
> > > > subset of DNF's functionality and improving the user experience.
> > > > 100% feature parity with DNF is currently out of scope.
> > >
> > > Hi,
> > >
> > > the roadmap is ambitious, but not impossible. Good luck!
> > > > Roadmap (tentative)
> > > > Mar 2020 - making the bigger API changes, upstream code barely compiles
> > > > May 2020 - COPR repo with first development snapshots
> > > > Jun 2020 - F33 module available for early adopters and tool developers
> > > > Oct 2020 - DNF 5 landing in F34 Rawhide
> > > > Feb 2021 - DNF 5 replacing DNF 4 in stable Fedora
> > > > DBus service
> > > >
> > > > DNF team has decided to create a new DBus service replacing
> > > > PackageKit to provide an interface to GUI applications. It's
> > > > probably going to take a while because we're planning to start from
> > > > scratch.
> > >
> > > Do you plan to make normal 'dnf' calls go through the dbus api?
> >
> > This would be interesting, but wouldn't that make using DNF in rescue
> > situations impossible?
>
> You mean due to the regular DBus daemon & system + session bus not running
> in some system rescue scenarios, right ?
>
> While possibly a bit tricky I could imagine some fallback mechanism where
> invocation of the CLI tool starts it's own DBus session & instance of the DNF
> service when it detects that the regular system & session buses are not available.
>
> Anaconda does something similar when it starts in Mock during some phases of the
> compose process & finds no system bus is available - it starts it's own DBus daemon
> process and uses that.
>
> Wow, using dbus to communicate between CLI binary and the shared library sounds like an awful idea. Why not do a simple shim around the shared library instead? (And not introduce another 2 moving parts into a critical system component: dnf dbus service and dnf dbus client)
>
> Fabio
>
> Aside of others, it will help to solve a dead locks when DNF is accessing system resources and DB. Right now, Anaconda is launching DNF/YUM in a separate process because otherwise we will be locked by Python GIL (Global Interpreter Lock) on every C call in the DNF library from python.
>
> I'll add one more reason why we are running DNF in a separate process during the installation - RPM apparently chroots its process intermittently when it installs packages.
>
> This is not really a problem when you use DNF over CLI - it already running in a separate process. But when you use the DNF Python API & RPM does it's chroot dance, it would effectively chroot the Anaconda process into the generally almost empty at that time target system chroot.
>
> Let's say GTK and other libraries do not play well with the files they want to read or write suddenly vanishing in split second only to show up back a second later, repeatedly. This effectively demonstrated as pretty strange deadlocks, randomly missing icons or even crashes before we tracked it down to the likely unlikely culprit of RPM.
>
> So at the moment we use the DNF Python API from a Python sub-process, but that's really hard from ideal, even just because I'm pretty sure we are likely the only users of the DNF Python API doing so.
>
> Really looking forward to the planed DNF DBus API we could use instead, like we already do with other system-level services and tooling, such as Network Manager.
>
> Another reason is to replace PackageKit by something which is not dead or have a possibility to replace it.
>
> Second benefit is DBus is language agnostic so you could use any programming language you want to use DNF.

Considering dnfdaemon[1] has existed for almost five years and
maintained by the ManaTools Project for three, I'm amazed that nobody
thought to contribute to that project to support Anaconda's use-case.
The only thing we don't have is an API to initialize the transaction
for an install root. The rest of the APIs we currently have would be
sufficient for Anaconda's use-case.

As for the whole "PackageKit is dead", I am not of the opinion that it
is, especially given the work done to switch things over to the
PackageKit abstraction in a lot of upstream software. Without
PackageKit, we have _nothing_ for generically handling package
management actions. I think it's a huge mistake to give that up.

[1]: https://github.com/manatools/dnfdaemon

--
真実はいつも一つ！/ Always, there's only one truth!

On Wed, 2020-03-04 at 18:55 -0500, Martin Kolman wrote:
> ----- Original Message -----
> > From: "Neal Gompa" 
> > To: "Development discussions related to Fedora" <
> > [e-mail]>
> > Sent: Wednesday, March 4, 2020 11:01:43 PM
> > Subject: Re: Announcing start of DNF 5 development
> >
> > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> >  wrote:
> > > On Wed, Mar 04, 2020 at 07:03:01PM +0100, Daniel Mach wrote:
> > > > Hello everyone,
> > > > I'm pleased to announce start of DNF 5 development. We are
> > > > planning
> > > > to deliver a module stream or a COPR repo during Fedora 33
> > > > development for early adopters and tool developers and we're
> > > > hoping
> > > > in getting a stable version into Fedora 34.
> > > >
> > > > More details follow.
> > > >
> > > > We've managed to drop a lot of redundant code across the whole
> > > > DNF
> > > > stack in the past years, but we have reached a point when it's
> > > > nearly impossible to consolidate the code any further without
> > > > breaking the API/ABI. Especially with PackageKit being dead[1],
> > > > we
> > > > can't move with the old "libhif" API in libdnf, because making
> > > > any
> > > > bigger changes to PackageKit is clearly out of scope.
> > > >
> > > > [1]
> > > > https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-well...
> > > >
> > > > That's why we decided to start working on a new version of the
> > > > DNF
> > > > stack: DNF 5. And this is the plan:
> > > > Priorities
> > > > Consistency, documentation and user experience is the top
> > > >
> > > > priority.
> > > > 2. Compatibility on the command line level.
> > > > 3. Compatibility on the API level.
> > > > Maintenance
> > > >
> > > > The existing DNF 4 stack stays in the current Fedoras and Red
> > > > Hat
> > > > Enterprise Linux 8. We'll keep maintaining it in dnf-4-master
> > > > branches on GitHub. PackageKit and rpm-ostree will stay on
> > > > libdnf
> > > > from the DNF 4 stack.
> > > > The existing Python API in DNF
> > > >
> > > > The Python API in DNF stays. We'll do our best to keep it
> > > > working.
> > > > If there is an incompatible change, we'll communicate and
> > > > document
> > > > it properly.
> > > > The new API in libdnf
> > > >
> > > > All business logic will move from DNF (Python) to libdnf (C++).
> > > > This
> > > > is the only way to ensure that package managers work
> > > > identically
> > > > across the whole distribution. We'll start with C++ API and
> > > > auto-generated Python bindings via SWIG. We'll focus on the
> > > > Python
> > > > bindings which are required by DNF and we will do our best to
> > > > provide bindings for Go, Perl5 and Ruby as well. C API will be
> > > > created later when the C++ API is stable. At that moment rpm-
> > > > ostree
> > > > will be ported to the new C API.
> > > > hawkey
> > > >
> > > > Hawkey Python API is going away and will be replaced with
> > > > libdnf Python
> > > > API.
> > > > DNF
> > > >
> > > > DNF stays as the primary command-line package manager. The
> > > > overall
> > > > functionality remains. We don't anticipate any negative impact
> > > > of
> > > > the API rewrite on the end-users. We have built an extensive
> > > > test
> > > > suite (over 1400 scenarios) that will help us to ensure that.
> > > > The
> > > > argument parser and outputs may slightly change in some cases
> > > > to
> > > > provide a more consistent user-experience. All such cases will
> > > > be
> > > > properly documented.
> > > > microdnf
> > > >
> > > > Microdnf is becoming important because it's part of many
> > > > containers
> > > > due to its small footprint. We're getting feedback that users
> > > > would
> > > > appreciate something closer to DNF. We'll focus on implementing
> > > > a
> > > > subset of DNF's functionality and improving the user
> > > > experience.
> > > > 100% feature parity with DNF is currently out of scope.
> > >
> > > Hi,
> > >
> > > the roadmap is ambitious, but not impossible. Good luck!
> > > > Roadmap (tentative)
> > > > Mar 2020 - making the bigger API changes, upstream code
> > > >
> > > > barely compiles
> > > > May 2020 - COPR repo with first development snapshots
> > > > Jun 2020 - F33 module available for early adopters and tool
> > > >
> > > > developers
> > > > Oct 2020 - DNF 5 landing in F34 Rawhide
> > > > Feb 2021 - DNF 5 replacing DNF 4 in stable Fedora
> > > > DBus service
> > > >
> > > > DNF team has decided to create a new DBus service replacing
> > > > PackageKit to provide an interface to GUI applications. It's
> > > > probably going to take a while because we're planning to start
> > > > from
> > > > scratch.
> > >
> > > Do you plan to make normal 'dnf' calls go through the dbus api?
> >
> > This would be interesting, but wouldn't that make using DNF in
> > rescue
> > situations impossible?
>
> You mean due to the regular DBus daemon & system + session bus not
> running
> in some system rescue scenarios, right ?
>
> While possibly a bit tricky I could imagine some fallback mechanism
> where
> invocation of the CLI tool starts it's own DBus session & instance of
> the DNF
> service when it detects that the regular system & session buses are
> not available.
>
> Anaconda does something similar when it starts in Mock during some
> phases of the
> compose process & finds no system bus is available - it starts it's
> own DBus daemon
> process and uses that.

That is not completely correct. Anaconda is always starting it's own
dbus session. Anaconda wants to have a separated dbus session because
there is no benefit of being on the existing ones and this way we can
tweak session settings.

But the above is of course not true for DNF. You want to see this
service on system bus to make it easier to be discovered.
> > > (And e.g. provide a single cache and privilege escalation through
> > > packagekit)?
> >
> > We can do the single cache thing *today* for PackageKit. The APIs
> > exist in libdnf _now_, it's just that they're not used
> > PackageKit-side.
> > > Apart from the dbus api, do you plan to provide some graphical
> > > application that uses this api?
> >
> > I would expect that dnfdragora will be the first consumer of this
> > new
> > API, since this plan would essentially involve taking over the role
> > of
> > my dnfdaemon.
> > > Are you going to use sd-bus for the dbus library?
> >
> > I'd hope not, given that we have cross-distro usage of DNF now, and
> > a
> > couple of them don't have systemd.
> >
> > --
> > 真実はいつも一つ！/ Always, there's only one truth!
> > _______________________________________________
> > devel mailing list -- [e-mail]
> > To unsubscribe send an email to [e-mail]
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: 
> > https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> > https://lists.fedoraproject.org/archives/list/[e-mail]
>
> devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: 
> https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Daniel Mach]

Dne 04. 03. 20 v 23:01 Neal Gompa napsal(a):
> On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
>  wrote:
> > Are you going to use sd-bus for the dbus library?
>
> I'd hope not, given that we have cross-distro usage of DNF now, and a
> couple of them don't have systemd.

Do you know which distros do not have systemd?

We have evaluated sd-bus to be the best dbus client available, but we 
may have underestimated it's adoption.

Couldn't systemd team split it into a separate library independent on 
the rest of the systemd eco-system? :)

[Neal Gompa]

On Thu, Mar 5, 2020 at 8:29 AM Daniel Mach  wrote:
> Dne 04. 03. 20 v 23:01 Neal Gompa napsal(a):
> > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> >  wrote:
> > > Are you going to use sd-bus for the dbus library?
> >
> > I'd hope not, given that we have cross-distro usage of DNF now, and a
> > couple of them don't have systemd.
>
> Do you know which distros do not have systemd?
>
> We have evaluated sd-bus to be the best dbus client available, but we
> may have underestimated it's adoption.
>
> Couldn't systemd team split it into a separate library independent on
> the rest of the systemd eco-system? :)

Off the top of my head: PLD and ALT do not mandate systemd usage, and
their default setup is still sysvinit.

PCLinuxOS also does not have systemd and does not currently intend on
including it.

Also, I can't use systemd stuff on darwin-based systems (macOS and
PureDarwin), even though that may not matter to you. :)

[Josh Boyer]

On Thu, Mar 5, 2020 at 8:35 AM Neal Gompa  wrote:
> On Thu, Mar 5, 2020 at 8:29 AM Daniel Mach  wrote:
> > Dne 04. 03. 20 v 23:01 Neal Gompa napsal(a):
> > > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> > >  wrote:
> > > > Are you going to use sd-bus for the dbus library?
> > >
> > > I'd hope not, given that we have cross-distro usage of DNF now, and a
> > > couple of them don't have systemd.
> >
> > Do you know which distros do not have systemd?
> >
> > We have evaluated sd-bus to be the best dbus client available, but we
> > may have underestimated it's adoption.
> >
> > Couldn't systemd team split it into a separate library independent on
> > the rest of the systemd eco-system? :)
>
> Off the top of my head: PLD and ALT do not mandate systemd usage, and
> their default setup is still sysvinit.
>
> PCLinuxOS also does not have systemd and does not currently intend on
> including it.

If I were weighing the value of having dnf work on those distributions
vs. building out a more robust and maintainable dnf for *most*
distributions, I would choose the later.  My point here is that
cross-distro compatibility is important, but it is not the most
important thing.

josh
> Also, I can't use systemd stuff on darwin-based systems (macOS and
> PureDarwin), even though that may not matter to you. :)
>
> --
> 真実はいつも一つ！/ Always, there's only one truth!
> _______________________________________________
> devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]

[Neal Gompa]

On Thu, Mar 5, 2020 at 9:02 AM Josh Boyer  wrote:
> On Thu, Mar 5, 2020 at 8:35 AM Neal Gompa  wrote:
> > On Thu, Mar 5, 2020 at 8:29 AM Daniel Mach  wrote:
> > > Dne 04. 03. 20 v 23:01 Neal Gompa napsal(a):
> > > > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> > > >  wrote:
> > > > > Are you going to use sd-bus for the dbus library?
> > > >
> > > > I'd hope not, given that we have cross-distro usage of DNF now, and a
> > > > couple of them don't have systemd.
> > >
> > > Do you know which distros do not have systemd?
> > >
> > > We have evaluated sd-bus to be the best dbus client available, but we
> > > may have underestimated it's adoption.
> > >
> > > Couldn't systemd team split it into a separate library independent on
> > > the rest of the systemd eco-system? :)
> >
> > Off the top of my head: PLD and ALT do not mandate systemd usage, and
> > their default setup is still sysvinit.
> >
> > PCLinuxOS also does not have systemd and does not currently intend on
> > including it.
>
> If I were weighing the value of having dnf work on those distributions
> vs. building out a more robust and maintainable dnf for *most*
> distributions, I would choose the later.  My point here is that
> cross-distro compatibility is important, but it is not the most
> important thing.

Perhaps, but no code has been written yet, so this decision isn't set
in stone. This is the perfect time to make sure we don't trap
ourselves.

And I also forgot all about Yocto, which switched to DNF back in Yocto
2.3. They do not use systemd by default in a lot of cases, and having
DNF work there without systemd is valuable because of all the
configurations they offer that systemd does not support.

[Zbigniew Jędrzejewski-Szmek]

On Thu, Mar 05, 2020 at 02:29:38PM +0100, Daniel Mach wrote:
> Dne 04. 03. 20 v 23:01 Neal Gompa napsal(a):
> > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> >  wrote:
> > > Are you going to use sd-bus for the dbus library?
> >
> > I'd hope not, given that we have cross-distro usage of DNF now, and a
> > couple of them don't have systemd.
>
> Do you know which distros do not have systemd?
>
> We have evaluated sd-bus to be the best dbus client available, but
> we may have underestimated it's adoption.
>
> Couldn't systemd team split it into a separate library independent
> on the rest of the systemd eco-system? :)

It's already a separate library independent from the rest of the
systemd eco-system ;)

(OK, it's a single library with other parts included, but the sd-bus
part is the biggest by far, and also sd-bus uses sd-event and sd-128
internally, so they'd be required anyway. The other parts are smaller,
and matter more because of dependencies they pull in than because of
the code size itself. But it is possible to compile libsystemd w/o
some features, notably without encryption, to save size and deps.
I don't think there's any significant barrier to using libsystemd
on non-systemd systems, except for psychological issues.)

Zbyszek

[Neal Gompa]

On Thu, Mar 5, 2020 at 9:00 AM Zbigniew Jędrzejewski-Szmek
 wrote:
> On Thu, Mar 05, 2020 at 02:29:38PM +0100, Daniel Mach wrote:
> > Dne 04. 03. 20 v 23:01 Neal Gompa napsal(a):
> > > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> > >  wrote:
> > > > Are you going to use sd-bus for the dbus library?
> > >
> > > I'd hope not, given that we have cross-distro usage of DNF now, and a
> > > couple of them don't have systemd.
> >
> > Do you know which distros do not have systemd?
> >
> > We have evaluated sd-bus to be the best dbus client available, but
> > we may have underestimated it's adoption.
> >
> > Couldn't systemd team split it into a separate library independent
> > on the rest of the systemd eco-system? :)
>
> It's already a separate library independent from the rest of the
> systemd eco-system ;)
>
> (OK, it's a single library with other parts included, but the sd-bus
> part is the biggest by far, and also sd-bus uses sd-event and sd-128
> internally, so they'd be required anyway. The other parts are smaller,
> and matter more because of dependencies they pull in than because of
> the code size itself. But it is possible to compile libsystemd w/o
> some features, notably without encryption, to save size and deps.
> I don't think there's any significant barrier to using libsystemd
> on non-systemd systems, except for psychological issues.)

Don't discount psychological issues. The original reason for DNF not
being named YUM was so that the project would be free to do better
without the YUM baggage. That had the added effect of making it easier
to propose for other distributions to adopt because YUM has a very
negative reputation outside of the RHEL space.

[Zbigniew Jędrzejewski-Szmek]

On Thu, Mar 05, 2020 at 09:02:47AM -0500, Neal Gompa wrote:
> On Thu, Mar 5, 2020 at 9:00 AM Zbigniew Jędrzejewski-Szmek
>  wrote:
> > On Thu, Mar 05, 2020 at 02:29:38PM +0100, Daniel Mach wrote:
> > > Dne 04. 03. 20 v 23:01 Neal Gompa napsal(a):
> > > > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> > > >  wrote:
> > > > > Are you going to use sd-bus for the dbus library?
> > > >
> > > > I'd hope not, given that we have cross-distro usage of DNF now, and a
> > > > couple of them don't have systemd.
> > >
> > > Do you know which distros do not have systemd?
> > >
> > > We have evaluated sd-bus to be the best dbus client available, but
> > > we may have underestimated it's adoption.
> > >
> > > Couldn't systemd team split it into a separate library independent
> > > on the rest of the systemd eco-system? :)
> >
> > It's already a separate library independent from the rest of the
> > systemd eco-system ;)
> >
> > (OK, it's a single library with other parts included, but the sd-bus
> > part is the biggest by far, and also sd-bus uses sd-event and sd-128
> > internally, so they'd be required anyway. The other parts are smaller,
> > and matter more because of dependencies they pull in than because of
> > the code size itself. But it is possible to compile libsystemd w/o
> > some features, notably without encryption, to save size and deps.
> > I don't think there's any significant barrier to using libsystemd
> > on non-systemd systems, except for psychological issues.)
>
> Don't discount psychological issues. The original reason for DNF not
> being named YUM was so that the project would be free to do better
> without the YUM baggage. That had the added effect of making it easier
> to propose for other distributions to adopt because YUM has a very
> negative reputation outside of the RHEL space.

;]]] Let's just hope that the underlying ipc library is something that
is not visible to users ;)

Zbyszek

[Neal Gompa]

On Thu, Mar 5, 2020 at 9:09 AM Zbigniew Jędrzejewski-Szmek
 wrote:
> On Thu, Mar 05, 2020 at 09:02:47AM -0500, Neal Gompa wrote:
> > On Thu, Mar 5, 2020 at 9:00 AM Zbigniew Jędrzejewski-Szmek
> >  wrote:
> > > On Thu, Mar 05, 2020 at 02:29:38PM +0100, Daniel Mach wrote:
> > > > Dne 04. 03. 20 v 23:01 Neal Gompa napsal(a):
> > > > > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> > > > >  wrote:
> > > > > > Are you going to use sd-bus for the dbus library?
> > > > >
> > > > > I'd hope not, given that we have cross-distro usage of DNF now, and a
> > > > > couple of them don't have systemd.
> > > >
> > > > Do you know which distros do not have systemd?
> > > >
> > > > We have evaluated sd-bus to be the best dbus client available, but
> > > > we may have underestimated it's adoption.
> > > >
> > > > Couldn't systemd team split it into a separate library independent
> > > > on the rest of the systemd eco-system? :)
> > >
> > > It's already a separate library independent from the rest of the
> > > systemd eco-system ;)
> > >
> > > (OK, it's a single library with other parts included, but the sd-bus
> > > part is the biggest by far, and also sd-bus uses sd-event and sd-128
> > > internally, so they'd be required anyway. The other parts are smaller,
> > > and matter more because of dependencies they pull in than because of
> > > the code size itself. But it is possible to compile libsystemd w/o
> > > some features, notably without encryption, to save size and deps.
> > > I don't think there's any significant barrier to using libsystemd
> > > on non-systemd systems, except for psychological issues.)
> >
> > Don't discount psychological issues. The original reason for DNF not
> > being named YUM was so that the project would be free to do better
> > without the YUM baggage. That had the added effect of making it easier
> > to propose for other distributions to adopt because YUM has a very
> > negative reputation outside of the RHEL space.
>
> ;]]] Let's just hope that the underlying ipc library is something that
> is not visible to users ;)

The BuildRequires: systemd-devel generally gives it away, and the
failure to build because of that missing dependency is usually a
problem.

[Adam Williamson]

On Thu, 2020-03-05 at 14:08 +0000, Zbigniew Jędrzejewski-Szmek wrote:
> > > (OK, it's a single library with other parts included, but the sd-bus
> > > part is the biggest by far, and also sd-bus uses sd-event and sd-128
> > > internally, so they'd be required anyway. The other parts are smaller,
> > > and matter more because of dependencies they pull in than because of
> > > the code size itself. But it is possible to compile libsystemd w/o
> > > some features, notably without encryption, to save size and deps.
> > > I don't think there's any significant barrier to using libsystemd
> > > on non-systemd systems, except for psychological issues.)
> >
> > Don't discount psychological issues. The original reason for DNF not
> > being named YUM was so that the project would be free to do better
> > without the YUM baggage. That had the added effect of making it easier
> > to propose for other distributions to adopt because YUM has a very
> > negative reputation outside of the RHEL space.
>
> ;]]] Let's just hope that the underlying ipc library is something that
> is not visible to users ;)

pull-request: s/libsystemd/libvoldemort/g/

[Chris Murphy]

On Thu, Mar 5, 2020 at 8:30 AM Daniel Mach  wrote:
> Dne 04. 03. 20 v 23:01 Neal Gompa napsal(a):
> > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> >  wrote:
> > > Are you going to use sd-bus for the dbus library?
> >
> > I'd hope not, given that we have cross-distro usage of DNF now, and a
> > couple of them don't have systemd.
>
> Do you know which distros do not have systemd?
>
> We have evaluated sd-bus to be the best dbus client available, but we
> may have underestimated it's adoption.
>
> Couldn't systemd team split it into a separate library independent on
> the rest of the systemd eco-system? :)

Is varlink applicable here? I mention it because I've come across it
being used by systemd-homed. It's intended to be available during
early boot.

[Daniel Mach]

Dne 12. 03. 20 v 6:21 Chris Murphy napsal(a):
> On Thu, Mar 5, 2020 at 8:30 AM Daniel Mach  wrote:
> > Dne 04. 03. 20 v 23:01 Neal Gompa napsal(a):
> > > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> > >  wrote:
> > > > Are you going to use sd-bus for the dbus library?
> > >
> > > I'd hope not, given that we have cross-distro usage of DNF now, and a
> > > couple of them don't have systemd.
> >
> > Do you know which distros do not have systemd?
> >
> > We have evaluated sd-bus to be the best dbus client available, but we
> > may have underestimated it's adoption.
> >
> > Couldn't systemd team split it into a separate library independent on
> > the rest of the systemd eco-system? :)
>
> Is varlink applicable here? I mention it because I've come across it
> being used by systemd-homed. It's intended to be available during
> early boot.

We have considered both dbus and varlink and decided to use dbus. Dbus 
has better adoption and we'll be replacing a dbus service (PackageKit) 
with another.

[Zbigniew Jędrzejewski-Szmek]

On Thu, Mar 12, 2020 at 08:22:48AM +0100, Daniel Mach wrote:
> Dne 12. 03. 20 v 6:21 Chris Murphy napsal(a):
> > On Thu, Mar 5, 2020 at 8:30 AM Daniel Mach  wrote:
> > > Dne 04. 03. 20 v 23:01 Neal Gompa napsal(a):
> > > > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
> > > >  wrote:
> > > > > Are you going to use sd-bus for the dbus library?
> > > >
> > > > I'd hope not, given that we have cross-distro usage of DNF now, and a
> > > > couple of them don't have systemd.
> > >
> > > Do you know which distros do not have systemd?
> > >
> > > We have evaluated sd-bus to be the best dbus client available, but we
> > > may have underestimated it's adoption.
> > >
> > > Couldn't systemd team split it into a separate library independent on
> > > the rest of the systemd eco-system? :)
> >
> > Is varlink applicable here? I mention it because I've come across it
> > being used by systemd-homed. It's intended to be available during
> > early boot.
>
> We have considered both dbus and varlink and decided to use dbus.
> Dbus has better adoption and we'll be replacing a dbus service
> (PackageKit) with another.

varlink is useful for cases where dbus is not available — early boot,
or as part of the supporting infra for the dbus broker daemon itself.
dbus is the preferred solution for the case where we want to expose a
service to many client, have unprivileged clients communicating with a
system daemon, and possibly do privilege escalation using polkit.
dbus seems very much to be the right choice for dnf.

Zbyszek

[Daniel Mach]

Dne 04. 03. 20 v 22:34 Zbigniew Jędrzejewski-Szmek napsal(a):
> On Wed, Mar 04, 2020 at 07:03:01PM +0100, Daniel Mach wrote:
> > Hello everyone,
> > I'm pleased to announce start of DNF 5 development. We are planning
> > to deliver a module stream or a COPR repo during Fedora 33
> > development for early adopters and tool developers and we're hoping
> > in getting a stable version into Fedora 34.
> >
> > More details follow.
> >
> > We've managed to drop a lot of redundant code across the whole DNF
> > stack in the past years, but we have reached a point when it's
> > nearly impossible to consolidate the code any further without
> > breaking the API/ABI. Especially with PackageKit being dead[1], we
> > can't move with the old "libhif" API in libdnf, because making any
> > bigger changes to PackageKit is clearly out of scope.
> >
> > [1] https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-well...
> >
> > That's why we decided to start working on a new version of the DNF
> > stack: DNF 5. And this is the plan:
> > Priorities
> > Consistency, documentation and user experience is the top priority.
> > Compatibility on the command line level.
> > Compatibility on the API level.
> > Maintenance
> >
> > The existing DNF 4 stack stays in the current Fedoras and Red Hat
> > Enterprise Linux 8. We'll keep maintaining it in dnf-4-master
> > branches on GitHub. PackageKit and rpm-ostree will stay on libdnf
> > from the DNF 4 stack.
> > The existing Python API in DNF
> >
> > The Python API in DNF stays. We'll do our best to keep it working.
> > If there is an incompatible change, we'll communicate and document
> > it properly.
> > The new API in libdnf
> >
> > All business logic will move from DNF (Python) to libdnf (C++). This
> > is the only way to ensure that package managers work identically
> > across the whole distribution. We'll start with C++ API and
> > auto-generated Python bindings via SWIG. We'll focus on the Python
> > bindings which are required by DNF and we will do our best to
> > provide bindings for Go, Perl5 and Ruby as well. C API will be
> > created later when the C++ API is stable. At that moment rpm-ostree
> > will be ported to the new C API.
> > hawkey
> >
> > Hawkey Python API is going away and will be replaced with libdnf Python API.
> > DNF
> >
> > DNF stays as the primary command-line package manager. The overall
> > functionality remains. We don't anticipate any negative impact of
> > the API rewrite on the end-users. We have built an extensive test
> > suite (over 1400 scenarios) that will help us to ensure that. The
> > argument parser and outputs may slightly change in some cases to
> > provide a more consistent user-experience. All such cases will be
> > properly documented.
> > microdnf
> >
> > Microdnf is becoming important because it's part of many containers
> > due to its small footprint. We're getting feedback that users would
> > appreciate something closer to DNF. We'll focus on implementing a
> > subset of DNF's functionality and improving the user experience.
> > 100% feature parity with DNF is currently out of scope.
>
> Hi,
>
> the roadmap is ambitious, but not impossible. Good luck!

Thanks!
> > Roadmap (tentative)
> > Mar 2020 - making the bigger API changes, upstream code barely compiles
> > May 2020 - COPR repo with first development snapshots
> > Jun 2020 - F33 module available for early adopters and tool developers
> > Oct 2020 - DNF 5 landing in F34 Rawhide
> > Feb 2021 - DNF 5 replacing DNF 4 in stable Fedora
> > DBus service
> >
> > DNF team has decided to create a new DBus service replacing
> > PackageKit to provide an interface to GUI applications. It's
> > probably going to take a while because we're planning to start from
> > scratch.
>
> Do you plan to make normal 'dnf' calls go through the dbus api?
> (And e.g. provide a single cache and privilege escalation through
> packagekit)?

No, dnf stays as it is, it's not going to switch to dbus.
Single cache is part of the plan.
> Apart from the dbus api, do you plan to provide some graphical
> application that uses this api?

Definitely not at this moment.
We need to build the new service and integrate it with the existing 
tools first. We may create a simple command-line tool replacing pkcon, 
something like microdnf over dbus.
> Are you going to use sd-bus for the dbus library?

Yes, it's going to be sd-bus + sdbus-cpp:
* https://github.com/Kistler-Group/sdbus-cpp
* https://src.fedoraproject.org/rpms/sdbus-cpp

[Marcin Juszkiewicz]

W dniu 04.03.2020 o 19:03, Daniel Mach pisze:
> Hello everyone, I'm pleased to announce start of DNF 5 development.
> microdnf
> Microdnf is becoming important because it's part of
> many containers due to its small footprint.

[root@puchatek hrw]# ldd /bin/microdnf |wc -l
70

hrw@j13-qrep-04:ceph-12.2.11$ ldd /usr/bin/apt|wc -l
20

Are there plans for picodnf then? Or cutting amount of 
libraries used by microdnf?

My problem with DNF is Python. There is a huge amount of
packages which need to be in proper state to be able to use
dnf. I remember when I did some experiments with RHEL7 and
managed to get to the point where 'yum' was unable to help
as Python was broken.

[Neal Gompa]

On Thu, Mar 5, 2020 at 7:06 AM Marcin Juszkiewicz
 wrote:
> W dniu 04.03.2020 o 19:03, Daniel Mach pisze:
> > Hello everyone, I'm pleased to announce start of DNF 5 development.
> > microdnf
> > Microdnf is becoming important because it's part of
> > many containers due to its small footprint.
>
> [root@puchatek hrw]# ldd /bin/microdnf |wc -l
> 70
>
> hrw@j13-qrep-04:ceph-12.2.11$ ldd /usr/bin/apt|wc -l
> 20

Apt's library dependencies are a lie if you look just at /usr/bin/apt,
as it relies very heavily on subprocessing for core functionality.
You'd need to check and add the dependencies of the helper binaries
that are required for apt functionality. Last I checked with 1.9.10,
it's fairly comparable.
> Are there plans for picodnf then? Or cutting amount of
> libraries used by microdnf?
>
> My problem with DNF is Python. There is a huge amount of
> packages which need to be in proper state to be able to use
> dnf. I remember when I did some experiments with RHEL7 and
> managed to get to the point where 'yum' was unable to help
> as Python was broken.

Micro DNF is in C (using glib/gobject) today and would be in C++ in
the future. The transition from C to C++ will allow the GNOME library
ecosystem to be fully dropped as a dependency from the DNF stack. That
is substantial baggage that will finally go away.

[Daniel Mach]

Dne 05. 03. 20 v 13:02 Marcin Juszkiewicz napsal(a):
> W dniu 04.03.2020 o 19:03, Daniel Mach pisze:
> > Hello everyone, I'm pleased to announce start of DNF 5 development.
> > microdnf
> > Microdnf is becoming important because it's part of
> > many containers due to its small footprint.
>
> [root@puchatek hrw]# ldd /bin/microdnf |wc -l
> 70Following deps should be going away:

libglib-2.0.so.0
libgobject-2.0.so.0
libffi.so.6
libpeas-1.0.so.0
libgmodule-2.0.so.0
libgirepository-1.0.so.1
libgio-2.0.so.0
> hrw@j13-qrep-04:ceph-12.2.11$ ldd /usr/bin/apt|wc -l
> 20
>
> Are there plans for picodnf then? Or cutting amount of
> libraries used by microdnf?

I don't think we can drop more without losing functionality.
If you take a closer look, libdnf links to libcurl which pulls most of 
the dependencies.
> My problem with DNF is Python. There is a huge amount of
> packages which need to be in proper state to be able to use
> dnf. I remember when I did some experiments with RHEL7 and
> managed to get to the point where 'yum' was unable to help
> as Python was broken.

This probably makes you one of the users we're improving microdnf for.
I hope that someday we'll get microdnf close to 100% feature parity with 
DNF (Python plugins excluding obviously). But that's going to take a 
while...

[Paul Howarth]

On Thu, 5 Mar 2020 20:08:17 +0100
Daniel Mach  wrote:
> Dne 05. 03. 20 v 13:02 Marcin Juszkiewicz napsal(a):
> > W dniu 04.03.2020 o 19:03, Daniel Mach pisze:  
> > Are there plans for picodnf then? Or cutting amount of
> > libraries used by microdnf?
>
> I don't think we can drop more without losing functionality.
> If you take a closer look, libdnf links to libcurl which pulls most
> of the dependencies.

In an environment where size matters a lot, you could use
curl-minimal and libcurl-minimal instead of curl and libcurl, which
pull in fewer dependencies.

Paul.

[Przemek Klosowski]

On 3/5/20 2:08 PM, Daniel Mach wrote:
> I hope that someday we'll get microdnf close to 100% feature parity 
> with DNF (Python plugins excluding obviously)

Great plan, but when it happens, could it also get a simple IPC to talk 
to the optional Python plugins, and thus replace dnf entirely? What is 
the intended differentiation between dnf and microdnf design goals?

[J. Randall Owens]

On 04/03/2020 18:03, Daniel Mach wrote:
> Hello everyone,
> I'm pleased to announce start of DNF 5 development. We are planning to
> deliver a module stream or a COPR repo during Fedora 33 development for
> early adopters and tool developers and we're hoping in getting a stable
> version into Fedora 34.
>
> More details follow.
>
> We've managed to drop a lot of redundant code across the whole DNF stack
> in the past years, but we have reached a point when it's nearly
> impossible to consolidate the code any further without breaking the
> API/ABI. Especially with PackageKit being dead[1], we can't move with
> the old "libhif" API in libdnf, because making any bigger changes to
> PackageKit is clearly out of scope.
>
> [1]
> https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-well...
>
> That's why we decided to start working on a new version of the DNF
> stack: DNF 5. And this is the plan:
> Priorities
> Consistency, documentation and user experience is the top priority.
> Compatibility on the command line level.
> Compatibility on the API level.

<snip>

In favour of 1., but going against 2., I've wanted since the YUM days to
see more consistency in the compound-word* options. E.g., --enablerepo
but --skip-broken, and I have a hard time remembering which ones get the
hyphen inbetween*, and which ones don't. Could we settle on one way or
the other, right now quite the majority are without hyphen, and
deprecate the other (but presumably enable both spellings for a good
long while, perhaps with a warning if someone spells it the old way)?
Probably not take away the old spellings as soon as DNF 5, but it might
be a good time to add the more consistent spellings.

* Yes, I did that on purpose.

[Michael Catanzaro]

Also: builddep vs. build-dep

[Daniel Mach]

Dne 05. 03. 20 v 19:38 J. Randall Owens napsal(a):
> On 04/03/2020 18:03, Daniel Mach wrote:
> > Hello everyone,
> > I'm pleased to announce start of DNF 5 development. We are planning to
> > deliver a module stream or a COPR repo during Fedora 33 development for
> > early adopters and tool developers and we're hoping in getting a stable
> > version into Fedora 34.
> >
> > More details follow.
> >
> > We've managed to drop a lot of redundant code across the whole DNF stack
> > in the past years, but we have reached a point when it's nearly
> > impossible to consolidate the code any further without breaking the
> > API/ABI. Especially with PackageKit being dead[1], we can't move with
> > the old "libhif" API in libdnf, because making any bigger changes to
> > PackageKit is clearly out of scope.
> >
> > [1]
> > https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-well...
> >
> > That's why we decided to start working on a new version of the DNF
> > stack: DNF 5. And this is the plan:
> > Priorities
> > Consistency, documentation and user experience is the top priority.
> > Compatibility on the command line level.
> > Compatibility on the API level.
>
> <snip>
>
> In favour of 1., but going against 2., I've wanted since the YUM days to
> see more consistency in the compound-word* options. E.g., --enablerepo
> but --skip-broken, and I have a hard time remembering which ones get the
> hyphen inbetween*, and which ones don't. Could we settle on one way or
> the other, right now quite the majority are without hyphen, and
> deprecate the other (but presumably enable both spellings for a good
> long while, perhaps with a warning if someone spells it the old way)?
> Probably not take away the old spellings as soon as DNF 5, but it might
> be a good time to add the more consistent spellings.
> Yes, I did that on purpose.

I'll definitely put this on the list.

We're primarily aiming at even worse stuff:
$ dnf list bash -> displays bash package
$ dnf list available -> displays all available packages
$ dnf list installed -> displays all installed packages
$ dnf list all -> displays all packages
What if there were packages named 'all', 'available' or 'installed'?
These are yum compatibility quirks which should be replaced with proper 
--options. There's definitely much more than this example.

[Adam Williamson]

On Thu, 2020-03-05 at 18:38 +0000, J. Randall Owens wrote:
> On 04/03/2020 18:03, Daniel Mach wrote:
> > Hello everyone,
> > I'm pleased to announce start of DNF 5 development. We are planning to
> > deliver a module stream or a COPR repo during Fedora 33 development for
> > early adopters and tool developers and we're hoping in getting a stable
> > version into Fedora 34.
> >
> > More details follow.
> >
> > We've managed to drop a lot of redundant code across the whole DNF stack
> > in the past years, but we have reached a point when it's nearly
> > impossible to consolidate the code any further without breaking the
> > API/ABI. Especially with PackageKit being dead[1], we can't move with
> > the old "libhif" API in libdnf, because making any bigger changes to
> > PackageKit is clearly out of scope.
> >
> > [1]
> > https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-well...
> >
> > That's why we decided to start working on a new version of the DNF
> > stack: DNF 5. And this is the plan:
> > Priorities
> > Consistency, documentation and user experience is the top priority.
> > Compatibility on the command line level.
> > Compatibility on the API level.
>
> <snip>
>
> In favour of 1., but going against 2., I've wanted since the YUM days to
> see more consistency in the compound-word* options. E.g., --enablerepo
> but --skip-broken, and I have a hard time remembering which ones get the
> hyphen inbetween*, and which ones don't. Could we settle on one way or
> the other, right now quite the majority are without hyphen, and
> deprecate the other (but presumably enable both spellings for a good
> long while, perhaps with a warning if someone spells it the old way)?
> Probably not take away the old spellings as soon as DNF 5, but it might
> be a good time to add the more consistent spellings.

I mean, it shouldn't be too hard to standardize on having them all work
both ways, even.

[José Abílio Matos]

On Thursday, 5 March 2020 21.11.37 WET Adam Williamson wrote:
> I mean, it shouldn't be too hard to standardize on having them all work
> both ways, even.

I agree with Adam. What would be the drawback from this change?

It is backwards compatible (an important issue in this context).
And there are other cases like:
--enable, --set-enabled

according to the documentation those are synonyms.

[Luya Tshimbalanga]

> We've managed to drop a lot of redundant code across the whole DNF stack 
> in the past years, but we have reached a point when it's nearly 
> impossible to consolidate the code any further without breaking the 
> API/ABI. Especially with PackageKit being dead[1], we can't move with 
> the old "libhif" API in libdnf, because making any bigger changes to 
> PackageKit is clearly out of scope.
>
> [1] 
> https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-w...

*Ahem* See the comments on the above links an the github is still active https://github.com/hughsie/PackageKit

The main issue related to PackageKit linked libdnf was the lack of synchronization of the database. PackageKit was made to be cross-platform for distributions with minimum reliance on specific distribution command, Losing PackageKit would lead to yet another step back with multiple commands difficult to remember thus another fragmentation.

[Neal Gompa]

On Thu, Mar 5, 2020 at 8:51 PM Luya Tshimbalanga  wrote:
> > We've managed to drop a lot of redundant code across the whole DNF stack
> > in the past years, but we have reached a point when it's nearly
> > impossible to consolidate the code any further without breaking the
> > API/ABI. Especially with PackageKit being dead[1], we can't move with
> > the old "libhif" API in libdnf, because making any bigger changes to
> > PackageKit is clearly out of scope.
> >
> > [1]
> > https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-w...
>
> *Ahem* See the comments on the above links an the github is still active https://github.com/hughsie/PackageKit
>
> The main issue related to PackageKit linked libdnf was the lack of synchronization of the database. PackageKit was made to be cross-platform for distributions with minimum reliance on specific distribution command, Losing PackageKit would lead to yet another step back with multiple commands difficult to remember thus another fragmentation.

The database has been synchronized since Fedora 24. However, the
caches are not, and that *does* need to be fixed. The library
interface for DNF already has APIs for this, PackageKit doesn't use
them.

[Richard Hughes]

On Fri, 6 Mar 2020 at 01:59, Neal Gompa  wrote:
> caches are not, and that *does* need to be fixed. The library
> interface for DNF already has APIs for this, PackageKit doesn't use
> them.

I'm 100% okay with a pull request to fix this, as long as the same
cache promises are met, i.e. PackageKit never downloads an incomplete
set of metadata and does an atomic swap.

Richard.

[Marcin Juszkiewicz]

W dniu 06.03.2020 o 02:57, Neal Gompa pisze:
> The database has been synchronized since Fedora 24. However, the
> caches are not, and that *does* need to be fixed.

And when user calls dnf let it use system cache by default...

17:38 (0s) hrw@puchatek:~$ time sudo dnf info nano
[
 fetching all repo data...
]
Dostępne pakiety
Nazwa        : nano
Wersja       : 4.3
Wydanie      : 3.fc31
Architektura : x86_64
Rozmiar      : 640 k
Źródło       : nano-4.3-3.fc31.src.rpm
Repozytorium : updates
Podsumowanie : A small text editor
Adres URL    : https://www.nano-editor.org
Licencja     : GPLv3+
Opis         : GNU nano is a small and friendly text editor.

real    0m49,066s
user    0m21,141s
sys     0m1,131s
17:38 (49s) hrw@puchatek:~$ time sudo dnf info nano
Ostatnio sprawdzono ważność metadanych: 0:00:09 temu w dniu pią, 6 mar 2020, 17:38:55.
Dostępne pakiety
Nazwa        : nano
Wersja       : 4.3
Wydanie      : 3.fc31
Architektura : x86_64
Rozmiar      : 640 k
Źródło       : nano-4.3-3.fc31.src.rpm
Repozytorium : updates
Podsumowanie : A small text editor
Adres URL    : https://www.nano-editor.org
Licencja     : GPLv3+
Opis         : GNU nano is a small and friendly text editor.

real    0m1,542s
user    0m1,368s
sys     0m0,142s
17:39 (2s) hrw@puchatek:~$ time dnf info nano
[
 fetching all repo data...
]
Dostępne pakiety
Nazwa        : nano
Wersja       : 4.3
Wydanie      : 3.fc31
Architektura : x86_64
Rozmiar      : 640 k
Źródło       : nano-4.3-3.fc31.src.rpm
Repozytorium : updates
Podsumowanie : A small text editor
Adres URL    : https://www.nano-editor.org
Licencja     : GPLv3+
Opis         : GNU nano is a small and friendly text editor.

real    0m56,689s
user    0m21,374s
sys     0m1,130s
17:40 (57s) hrw@puchatek:~$

I have never understood a need for per-user copy of repo metadata.

[Stephen John Smoogen]

On Fri, 6 Mar 2020 at 11:42, Marcin Juszkiewicz 
wrote:
> W dniu 06.03.2020 o 02:57, Neal Gompa pisze:
> > The database has been synchronized since Fedora 24. However, the
> > caches are not, and that *does* need to be fixed.
>
> And when user calls dnf let it use system cache by default...

....
> I have never understood a need for per-user copy of repo metadata.

I am remembering that attempts to do this with either yum or dnf did not go
over well with the community as a whole. To make it work, the cache has to
be either constantly polling or be able to be triggered by the user to keep
up to date. Having it running all the time got complaints about memory and
cpu usage. Letting the user trigger this allowed for various bad
actor/troll ways of polluting the cache with bad data that affected
everyone. Having a daemon which runs to keep things up to date was
considered wasteful for resources (why is dnfd sitting in the background at
2GB of ram??).

Next there is always the person who wants to cache stuff outside of what is
system on. If you are on a F31 box and need to develop against F33 and F32,
then the system has to cache all of those. If you have it just caching for
the user defined ones.. you might as well have it cache for F31 also at the
user level as the speed for doing that is not much different.

Finally there are the people who expect it to work that way for whatever
tool they have and not doing it that way breaks their carefully worked out
workflows.

[Chris Murphy]

On Fri, Mar 6, 2020 at 10:06 AM Stephen John Smoogen  wrote:
> On Fri, 6 Mar 2020 at 11:42, Marcin Juszkiewicz  wrote:
> > I have never understood a need for per-user copy of repo metadata.
>
> I am remembering that attempts to do this with either yum or dnf did not go over well with the community as a whole. To make it work, the cache has to be either constantly polling or be able to be triggered by the user to keep up to date.

Every hour it's updated by dnf makecache, which only updates the one
in /var/cache/dnf, not the one in /var/tmp if the user has their own.

[Robert-André Mauchin]

On Wednesday, 4 March 2020 19:03:01 CET Daniel Mach wrote:
> Hello everyone,
> I'm pleased to announce start of DNF 5 development. We are planning to 
> deliver a module stream or a COPR repo during Fedora 33 development for 
> early adopters and tool developers and we're hoping in getting a stable 
> version into Fedora 34.

Can it be:
 1. faster
 2. have a better dnf history (like one we could interact with)

Also I still don't understand why my dnf redownloads the repolist almost 
everytime I use it for a search. I thought thiswas supposed to be fixed with 
delta repo or zchunk something. This past is the really slow one.

Best regards,

Robert-André

[Peter Robinson]

> > Hello everyone,
> > I'm pleased to announce start of DNF 5 development. We are planning to
> > deliver a module stream or a COPR repo during Fedora 33 development for
> > early adopters and tool developers and we're hoping in getting a stable
> > version into Fedora 34.
>
> Can it be:
> faster
> have a better dnf history (like one we could interact with)
>
> Also I still don't understand why my dnf redownloads the repolist almost
> everytime I use it for a search. I thought thiswas supposed to be fixed with
> delta repo or zchunk something. This past is the really slow one.

It also regularly downloads the entire cache again even if the cache
is not very old. Things like a "dnf repoquery --whatrequires" pretty
much is guaranteed to re-dowload the cache again even if a repoquery
commands was only run a few minutes earlier.

[Tom Seewald]

Yep, I just ran "dnf info kernel" and then right after that "dnf changelog kernel", in both cases dnf spent over 20 seconds syncing.  I haven't seen other package managers require this much network traffic, and I wonder if a lot of it could be avoided.

[Adam Williamson]

On Mon, 2020-04-06 at 23:56 +0000, Tom Seewald wrote:
> Yep, I just ran "dnf info kernel" and then right after that "dnf
> changelog kernel", in both cases dnf spent over 20 seconds syncing. 
> I haven't seen other package managers require this much network
> traffic, and I wonder if a lot of it could be avoided.

If you only want to query *locally installed* packages, you can just
use rpm:

rpm -qi kernel
rpm -q --changelog kernel

if you use DNF, it will get the info for packages that are in the
repositories but not locally installed, as well. changelogs are kept
and retrieved separately, I believe, as they aren't needed for any
other operations, so that's why you see remote trips on both commands,
I think.

[Neal Gompa]

On Mon, Apr 6, 2020 at 8:08 PM Adam Williamson
 wrote:
> On Mon, 2020-04-06 at 23:56 +0000, Tom Seewald wrote:
> > Yep, I just ran "dnf info kernel" and then right after that "dnf
> > changelog kernel", in both cases dnf spent over 20 seconds syncing.
> > I haven't seen other package managers require this much network
> > traffic, and I wonder if a lot of it could be avoided.
>
> If you only want to query *locally installed* packages, you can just
> use rpm:
>
> rpm -qi kernel
> rpm -q --changelog kernel
>
> if you use DNF, it will get the info for packages that are in the
> repositories but not locally installed, as well. changelogs are kept
> and retrieved separately, I believe, as they aren't needed for any
> other operations, so that's why you see remote trips on both commands,
> I think.

With the exception of changelogs (which requires downloading other.xml
metadata), all query actions should be possible to perform with DNF
with --cacheonly flag, which will use the system cache or local user
cache if it hasn't expired yet.

I have heard from people that the cache is expiring prematurely in
some cases (thus forcing more metadata downloads), but I haven't been
able to reproduce it myself.