hawkey replaced by libhif, DNF into C initiative started

[Honza Silhan]

TL;DR

Hawkey project [1] is being obsoleted, use libhif [2] instead. For hawkey Python API
consumers,
the transition should be painless.


Why we have done that?

Nowadays there are three major consumers of hawkey - DNF, PackageKit and rpm-ostree. The
hawkey
API was not in final form yet and was changed constantly based on demands from these
package
managers. We have merged hawkey project inside libhif and hidden some of not yet stable
API.

Merging hawkey into libhif was another step to move more code base of DNF into C. DNF will
reuse
some of the existing code of libhif. Having this shared library can eliminate
inconsistencies
about installed packages when DNF and PackageKit is used alternately. Moreover we would
like to
reuse the same metadata for all package managers to save your bandwidth.

Libhif should contain the common functionality for all package managers. So far libhif is
providing high level API by taking care of fetching metadata from mirrors, doing
dependency
solving and executing RPM transaction. In the future it will support repository
configuration
parsing, GPG checking and so on. At this time, this is handled by all package managers
separately.


Facts for Hawkey consumers:

* libhif-0.7.0 will obsolete hawkey package
* some of the C hawkey API from libhif will not be exposed anymore, please use libhif
functions
instead
* python bindings will not change and the libhif package will still provide
`python2-hawkey` and
`python3-hawkey`
* API in libhif is still not considered as fully stable yet
* first release of libhif with hawkey inside is targeted for Fedora 25

Please watch libhif project on github [2] and participate in PR discussions so you can
influence
the development. Do not hesitate to respond with your concerns / opinions.


Honza

[0] https://dnf.baseurl.org/2016/02/24/dnf-into-c-initiative-started/
[1] https://github.com/rpm-software-management/hawkey
[2] https://github.com/rpm-software-management/libhif

[Josh Boyer]

On Wed, Feb 24, 2016 at 12:35 PM, Honza Šilhan <jsilhan(a)redhat.com&gt; wrote:
>  TL;DR
>
>  Hawkey project [1] is being obsoleted, use libhif [2] instead. For hawkey Python API
> consumers,
>  the transition should be painless.
>
>
>  Why we have done that?
>
>  Nowadays there are three major consumers of hawkey - DNF, PackageKit and rpm-ostree. The
> hawkey
>  API was not in final form yet and was changed constantly based on demands from these
> package
>  managers. We have merged hawkey project inside libhif and hidden some of not yet stable
> API.
>
>  Merging hawkey into libhif was another step to move more code base of DNF into C. DNF
> will reuse
>  some of the existing code of libhif. Having this shared library can eliminate
> inconsistencies
>  about installed packages when DNF and PackageKit is used alternately. Moreover we would
> like to
>  reuse the same metadata for all package managers to save your bandwidth. 
Could you please elaborate (perhaps in a separate thread) the full
desire and requirements behind rewriting DNF into C?  The first link
in your email says the initiative is started, but it doesn't really
explain why aside from the libhif step.

I can think of many reasons for doing this, but it would probably be
better coming from the development team doing the work.  It would also
be nice if you could explain how the recently proposed and accepted
system-python Change plays into this long term.  I believe DNF was one
of the targeted "system tools" that would use it, so if that isn't
really going to be the case it would be good to know.

josh

[Honza Silhan]

>  From: "Josh Boyer" <jwboyer(a)fedoraproject.org&gt;
>  Could you please elaborate (perhaps in a separate thread) the full
>  desire and requirements behind rewriting DNF into C?  The first link
>  in your email says the initiative is started, but it doesn't really
>  explain why aside from the libhif step. 
Having DNF in C is desire of many users. If you could notice the thread
from Courtney Pacheco, she did investigation [1] to minimize the Fedora
image (which is bigger than in other distros). The DNF will probably be
taken out of that image. Mostly because of the Python requirement.

By dropping Python requirement, it could speed up the the process start-up
(no need to load Python libs then). That is also the reason why PK does not
call DNF Python API from C and rather have a special library in C instead.

>  I can think of many reasons for doing this, but it would probably be
>  better coming from the development team doing the work.  It would also
>  be nice if you could explain how the recently proposed and accepted
>  system-python Change plays into this long term.  I believe DNF was one
>  of the targeted "system tools" that would use it, so if that isn't
>  really going to be the case it would be good to know. 
Gradually DNF is moving more code into C but still a lot of code is in
Python and plugins are written in Python too. Having rewritten DNF into
C would take a few years while maintaining the UX compatibility and
still support Python API for plugins.

The main message was to inform the consumers of hawkey to prepare their
apps for the libhif change.


Honza

[1] https://gist.github.com/iamcourtney/b8709ed897b7ecc9ac0f

[David Malcolm]

On Thu, 2016-02-25 at 05:42 -0500, Honza Šilhan wrote:
>  > From: "Josh Boyer" <jwboyer(a)fedoraproject.org&gt;
>  > Could you please elaborate (perhaps in a separate thread) the full
>  > desire and requirements behind rewriting DNF into C?  The first
>  > link
>  > in your email says the initiative is started, but it doesn't really
>  > explain why aside from the libhif step.
>  
>  Having DNF in C is desire of many users. If you could notice the
>  thread
>  from Courtney Pacheco, she did investigation [1] to minimize the
>  Fedora
>  image (which is bigger than in other distros). The DNF will probably
>  be
>  taken out of that image. Mostly because of the Python requirement. 
It appears the requirement is not so much
  "rewrite DNF in C"
as
  "rewrite DNF in a language that can be compiled to efficient machine
code, which doesn't require a large runtime support library, and which
can interoperate with C".

I think I'm only semi-serious here [1], but have you considered Rust?


[1] e.g. it's not yet in Fedora.

>  By dropping Python requirement, it could speed up the the process
>  start-up
>  (no need to load Python libs then). That is also the reason why PK
>  does not
>  call DNF Python API from C and rather have a special library in C
>  instead.
>  
>  > I can think of many reasons for doing this, but it would probably
>  > be
>  > better coming from the development team doing the work.  It would
>  > also
>  > be nice if you could explain how the recently proposed and accepted
>  > system-python Change plays into this long term.  I believe DNF was
>  > one
>  > of the targeted "system tools" that would use it, so if that isn't
>  > really going to be the case it would be good to know.
>  
>  Gradually DNF is moving more code into C but still a lot of code is
>  in
>  Python and plugins are written in Python too. Having rewritten DNF
>  into
>  C would take a few years while maintaining the UX compatibility and
>  still support Python API for plugins.
>  
>  The main message was to inform the consumers of hawkey to prepare
>  their
>  apps for the libhif change.
>  
>  
>  Honza
>  
>  [1] https://gist.github.com/iamcourtney/b8709ed897b7ecc9ac0f

[Matthew Miller]

On Thu, Feb 25, 2016 at 12:03:52PM -0500, David Malcolm wrote:
>    "rewrite DNF in a language that can be compiled to efficient
> machine
>  code, which doesn't require a large runtime support library, and which
>  can interoperate with C".
>  I think I'm only semi-serious here [1], but have you considered Rust? 
Also on the same level of semi-serious: Go, because this seems to be
The Language Of Choice For Container Tech, which means that Red Hat and
Fedora is rapidly building up expertise. (Is Go the new Python?)

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Neal Gompa]

On Thu, Feb 25, 2016 at 12:25 PM, Matthew Miller
<mattdm(a)fedoraproject.org&gt; wrote:
>  On Thu, Feb 25, 2016 at 12:03:52PM -0500, David Malcolm wrote:
> >   "rewrite DNF in a language that can be compiled to efficient machine
> > code, which doesn't require a large runtime support library, and which
> > can interoperate with C".
> > I think I'm only semi-serious here [1], but have you considered Rust?
>
>  Also on the same level of semi-serious: Go, because this seems to be
>  The Language Of Choice For Container Tech, which means that Red Hat and
>  Fedora is rapidly building up expertise. (Is Go the new Python?)
>
>  --
>  Matthew Miller
>  <mattdm(a)fedoraproject.org&gt;
>  Fedora Project Leader 
At least Go now supports shared libraries and dynamic linking, though
I like the concepts in Rust more, language wise. Does Rust support
shared libraries and dynamic linking?



-- 
真実はいつも一つ！/ Always, there's only one truth!

[David Malcolm]

On Thu, 2016-02-25 at 12:27 -0500, Neal Gompa wrote:
>  On Thu, Feb 25, 2016 at 12:25 PM, Matthew Miller
>  <mattdm(a)fedoraproject.org&gt; wrote:
>  > On Thu, Feb 25, 2016 at 12:03:52PM -0500, David Malcolm wrote:
>  > >   "rewrite DNF in a language that can be compiled to efficient
>  > > machine
>  > > code, which doesn't require a large runtime support library, and
>  > > which
>  > > can interoperate with C".
>  > > I think I'm only semi-serious here [1], but have you considered
>  > > Rust?
>  > 
>  > Also on the same level of semi-serious: Go, because this seems to
>  > be
>  > The Language Of Choice For Container Tech, which means that Red Hat
>  > and
>  > Fedora is rapidly building up expertise. (Is Go the new Python?)
>  > 
>  > --
>  > Matthew Miller
>  > <mattdm(a)fedoraproject.org&gt;
>  > Fedora Project Leader
>  
>  At least Go now supports shared libraries and dynamic linking, though
>  I like the concepts in Rust more, language wise. Does Rust support
>  shared libraries and dynamic linking? 
I believe so; here's an example of building a shared library in Rust
and calling into it from various languages:
https://doc.rust-lang.org/book/rust-inside-other-languages.html

[Jan Kratochvil]

On Thu, 25 Feb 2016 18:03:52 +0100, David Malcolm wrote:
>  I think I'm only semi-serious here [1], but have you considered
> Rust?
>  [1] e.g. it's not yet in Fedora. 
or proven C++11(/14/17)?
(it is already in Fedora)


Jan

[Adam Jackson]

On Thu, 2016-02-25 at 18:58 +0100, Jan Kratochvil wrote:
>  On Thu, 25 Feb 2016 18:03:52 +0100, David Malcolm wrote:
>  > I think I'm only semi-serious here [1], but have you considered
>  > Rust?
>  > [1] e.g. it's not yet in Fedora.
>  
>  or proven C++11(/14/17)?
>  (it is already in Fedora) 
C++ is sort of the reason Go and Rust exist.

- ajax

[Jakub Filak]

On 02/25/2016 08:11 PM, Adam Jackson wrote:
>  On Thu, 2016-02-25 at 18:58 +0100, Jan Kratochvil wrote:
> > On Thu, 25 Feb 2016 18:03:52 +0100, David Malcolm wrote:
> >> I think I'm only semi-serious here [1], but have you considered
> >> Rust?
> >> [1] e.g. it's not yet in Fedora.
> > or proven C++11(/14/17)?
> > (it is already in Fedora)
>  C++ is sort of the reason Go and Rust exist. 
Well, not C++ but C++98 and I understand the reason why these languages 
exist,  though, C++ is my preferred language.
However C+11(/14/17) is almost completely different language.


Jakub