F27 System Wide Change: Switch libidn-using applications to IDNA2008

[Jan Kurik]

= Proposed System Wide Change: Switch libidn-using applications to IDNA2008 =
https://fedoraproject.org/wiki/Changes/IDNA2008

Change owner(s):
* Nikos Mavrogiannopoulos <nmav AT redhat DOT com>
* Robert Scheck <robert AT fedoraproject DOT org>

The proposed change is about deprecating libidn, which supports
IDNA2003, and switch all applications using libidn, to libidn2 2.0.0,
which supports IDNA2008.


== Detailed Description ==
Internationalized domain names exist for quite some time (IDNA2003),
although the protocols describing them have evolved in an incompatible
way (IDNA2008). These incompatibilities will prevent applications
written for IDNA2003 to access certain problematic domain names
defined with IDNA2008, e.g., faß.de is translated to domain
xn--fa-hia.de with IDNA2008, while in IDNA2003 it is translated to
fass.de domain. That not only causes incompatibility problems, but may
be used as an attack vector to redirect users to different web sites.

The proposed change is about deprecating libidn, which supports
IDNA2003, and switch all applications using libidn, to libidn2 2.0.0,
which supports IDNA2008. The switch should be transparent as the
libidn2 library is API compatible.

Note that even in the web browsers, field there is much confusion on
the topic. Chromium appears to use IDNA2008 transitional (i.e.,
IDNA2003 mapping for the problematic characters), while Firefox and
Safari have already moved to IDNA2008.

For more information see:
* http://nmav.gnutls.org/2017/04/the-mess-with-internationalized-domain.html
* https://www.plesk.com/blog/what-is-the-problem-with-s/
* http://unicode.org/faq/idn.html#6


== Scope ==
* Proposal owners:
The proposal owner is expected to co-ordinate and fill the required
bugs on the distribution.

* Other developers:
Maintainers, should
- Verify that their software is linked with the libidn library
- Update the software from upstream if it already has been converted to libidn2
- Check the libidn2 instructions on converting a package to libidn2.
- Propose patches (trivial task) to convert to libidn2, and notify
upstream about it.
In short switch software from libidn to libidn2, it is sufficient
replacing idna.h header with idn2.h.

* Release engineering:
This feature requires not action from release engineering.

* Policies and guidelines:
Currently Fedora has no guidelines for IDNA support. With this change
the recommended guideline for applications would be to support
IDNA2008 by default.

* Trademark approval:
N/A (not needed for this Change)
-- 
Jan Kuřík
Platform & Fedora Program Manager
Red Hat Czech s.r.o., Purkynova 99/71, 612 45 Brno, Czech Republic

IDNA2008 change impact on installation size

[Zbigniew Jędrzejewski-Szmek]

Hi,

has any consideration been give to the size increase required by the
change from libidn (678k) to libidn2 + libunistring (228k + 1246k)?
That's not *too* bad, since currently none of the things which depend
on idn end up in the initramfs, but still, it's noticeable.

Zbyszek


On Tue, Apr 04, 2017 at 08:44:39AM +0200, Jan Kurik wrote:
> = Proposed System Wide Change: Switch libidn-using applications to IDNA2008 =
> https://fedoraproject.org/wiki/Changes/IDNA2008
> 
> Change owner(s):
> * Nikos Mavrogiannopoulos <nmav AT redhat DOT com>
> * Robert Scheck <robert AT fedoraproject DOT org>
> 
> The proposed change is about deprecating libidn, which supports
> IDNA2003, and switch all applications using libidn, to libidn2 2.0.0,
> which supports IDNA2008.
> 
> 
> == Detailed Description ==
> Internationalized domain names exist for quite some time (IDNA2003),
> although the protocols describing them have evolved in an incompatible
> way (IDNA2008). These incompatibilities will prevent applications
> written for IDNA2003 to access certain problematic domain names
> defined with IDNA2008, e.g., faß.de is translated to domain
> xn--fa-hia.de with IDNA2008, while in IDNA2003 it is translated to
> fass.de domain. That not only causes incompatibility problems, but may
> be used as an attack vector to redirect users to different web sites.
> 
> The proposed change is about deprecating libidn, which supports
> IDNA2003, and switch all applications using libidn, to libidn2 2.0.0,
> which supports IDNA2008. The switch should be transparent as the
> libidn2 library is API compatible.
> 
> Note that even in the web browsers, field there is much confusion on
> the topic. Chromium appears to use IDNA2008 transitional (i.e.,
> IDNA2003 mapping for the problematic characters), while Firefox and
> Safari have already moved to IDNA2008.
> 
> For more information see:
> * http://nmav.gnutls.org/2017/04/the-mess-with-internationalized-domain.html
> * https://www.plesk.com/blog/what-is-the-problem-with-s/
> * http://unicode.org/faq/idn.html#6
> 
> 
> == Scope ==
> * Proposal owners:
> The proposal owner is expected to co-ordinate and fill the required
> bugs on the distribution.
> 
> * Other developers:
> Maintainers, should
> - Verify that their software is linked with the libidn library
> - Update the software from upstream if it already has been converted to libidn2
> - Check the libidn2 instructions on converting a package to libidn2.
> - Propose patches (trivial task) to convert to libidn2, and notify
> upstream about it.
> In short switch software from libidn to libidn2, it is sufficient
> replacing idna.h header with idn2.h.
> 
> * Release engineering:
> This feature requires not action from release engineering.
> 
> * Policies and guidelines:
> Currently Fedora has no guidelines for IDNA support. With this change
> the recommended guideline for applications would be to support
> IDNA2008 by default.
> 
> * Trademark approval:
> N/A (not needed for this Change)
> -- 
> Jan Kuřík
> Platform & Fedora Program Manager
> Red Hat Czech s.r.o., Purkynova 99/71, 612 45 Brno, Czech Republic
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org

IDNA2008 change impact on installation size

[Matthew Miller]

On Fri, May 12, 2017 at 04:05:50PM +0000, Zbigniew Jędrzejewski-Szmek wrote:
>  has any consideration been give to the size increase required by the
>  change from libidn (678k) to libidn2 + libunistring (228k + 1246k)?
>  That's not *too* bad, since currently none of the things which depend
>  on idn end up in the initramfs, but still, it's noticeable. 
Thanks for considering this angle. Is there a way to reduce that? Looks
like libunistring is just... large.


-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

IDNA2008 change impact on installation size

[Nikos Mavrogiannopoulos]

On Mon, 2017-05-15 at 09:42 -0400, Matthew Miller wrote:
>  On Fri, May 12, 2017 at 04:05:50PM +0000, Zbigniew Jędrzejewski-Szmek
>
>  wrote:
>  > has any consideration been give to the size increase required by
>  > the
>  > change from libidn (678k) to libidn2 + libunistring (228k + 1246k)?
>  > That's not *too* bad, since currently none of the things which
>  > depend
>  > on idn end up in the initramfs, but still, it's noticeable.
>  
>  Thanks for considering this angle. Is there a way to reduce that?
>  Looks like libunistring is just... large. 
We could include a libidn2 which only contains the parts of
libunistring that it needs via gnulib. That could reduce the size
significantly, at the cost of including a library as static.

My understanding of libunistring is that it is going to be widely used
for UTF conversions and internationalized string handling given that
the posix APIs in libc are terrible in that aspect. I believe it is
already widely used mostly as gnulib module (copylib).

regards,
Nikos

Switch to IDNA2008 (should we keep libidn?)

[Nikos Mavrogiannopoulos]

On Tue, 2017-04-04 at 08:44 +0200, Jan Kurik wrote:
>  = Proposed System Wide Change: Switch libidn-using applications to
>  IDNA2008 =
>  https://fedoraproject.org/wiki/Changes/IDNA2008
>  
>  Change owner(s):
>  * Nikos Mavrogiannopoulos <nmav AT redhat DOT com>
>  * Robert Scheck <robert AT fedoraproject DOT org>
>  
>  The proposed change is about deprecating libidn, which supports
>  IDNA2003, and switch all applications using libidn, to libidn2 2.0.0,
>  which supports IDNA2008.
>  
>  
>  == Detailed Description ==
>  Internationalized domain names exist for quite some time (IDNA2003),
>  although the protocols describing them have evolved in an
>  incompatible
>  way (IDNA2008). These incompatibilities will prevent applications
>  written for IDNA2003 to access certain problematic domain names
>  defined with IDNA2008, e.g., faß.de is translated to domain
>  xn--fa-hia.de with IDNA2008, while in IDNA2003 it is translated to
>  fass.de domain. That not only causes incompatibility problems, but
>  may
>  be used as an attack vector to redirect users to different web sites.
>  
>  The proposed change is about deprecating libidn, which supports
>  IDNA2003, and switch all applications using libidn, to libidn2 2.0.0,
>  which supports IDNA2008. The switch should be transparent as the
>  libidn2 library is API compatible.
>  
>  Note that even in the web browsers, field there is much confusion on
>  the topic. Chromium appears to use IDNA2008 transitional (i.e.,
>  IDNA2003 mapping for the problematic characters), while Firefox and
>  Safari have already moved to IDNA2008.
>  
>  For more information see:
>  * http://nmav.gnutls.org/2017/04/the-mess-with-internationalized-doma
>  in.html
>  * https://www.plesk.com/blog/what-is-the-problem-with-s/
>  * http://unicode.org/faq/idn.html#6
>  
>  
>  == Scope ==
>  * Proposal owners:
>  The proposal owner is expected to co-ordinate and fill the required
>  bugs on the distribution. 
What was left out of the proposal is whether we should keep libidn in
the OS. IDNA2003 may still be in use somewhere, but even when it is, it
is via IDNA2008-transitional mode. Are there any other reasons to keep
libidn in Fedora?

regards,
Nikos

IDNA2008 change impact on installation size

[Nikos Mavrogiannopoulos]

On Fri, 2017-05-12 at 16:05 +0000, Zbigniew Jędrzejewski-Szmek wrote:
>  Hi,
>  
>  has any consideration been give to the size increase required by the
>  change from libidn (678k) to libidn2 + libunistring (228k + 1246k)?
>  That's not *too* bad, since currently none of the things which depend
>  on idn end up in the initramfs, but still, it's noticeable. 
Note also that these packages will increase in size as unicode standard
evolves. libidn due to IDNA2003 was tied to a specific old standard
while libunistring gets updated to the latest one thus includes much
more data than the previous ones.

I do not know whether the glibc switch from libidn to libidn2 would
provide some benefit in that aspect.

regards,
Nikos