12:25 p.m.
Today the Packaging Committee began a discussion of whether package
names should be allowed to contain the full range of Unicode characters
(encoded as UTF-8) or be restricted to the ASCii subset.
This is a bit of a contentious issue with the Packaging Committee
members split but not yet set in stone about how to proceed.
The main arguments seem to be:
Pro Unicode:
* Upstream knows best what name is most appropriate for their users.
For us to change it locally in Fedora doesn't make much sense.
* We allow Unicode in every other piece of the spec so why not in the
package name?
* We should be shaking out bugs in the handling of Unicode in our
software rather than hiding issues with it.
Pro ASCII:
* Hard to type unicode package names, therefore it is a usability problem.
* Is there a limit? Even if European letters are fine what about Kanji
or Sanskrit?
* Some pieces of software won't handle unicode package names and will
need to be fixed.
One package has been submitted for review with a unicode using package
name and has some applicable comments:
https://bugzilla.redhat.com/show_bug.cgi?id=261881
= Section of Packaging Committee Logs about Unicode Package Names =
(09:32:18 AM) racor: banning non-ascii chars from package names
(09:32:29 AM) spot: racor: eww. is someone actually doing that?
(09:32:39 AM) f13: somebody did a + in a version I thought
(09:32:40 AM) abadger1999: ivazquez sent something to the list. No
actual draft but it was very simple.
(09:32:47 AM) tibbs|h: spot: There's a review submitted.
(09:32:49 AM) svahl left the room (quit: ).
(09:32:51 AM) f13: but that doesn't count.
(09:32:51 AM) rdieter: f13: inkscape, yeah, but fixed now.
(09:32:59 AM) abadger1999: racor: What is the ratioinale?
(09:33:01 AM) f13: tibbs|h: I bet its from nim-nim isn't it?
(09:33:06 AM) racor: yes, there is a packaging under review ... <digging>
(09:33:07 AM) tibbs|h: Yes, some fonts thing.
(09:33:13 AM) spot: that seems like a no-brainer to me.
(09:33:19 AM) abadger1999: spot: Why?
(09:33:22 AM) tibbs|h:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=261881
(09:33:24 AM) buggbot: Bug 261881: medium, medium, ---, Nobody's working
on this, feel free to take it, NEW , Review Request:
écolier-court-fonts - Écolier court fonts
(09:33:37 AM) spot: because xchat rendered that two different ways for me?
(09:33:45 AM) spot: i don't even want to think about yum.
(09:34:02 AM) abadger1999: spot: yum does need a fix. There's an open bug.
(09:34:05 AM) f13: well, it would be good to test our infrastructure.
(09:34:08 AM) tibbs|h: I don't really have an issue with non-ascii
package names; we can't keep falling back on "our infrastructure sucks"
forever.
(09:34:11 AM) abadger1999: But I'm wondering why it's a bad thing.
(09:34:21 AM) abadger1999: Shouldn't we consider places where utf-8
fails to be bugs?
(09:34:44 AM) f13: non-ascii packages may have to be renamed if they
ever show up in RHEL
(09:34:53 AM) f13: I'm not sure how the RHN beast will handle it.
(09:35:07 AM) spot: well, that is RHEL's problem.
(09:35:10 AM) f13: yep
(09:35:17 AM) f13: I'm not saying it should have much bearing on our
decision
(09:35:27 AM) spot: racor: are you against it for aesthetic reasons or
technical ones?
(09:35:31 AM) racor: IMO, the technical issues are minor, the real issue
is usabilitsy
(09:35:44 AM) racor: spot: neither usability
(09:35:51 AM) tibbs|h: I always said that I won't review what I can't type.
(09:36:05 AM) tibbs|h: But my inability to type that is my dysfunction.
(09:36:11 AM) spot: well, i suppose that it would make it more difficult
for english typists to install/use a package
(09:36:23 AM) spot: but not impossible
(09:36:25 AM) racor: consider: 90% of US users are not even able to type
accented chars
(09:36:27 AM) tibbs|h: But is that a reason to ban something?
(09:36:47 AM) tibbs|h: I can't read German either, so let's kick out
German translations.
(09:36:53 AM) racor: 99.89% of Western folks are not able to type east
asian chars
(09:37:07 AM) f13: and 40% of all statistics are made up on the spot
(09:37:20 AM) spot: my concern is when people start naming packages in
kanji.
(09:37:40 AM) racor: tibbs: right type the char ß (this is not a beta
it's a sharp ss)
(09:37:46 AM) spot: we already mandate that the spec file must be
written in american english
(09:38:11 AM) tibbs|h: racor: I already said I can't type those
characters; I don't see what point your question serves.
(09:38:28 AM) rdieter: spot: +1, I think that covers the case here then, no?
(09:38:48 AM) tibbs|h: But I don't see the practical difference between
ideograms and accented vowels, either.
(09:38:57 AM) racor: tibbs: my point: anything outside of ascii not
universal enough
(09:38:59 AM) abadger1999: Yum bug:
https://bugzilla.redhat.com/show_bug.cgi?id=261961
(09:39:01 AM) buggbot: Bug 261961: low, medium, ---, Jeremy Katz, NEW ,
Yum does not like non-ascii package names
(09:39:14 AM) spot: well, technically we only say summary and description
(09:39:18 AM) spot: "Please put personal preferences aside and use
American English spelling in the summary and description."
(09:39:18 AM) tibbs|h: Either we say "ASCII" or "UTF-8"; there's
no
point in anything in the middle.
(09:39:39 AM) tibbs|h: We do not mandate that the entire spec be in English.
(09:39:42 AM) f13: we already support non-ascii file names
(09:39:56 AM) f13: so lon gas they are UTF-8
(09:40:00 AM) tibbs|h: We permit translated descriptions.
(09:40:28 AM) rdieter: Can we at least agree that ASCII *SHOULD* be
used? Not sure if it warrants a MUST.
(09:40:40 AM) spot: yeah, i can support that ASCII should be used
(09:40:43 AM) abadger1999: rdieter: I'm not sure I would agree with that.
(09:40:51 AM) tibbs|h: I don't know if there's really a point.
(09:41:05 AM) racor: package names!!! descr, etc. are legacy, convenience,
(09:41:21 AM) tibbs|h: But we have rules about naming packages after
things like the upstream tarball.
(09:41:27 AM) abadger1999: I mean, nim-nim's point is also valid -- if
the upstream name is non-ascii who are we to differ from upstream?
(09:41:40 AM) rdieter: shrug, I'm ok with pushing the envelope here too.
(09:41:58 AM) spot: if only to encourage people not to be stupid and
spell things like this: ƁƎƗȂ
(09:42:11 AM) f13: can we try this package as a trial run and see what
all it breaks?
(09:42:18 AM) racor: abadger1999: would you say the same if the package
name was in cyrillian or turk?
(09:42:28 AM) ***spot is getting pulled away
(09:42:40 AM) spot: we'll have to pick this up next meeting
(09:42:41 AM) spot: sorry. :(
(09:42:52 AM) rdieter: f13: +1 :)
(09:42:53 AM) abadger1999: racor: What does the package do? Is it a
cyrillic font package? Is it something specific to Russian language
speakers?
(09:43:26 AM) abadger1999: spot: I think that needs to be addressed
upstream.
(09:43:35 AM) racor: abadger1999: not necessarily. Just author preference.
(09:44:09 AM) abadger1999: racor: So that's the gray line for me. I
think I'd say that we can try to influence upstream but it is an
upstream decision.
(09:44:09 AM) tibbs|h: We have a practical issue in any case, because
our infrastructure doesn't properly support non-ASCII package names
currently.
(09:44:27 AM) racor: I could call a package: bärensößchen ...
(09:44:33 AM) tibbs|h: Why not?
(09:45:00 AM) tibbs|h: Looks like as good a name as anything else.
(09:45:54 AM) racor: tibbs: except that most people would not be able to
type it .... yum install <package-name>
(09:46:09 AM) abadger1999: Copy and paste....
(09:46:19 AM) tibbs|h: Graphical interface.
(09:46:25 AM) tibbs|h: Tab completion.
(09:46:34 AM) tibbs|h: Learn to type German.
(09:47:06 AM) racor: abadger1999, tibbs: that's laughable.
(09:47:16 AM) tibbs|h: It's weird to see this argument backwards;
usually the Americans are arguing for "nothing I can't understand" while
the Europeans and Asians just laugh at the idiot luddites.
(09:47:46 AM) tibbs|h: "Nothing in Fedora should use the metric system."
(09:48:26 AM) racor: tibbs: learn to type Thai
(09:49:15 AM) tibbs|h: I personally have no interest in doing so.
(09:49:24 AM) tibbs|h: I don't see how that has any bearing on anything,
though.
-Toshio
12:44 p.m.
Toshio Kuratomi (a.badger(a)gmail.com) said:
Today the Packaging Committee began a discussion of whether package
names
should be allowed to contain the full range of Unicode characters (encoded
as UTF-8) or be restricted to the ASCii subset.
This is a bit of a contentious issue with the Packaging Committee members
split but not yet set in stone about how to proceed.
The main arguments seem to be:
Pro Unicode:
* Upstream knows best what name is most appropriate for their users. For us
to change it locally in Fedora doesn't make much sense.
* We allow Unicode in every other piece of the spec so why not in the
package name?
* We should be shaking out bugs in the handling of Unicode in our software
rather than hiding issues with it.
Pro ASCII:
* Hard to type unicode package names, therefore it is a usability problem.
* Is there a limit? Even if European letters are fine what about Kanji or
Sanskrit?
* Some pieces of software won't handle unicode package names and will need
to be fixed.
The biggest issue I could see is that any issues in handling such a package
would need to be fixed *everywhere* that package is deployed, whether it
be current Fedora, earlier Fedora, or EPEL. It may not be practical to
deploy such fixes everywhere.
Bill
12:49 p.m.
Bill Nottingham (notting(a)redhat.com) said:
The biggest issue I could see is that any issues in handling such a
package
would need to be fixed *everywhere* that package is deployed, whether it
be current Fedora, earlier Fedora, or EPEL. It may not be practical to
deploy such fixes everywhere.
Consider, for example, that such a package breaks bugzilla. Although I
suppose that's one way to avoid getting bug reports.
Bill
12:54 p.m.
On Tue, 2008-02-26 at 13:49 -0500, Bill Nottingham wrote:
Bill Nottingham (notting(a)redhat.com) said:
> The biggest issue I could see is that any issues in handling such a package
> would need to be fixed *everywhere* that package is deployed, whether it
> be current Fedora, earlier Fedora, or EPEL. It may not be practical to
> deploy such fixes everywhere.
Consider, for example, that such a package breaks bugzilla. Although I
suppose that's one way to avoid getting bug reports.
/me renames yum to ᶨᶬⱴ
:-D
-sv
4:09 p.m.
On Tue, 2008-02-26 at 13:54 -0500, seth vidal wrote:
On Tue, 2008-02-26 at 13:49 -0500, Bill Nottingham wrote:
> Bill Nottingham (notting(a)redhat.com) said:
> > The biggest issue I could see is that any issues in handling such a package
> > would need to be fixed *everywhere* that package is deployed, whether it
> > be current Fedora, earlier Fedora, or EPEL. It may not be practical to
> > deploy such fixes everywhere.
To me, these are the least issues. They are
of a technical nature and
can be overcome/fixed (utf-8)
My concern is usability of the distro.
IMO, we can not avoid to restrict certain aspects of the distro to the
least common denominator. Package-names are such a case.
> Consider, for example, that such a package breaks bugzilla.
Although I
> suppose that's one way to avoid getting bug reports.
>
/me renames yum to ᶨᶬⱴ
Is this Farsi, Arab or Hebrew?
Proposal: Let's rename the bodhi and koji packages + their primary
executables into their Indian rsp. Japanese equivalents :)
Ralf
6:56 p.m.
On Tuesday, 26 February 2008 at 23:09, Ralf Corsepius wrote:
On Tue, 2008-02-26 at 13:54 -0500, seth vidal wrote:
> On Tue, 2008-02-26 at 13:49 -0500, Bill Nottingham wrote:
> > Bill Nottingham (notting(a)redhat.com) said:
> > > The biggest issue I could see is that any issues in handling such a
package
> > > would need to be fixed *everywhere* that package is deployed, whether it
> > > be current Fedora, earlier Fedora, or EPEL. It may not be practical to
> > > deploy such fixes everywhere.
To me, these are the least issues. They are of a technical nature and
can be overcome/fixed (utf-8)
My concern is usability of the distro.
IMO, we can not avoid to restrict certain aspects of the distro to the
least common denominator. Package-names are such a case.
> > Consider, for example, that such a package breaks bugzilla. Although I
> > suppose that's one way to avoid getting bug reports.
> >
>
> /me renames yum to ᶨᶬⱴ
Is this Farsi, Arab or Hebrew?
Proposal: Let's rename the bodhi and koji packages + their primary
executables into their Indian rsp. Japanese equivalents :)
I, for one, have nothing against 工事. At least I wouldn't have to cringe
at the bad romanization that's used all over the place. :P
As for executables, you can always do something like
# ln -s 工事 /usr/bin/koji
and ship both... just kidding.
Although this discussion sheds an entirely new light on l10n/i18n. Imagine
a fully localised system, where not only messages, but also everything else
is translated into local language.
/me imagines typing (pl_PL.UTF-8 locale)
$ znajdź . -typ p -upr 0600 -drukuj
Oh, the horror. ;)
Regards,
R.
--
Fedora contributor http://fedoraproject.org/wiki/DominikMierzejewski
Livna contributor http://rpm.livna.org MPlayer developer http://mplayerhq.hu
"Faith manages."
-- Delenn to Lennier in Babylon 5:"Confessions and Lamentations"
7:33 p.m.
Dominik 'Rathann' Mierzejewski wrote:
On Tuesday, 26 February 2008 at 23:09, Ralf Corsepius wrote:
> On Tue, 2008-02-26 at 13:54 -0500, seth vidal wrote:
>> On Tue, 2008-02-26 at 13:49 -0500, Bill Nottingham wrote:
>>> Bill Nottingham (notting(a)redhat.com) said:
>>>> The biggest issue I could see is that any issues in handling such a
package
>>>> would need to be fixed *everywhere* that package is deployed, whether it
>>>> be current Fedora, earlier Fedora, or EPEL. It may not be practical to
>>>> deploy such fixes everywhere.
> To me, these are the least issues. They are of a technical nature and
> can be overcome/fixed (utf-8)
>
> My concern is usability of the distro.
>
> IMO, we can not avoid to restrict certain aspects of the distro to the
> least common denominator. Package-names are such a case.
>
>>> Consider, for example, that such a package breaks bugzilla. Although I
>>> suppose that's one way to avoid getting bug reports.
>>>
>> /me renames yum to ᶨᶬⱴ
> Is this Farsi, Arab or Hebrew?
>
> Proposal: Let's rename the bodhi and koji packages + their primary
> executables into their Indian rsp. Japanese equivalents :)
I, for one, have nothing against 工事. At least I wouldn't have to cringe
at the bad romanization that's used all over the place. :P
As for executables, you can always do something like
# ln -s 工事 /usr/bin/koji
and ship both... just kidding.
This does bring up on interesting point. The package name has been
brought up as a usability concern for those who aren't used to trying to
type non-ASCii characters. However if an upstream was producing a
program whose binary was non-ASCii we'd currently allow inclusion (and I
would be even more opposed to changing that without getting upstream to
make the change than changing the package name.)
For a possible valid use, think of this potential pair of tools for
students learning Japanese:
/usr/bin/ひらがな2rōmaji
/usr/bin/rōmaji2ひらがな
Although this discussion sheds an entirely new light on l10n/i18n.
Imagine
a fully localised system, where not only messages, but also everything else
is translated into local language.
/me imagines typing (pl_PL.UTF-8 locale)
$ znajdź . -typ p -upr 0600 -drukuj
Oh, the horror. ;)
Nonsense! Since there are so many things on the filesystem which aren't
real words (what's a "usr", what's a "gnumeric"?) we should
obviously
forgo translation and instead do transliteration! :-)
-Toshio
9:56 a.m.
On Tue, Feb 26, 2008 at 05:33:33PM -0800, Toshio Kuratomi wrote:
For a possible valid use, think of this potential pair of tools for
students learning Japanese:
/usr/bin/ひらがな2rōmaji
/usr/bin/rōmaji2ひらがな
If I were going to learn Japanese tomorrow and were looking for such a
tool I wouldn't be actually able to even comprehend that these tools
can help me.
--
Axel.Thimm at ATrpms.net
10:33 a.m.
Axel Thimm wrote:
On Tue, Feb 26, 2008 at 05:33:33PM -0800, Toshio Kuratomi wrote:
> For a possible valid use, think of this potential pair of tools for
> students learning Japanese:
> /usr/bin/ひらがな2rōmaji
> /usr/bin/rōmaji2ひらがな
If I were going to learn Japanese tomorrow and were looking for such a
tool I wouldn't be actually able to even comprehend that these tools
can help me.
Sure. But that can happen regardless of the language. If I were trying
to avoid getting computer viruses how would I know that replacing
Windows with a *nix variant is an option? How do I know that I can
browse the web with "Firefox"? You need to know something about the
problem space before you can search for a tool to help you.
-Toshio
10:47 p.m.
On Tue, Feb 26, 2008 at 8:52 PM, seth vidal <skvidal(a)fedoraproject.org> wrote:
On Tue, 2008-02-26 at 23:09 +0100, Ralf Corsepius wrote:
> >
> > /me renames yum to ᶨᶬⱴ
> Is this Farsi, Arab or Hebrew?
It is 3 random chars pulled off of the gnome-character-map :)
The area that I am going to look forward to is when a package has what
looks like an ascii character but turns out to be a different UTF
version of that letter. The example I remember from class was where
the word nonny-nonny had 6 different 'types' of n. Looking at it where
they all showed up as \uBLAH was easy but in a completely UTF
compliant screen you couldnt just type it. Not that Fedora has to deal
with support tickets..
--
Stephen J Smoogen. -- CSIRT/Linux System Administrator
How far that little candle throws his beams! So shines a good deed
in a naughty world. = Shakespeare. "The Merchant of Venice"
10:05 a.m.
On Tue, Feb 26, 2008 at 09:47:39PM -0700, Stephen John Smoogen wrote:
On Tue, Feb 26, 2008 at 8:52 PM, seth vidal
<skvidal(a)fedoraproject.org> wrote:
>
> On Tue, 2008-02-26 at 23:09 +0100, Ralf Corsepius wrote:
> > >
> > > /me renames yum to ᶨᶬⱴ
> > Is this Farsi, Arab or Hebrew?
>
> It is 3 random chars pulled off of the gnome-character-map :)
>
The area that I am going to look forward to is when a package has what
looks like an ascii character but turns out to be a different UTF
version of that letter. The example I remember from class was where
the word nonny-nonny had 6 different 'types' of n. Looking at it where
they all showed up as \uBLAH was easy but in a completely UTF
compliant screen you couldnt just type it. Not that Fedora has to deal
with support tickets..
5+ years ago there was an example where this could be even used in a
malicious way to spoof URLs like microsoft.com with non-ASCII characters.
http://portal.acm.org/citation.cfm?id=503156
--
Axel.Thimm at ATrpms.net
1:48 p.m.
Le mardi 26 février 2008 à 13:49 -0500, Bill Nottingham a écrit :
Bill Nottingham (notting(a)redhat.com) said:
> The biggest issue I could see is that any issues in handling such a package
> would need to be fixed *everywhere* that package is deployed, whether it
> be current Fedora, earlier Fedora, or EPEL. It may not be practical to
> deploy such fixes everywhere.
Consider, for example, that such a package breaks bugzilla. Although I
suppose that's one way to avoid getting bug reports.
The latest bugzilla versions are supposed to be UTF-8 compatible.
Previous ones are just an encoding mess.
--
Nicolas Mailhot
4:29 p.m.
On Tue, 26 Feb 2008 20:48:11 +0100
Nicolas Mailhot <nicolas.mailhot(a)laposte.net> wrote:
The latest bugzilla versions are supposed to be UTF-8 compatible.
Previous ones are just an encoding mess.
Sure, but we're not running the latest one, yet.
--
Jesse Keating
Fedora -- All my bits are free, are yours?
12:51 p.m.
Bill Nottingham wrote :
Toshio Kuratomi (a.badger(a)gmail.com) said:
> Today the Packaging Committee began a discussion of whether package names
> should be allowed to contain the full range of Unicode characters (encoded
> as UTF-8) or be restricted to the ASCii subset.
>
> This is a bit of a contentious issue with the Packaging Committee members
> split but not yet set in stone about how to proceed.
>
> The main arguments seem to be:
> Pro Unicode:
> * Upstream knows best what name is most appropriate for their users. For us
> to change it locally in Fedora doesn't make much sense.
> * We allow Unicode in every other piece of the spec so why not in the
> package name?
> * We should be shaking out bugs in the handling of Unicode in our software
> rather than hiding issues with it.
>
> Pro ASCII:
> * Hard to type unicode package names, therefore it is a usability problem.
> * Is there a limit? Even if European letters are fine what about Kanji or
> Sanskrit?
> * Some pieces of software won't handle unicode package names and will need
> to be fixed.
The biggest issue I could see is that any issues in handling such a package
would need to be fixed *everywhere* that package is deployed, whether it
be current Fedora, earlier Fedora, or EPEL. It may not be practical to
deploy such fixes everywhere.
Not only where the packages are deployed, but also available from :
Some of the ftp/http mirror servers that will be serving the packages
might become a problem...
Matthias
--
Clean custom Red Hat Linux rpm packages : http://freshrpms.net/
Fedora release 8 (Werewolf) - Linux kernel 2.6.23.15-137.fc8
Load : 0.21 0.14 0.10
12:55 p.m.
On Tue, Feb 26, 2008 at 10:25:45AM -0800, Toshio Kuratomi wrote:
Pro ASCII:
* Hard to type unicode package names, therefore it is a usability problem.
* Is there a limit? Even if European letters are fine what about Kanji or
Sanskrit?
* Some pieces of software won't handle unicode package names and will need
to be fixed.
Also consider that package names define the file set that is to
bemirrored and you thus need to assume that all mirrors properly
support utf on their filesystems and their ftp/web/rsync servers.
If not you end up with changed package names on their way to the
consumer and non reproducable bug reports about yum not finding
packages (as the metadata will contain the proper UTF8 name while the
filesystem may present another).
So it's not about UTF8 deficiencies in any Fedora component, but
rather the (non) globality of UTF8.
--
Axel.Thimm at ATrpms.net
1:46 p.m.
Le mardi 26 février 2008 à 20:55 +0200, Axel Thimm a écrit :
On Tue, Feb 26, 2008 at 10:25:45AM -0800, Toshio Kuratomi wrote:
> Pro ASCII:
> * Hard to type unicode package names, therefore it is a usability problem.
The solution to this particular problem has already been suggested by
someone else: just have an ASCII transliteration alias as additional
Provides.
> * Some pieces of software won't handle unicode package names
and will need
> to be fixed.
That's too bad for them :p
Also consider that package names define the file set that is to
bemirrored and you thus need to assume that all mirrors properly
support utf on their filesystems and their ftp/web/rsync servers.
Is this really a problem? File systems do not interpret file names, so
they are going to store byte strings as they get them. The typical UTF-8
problems you get are apps that write invalid UTF-8 names because they
think they'll be interpreted as something else, or apps that display
UTF-8 names wrong because they think they've been written in another
encoding (and try to translate from this encoding to UTF-8), but
low-level mirroring tools are just going to reproduce file names as-is.
(that is unless you use one of the few non-unix/non-posix systems that
store filename encoding information in the filesystem, but those can not
import foreign filesystem data without the admin defining the encoding
to use anyway)
--
Nicolas Mailhot
3:19 p.m.
On Tue, Feb 26, 2008 at 08:46:15PM +0100, Nicolas Mailhot wrote:
> Also consider that package names define the file set that is to
> bemirrored and you thus need to assume that all mirrors properly
> support utf on their filesystems and their ftp/web/rsync servers.
Is this really a problem? File systems do not interpret file names, so
they are going to store byte strings as they get them.
Which is the problem. Since this is not any metadata you store
anywhere it is something defined locally. And if you mirror the raw
UTF8 filenames onto a mirror that has set local policy to be latin1
the web server will serve funny ~A names.
There are many issues with filename encoding otherwise there wouldn't
exist any tools like convmv which seems to be still widely used and
had an update release just a month ago:
http://www.j3e.de/linux/convmv/man/
BTW it quotes NFS4 and JFS as being encoding aware.
--
Axel.Thimm at ATrpms.net
3:58 p.m.
On Tue, 2008-02-26 at 23:19 +0200, Axel Thimm wrote:
Which is the problem. Since this is not any metadata you store
anywhere it is something defined locally. And if you mirror the raw
UTF8 filenames onto a mirror that has set local policy to be latin1
the web server will serve funny ~A names.
So are you claiming that mirrors can't be fixed ?
You know, it is very easy to test how many would break with an utf-8
name, just add somewhere an unused file with an utf-8 name and check if
any mirror breaks.
(not referring to you Axel from now on)
I think that resistance against standardizing around utf8 is getting
ridiculous at this point. I understand that people that never used
anything but ASCII may find it annoying that there are people out there
that use funny characters, but I wonder when they will realize that they
are the minority in the world, and that the others would like to be
treated like first class citizens like everybody else.
Unless there is a very compelling technical problem, I think we should
try as hard as possible to support and use by default utf8 everywhere.
The more we use it, the more we uncover bugs that we can hopefully fix.
And yes, I understand that a package name that is written in pure
Cyrillic or Chinese, or Japanese maybe hard initially to swallow (and
type), but how many chances are that everybody suddenly will start doing
that? Upstream is usually not stupid and understands that using non
ASCII characters may limit distribution. I guess only packages that make
sense only locally "may" end up with something like a completely
non-ASCII name.
If this is really a usability problem I am sure that the proposal to
provide an alternate ASCII only name (need rules to determine how you
get from non-ASCII to ASCII) is a very good one, even if copy&paste, or
other UIs can be used as well.
Simo.
--
Simo Sorce * Red Hat, Inc * New York
7:43 p.m.
On Tue, 2008-02-26 at 16:58 -0500, Simo Sorce wrote:
On Tue, 2008-02-26 at 23:19 +0200, Axel Thimm wrote:
And yes, I understand that a package name that is written in pure
Cyrillic or Chinese, or Japanese maybe hard initially to swallow (and
type),
Whether something is hard to type is a matter of the locale one is
using. It's relative to one's cultural background.
Have a look at the package currently under review:
écolier-courts-fonts
How many people would be able to type this package's name to install it?
Personal experience with my last name (contains an "é") tells me, the
majority of people with a French, Portuguese or Spanish language
background are able to do so. The majority of people outside of this
group (Noteworthy: The naive ASCII-typing/English-speaking world) isn't.
Similar considerations apply to any other arbitrary non-ASCII char.
but how many chances are that everybody suddenly will start doing
that?
Devil's advocate question: Why should a 16 year old student in an
arbitrary country not use his native language/charset for his package's
name?
That said, I would consider these chances to be increasing with the size
of the non-English speaking Linux user-base, who are not aware about the
historic and cultural issues of i18n.
If this is really a usability problem I am sure that the proposal to
provide an alternate ASCII only name (need rules to determine how you
get from non-ASCII to ASCII) is a very good one, even if copy&paste, or
other UIs can be used as well.
In bugzilla, I had proposed the opposite:
Mandate ASCII-only _package names_ (=> ASCII-only file names), but
additionally allow alternative (utf-8) "Provides" if desired.
This would allow GUI tools to display these utf-names, while it would
help keep things simple for command-line tools.
Ralf
1:26 p.m.
On Wednesday 27 February 2008, Ralf Corsepius wrote:
On Tue, 2008-02-26 at 16:58 -0500, Simo Sorce wrote:
> If this is really a usability problem I am sure that the
proposal to
> provide an alternate ASCII only name (need rules to determine how you
> get from non-ASCII to ASCII) is a very good one, even if copy&paste, or
> other UIs can be used as well.
In bugzilla, I had proposed the opposite:
Me too.
Mandate ASCII-only _package names_ (=> ASCII-only file names),
but
additionally allow alternative (utf-8) "Provides" if desired.
This still gets my +1. I have no problem with fixing infrastructure for more
UTF-8 awareness though.
This would allow GUI tools to display these utf-names, while it
would
help keep things simple for command-line tools.
I can't think of a use case where it would be desirable for GUI tools to
display anything else but the package's real name as its name. Dunno if this
is what you meant.
10:23 p.m.
On Wed, 2008-03-05 at 21:26 +0200, Ville Skyttä wrote:
On Wednesday 27 February 2008, Ralf Corsepius wrote:
> On Tue, 2008-02-26 at 16:58 -0500, Simo Sorce wrote:
> > If this is really a usability problem I am sure that the proposal to
> > provide an alternate ASCII only name (need rules to determine how you
> > get from non-ASCII to ASCII) is a very good one, even if copy&paste, or
> > other UIs can be used as well.
>
> In bugzilla, I had proposed the opposite:
Me too.
> Mandate ASCII-only _package names_ (=> ASCII-only file names), but
> additionally allow alternative (utf-8) "Provides" if desired.
This still gets my +1. I have no problem with fixing infrastructure for more
UTF-8 awareness though.
So do I.
> This would allow GUI tools to display these utf-names, while it
would
> help keep things simple for command-line tools.
I can't think of a use case where it would be desirable for GUI tools to
display anything else but the package's real name as its name.
Neither do I.
Dunno if this is what you meant.
What I meant was, to allow
people to install packages using their "real"
non-ASCII file names (e.g. yum install écollier-fonts).
As these people had proposed using installer GUIs, I had been referring
to GUI-tools in the sentence above.
Ralf
9:52 a.m.
On Tue, Feb 26, 2008 at 04:58:44PM -0500, Simo Sorce wrote:
On Tue, 2008-02-26 at 23:19 +0200, Axel Thimm wrote:
> Which is the problem. Since this is not any metadata you store
> anywhere it is something defined locally. And if you mirror the raw
> UTF8 filenames onto a mirror that has set local policy to be latin1
> the web server will serve funny ~A names.
So are you claiming that mirrors can't be fixed ?
No, anything, but my car can be fixed, but are you willing to take on
the task to support all admins current and future to properly migrate
to utf8 whether it's a Fedora/RHEL/Debian/Solaris OS of arbitrary age
running on the mirror?
I think that resistance against standardizing around utf8 is getting
ridiculous at this point. I understand that people that never used
anything but ASCII may find it annoying that there are people out there
that use funny characters, but I wonder when they will realize that they
are the minority in the world, and that the others would like to be
treated like first class citizens like everybody else.
Actually most people against using UTF8 are such that got bitten by
the encoding problems, e.g. living in a locale that actually uses
non-ASCII characters. I have had to fix too many times encoding errors
of say Übersetzungen.tex and �bersetzungen.tex to use the
transiliterated Uebersetzungen.tex.
Unless there is a very compelling technical problem, I think we
should
try as hard as possible to support and use by default utf8 everywhere.
The more we use it, the more we uncover bugs that we can hopefully fix.
And what will happen if I can't even read the package's name like say
an arabic version of libenese-fonts. Even if I can read libanese, if I
haven'z installed the font, the package will display as many empty
boxes to me to select from (among many other packages with empty
boxes). You run into a catch22 situation here.
--
Axel.Thimm at ATrpms.net
2:16 p.m.
On Tue, 2008-02-26 at 10:25 -0800, Toshio Kuratomi wrote:
Today the Packaging Committee began a discussion of whether package
names should be allowed to contain the full range of Unicode
characters
(encoded as UTF-8) or be restricted to the ASCii subset.
1. If the upstream package is an i18n name, how do you make up an ASCII
name for it?
2. What is the technical reason to restrict package names to ASCII?
NOTE: I do not consider broken apps a good reason, although I could
understand for example having to wait before allowing non-ASCII name to
when python is fixed, just to name a core package that may have
problems.
Simo.
--
Simo Sorce * Red Hat, Inc * New York
8:08 p.m.
On Tue, 2008-02-26 at 10:25 -0800, Toshio Kuratomi wrote:
Pro ASCII:
* Hard to type unicode package names, therefore it is a usability problem.
* Is there a limit? Even if European letters are fine what about Kanji
or Sanskrit?
Japanese package names would really be odd here. Would we spell the
package name with its kanji or its phonetic (e.g., hiragana) reading?
For example, say there were a package called 「勉強」 (Rōmaji:
"benkyoo", English: study) which had flash-cards or some helpful
studying software. Would we name this package by this Kanji, or its
hiragana equivalent
「べんきょう」? Would we require the package to have Provides for this
kana reading if named in Kanji, and vice-versa? What about
transliterations (so-called "Rōmaji"): What transliteration system [1]
should we use?
If we do require the Provides, what if two packages end up being
different kanji names that are homophones (read the same, phonetically)?
One example that comes to mind is between 花 and 華 (both flower) and 鼻
(nose), all read as "hana" (hiragana: はな)? For even more fun, 神
(god), 紙 (paper), and 髮 (hair) all have readings of "kami" (かみ). And
extending this to kanji compounds will yield even further enjoyment:
明日 (tomorrow) can be read as "asu" (あす) or "ashita" (あした), and 昨
日 (yesterday) can be read as "kinoo" (きのう) or "sakujitsu" (さくじ
つ) depending on formality.
I suppose it would be similar for other languages based on both phonetic
and logographic scripts, but I use Japanese as my example since that's
what I'm attempting to learn currently. :)
What about misc technical characters too - arrows (← → ↑ ↓) or the like?
This can get quite overwhelming if we're not very careful.
In closing, I think it would be best to limit this to diacritic/accented
characters. With an additional transliterated Provides, the ease case
would be satisified, without the complexities provided by such writing
systems as above.
[1] http://en.wikipedia.org/wiki/Rōmaji#Modern_systems
(And as a completely tangential aside, SCIM is *awesome*.)
--
Peter Gordon (codergeek42)
GnuPG Public Key ID: 0xFFC19479 / Fingerprint:
DD68 A414 56BD 6368 D957 9666 4268 CB7A FFC1 9479
8:52 p.m.
Peter Gordon wrote:
On Tue, 2008-02-26 at 10:25 -0800, Toshio Kuratomi wrote:
> Pro ASCII:
> * Hard to type unicode package names, therefore it is a usability problem.
> * Is there a limit? Even if European letters are fine what about Kanji
> or Sanskrit?
Japanese package names would really be odd here. Would we spell the
package name with its kanji or its phonetic (e.g., hiragana) reading?
For example, say there were a package called 「勉強」 (Rōmaji:
"benkyoo", English: study) which had flash-cards or some helpful
studying software. Would we name this package by this Kanji, or its
hiragana equivalent
「べんきょう」? Would we require the package to have Provides for this
kana reading if named in Kanji, and vice-versa? What about
transliterations (so-called "Rōmaji"): What transliteration system [1]
should we use?
If we do require the Provides, what if two packages end up being
different kanji names that are homophones (read the same, phonetically)?
One example that comes to mind is between 花 and 華 (both flower) and 鼻
(nose), all read as "hana" (hiragana: はな)? For even more fun, 神
(god), 紙 (paper), and 髮 (hair) all have readings of "kami" (かみ). And
extending this to kanji compounds will yield even further enjoyment:
明日 (tomorrow) can be read as "asu" (あす) or "ashita" (あした), and 昨
日 (yesterday) can be read as "kinoo" (きのう) or "sakujitsu" (さくじ
つ) depending on formality.
I suppose it would be similar for other languages based on both phonetic
and logographic scripts, but I use Japanese as my example since that's
what I'm attempting to learn currently. :)
What about misc technical characters too - arrows (← → ↑ ↓) or the like?
This can get quite overwhelming if we're not very careful.
Well, there was ☠ for a while but that looks to be pretty dead upstream.
I'm sure that there will be more at some point, though.
In closing, I think it would be best to limit this to
diacritic/accented
characters. With an additional transliterated Provides, the ease case
would be satisified, without the complexities provided by such writing
systems as above.
You do a wonderful job of explaining what's wrong with us trying to
adjust upstream's name to be ASCII but I just want to be certain we're
on the same page by the end:
Package names should follow upstream since attempting to transliterate
or translate upstream names can't be done sanely on our side. For
things that map easily into the ASCii set (diacritic/accented
characters, for instance, as found in latin-1) a transliterated Provides
can be added to make installation easier for ASCii-conditioned users but
carrying this on to other scripts is a losing proposition.
Thanks,
-Toshio
10:05 p.m.
On Tue, 2008-02-26 at 18:52 -0800, Toshio Kuratomi wrote:
You do a wonderful job of explaining what's wrong with us trying
to
adjust upstream's name to be ASCII but I just want to be certain we're
on the same page by the end:
Package names should follow upstream since attempting to transliterate
or translate upstream names can't be done sanely on our side. For
things that map easily into the ASCii set (diacritic/accented
characters, for instance, as found in latin-1) a transliterated Provides
can be added to make installation easier for ASCii-conditioned users but
carrying this on to other scripts is a losing proposition.
That's exactly what I was trying to explain, yes. Sorry for the rambling
if it seemed that way.
Rethinking this though, I feel that accented and diacritic-related
characters (á, ë, õ, ñ, æ, et al.) would be quite suitable but we would
have to pay attention to the language of origin. For our example here,
the transliterations would probably be simply removing the diacritic as
appropriate: a, e, o, n, and ae.
In some languages, though, the diacritic differentiates the character
from the "plain" form. For example, a Spanish package name for a similar
studying software could be "¡Estudiará!" (third-person indicative future
tense; literally, "You will study!"). However, we would need to be
careful here because, without that accent, this changes the conjugation
to "estudiara," which is the first- and third-person imperfect
subjunctive (which really makes no sense on its own, since the
subjunctive tense is meant to be used in a subjective or predictive
clause of a sentence, such as referring to one's wants and desires for
the future).
Also, transliterating ñ if we know the source language to be Spanish (or
other similar languages) would change it a bit, since it's actually
pronounced as "ny-": "Ño" would be pronounced as "nyo";
"ñat" would be
"nyat" and so on. (Not that these are actual words, but they are parts
thereof.) If we allowed accents and whatnot, maybe we should restrict
the transliteration to simply removing all diacritics and splitting
"merged" letters (æ --> ae, etc.). That way we would not have to worry
too much more.
(Again, using Spanish as an example due to familiarity.)
However, trying to force things into ASCII or even some variation of
roman-alphabet text is going to be extremely difficult and quite
confusing for languages for which that is not the norm.
I just though of another example for my previous message. (Jee, I seem
to love these. Apologies...) What happens when the package name is only
Han characters? Do we transliterate it to the Japanese (Kanji) reading,
or the Korean (Hanja) reading, or the native Chinese reading? :P
But I digress. Thanks for your time.
--
Peter Gordon (codergeek42)
GnuPG Public Key ID: 0xFFC19479 / Fingerprint:
DD68 A414 56BD 6368 D957 9666 4268 CB7A FFC1 9479
3:24 a.m.
On Tue, Feb 26, 2008 at 08:05:01PM -0800, Peter Gordon wrote:
In some languages, though, the diacritic differentiates the character
from the "plain" form. For example, a Spanish package name for a similar
studying software could be "¡Estudiará!" (third-person indicative future
tense; literally, "You will study!"). However, we would need to be
careful here because, without that accent, this changes the conjugation
to "estudiara," which is the first- and third-person imperfect
subjunctive (which really makes no sense on its own, since the
subjunctive tense is meant to be used in a subjective or predictive
clause of a sentence, such as referring to one's wants and desires for
the future).
If wikipedia is right in http://en.wikipedia.org/wiki/Transliteration
what you are trying to do is not transliteration (in a narrow sense),
but transcription. Transliterated word are not necessarily pronounced
the same. There is an automatic mapping between characters only,
irrespective of the correctness of the result in the original language
(the character mapping is in general based on characters similitude or
sounds in english when it comes to transliteration in ASCII 7 bit).
I think that we should not permit non ASCII 7 bit letters in package
names, but the transliteration or transcription scheme used should be
left to the packager.
We can have aids, still, for packagers who don't have an idea on how to
transliterate an upstream name. For example in texi2html/texinfo we
transliterate non ascii characters in file names. There are tables in
texinfo for some characters, and I use Text::Unidecode to complete in
texi2html. That way the file names are ascii 7 bit and are unlikely to
be problematic in any platform (but the portability issue is more severe
than in fedora, since we want these file names to be usable everywhere).
This scheme seems to work for a lot of characters -- though maybe better
schemes could be devised, if needed. But I don't think this would be
needed, instead left to the packager.
--
Pat
10:10 a.m.
On Tue, Feb 26, 2008 at 06:52:24PM -0800, Toshio Kuratomi wrote:
Package names should follow upstream since attempting to
transliterate or
translate upstream names can't be done sanely on our side. For things that
map easily into the ASCii set (diacritic/accented characters, for instance,
as found in latin-1) a transliterated Provides can be added to make
installation easier for ASCii-conditioned users but carrying this on to
other scripts is a losing proposition.
We violate this rule with capitalization,
python/perl/php/.... packages and what not and we'll stick with an
upstream's name that seems to want to ship its project only is certain
locales?
I would instead propose a rule that says "if the transliteration to
ASCII can't be done by the packager, he should contact upstream to
provide one and use that".
--
Axel.Thimm at ATrpms.net
11 a.m.
Le Mer 27 février 2008 17:10, Axel Thimm a écrit :
We violate this rule with capitalization,
python/perl/php/.... packages and what not and we'll stick with an
upstream's name that seems to want to ship its project only is certain
locales?
I would instead propose a rule that says "if the transliteration to
ASCII can't be done by the packager, he should contact upstream to
provide one and use that".
Nevertheless even if a transliteration can be found (and I agree with
you asking upstream is best as transliteration rules vary from country
to country and some like China even had different transliteration
styles at different points of time) that's no reason to restrict the
main package name (that is displayed in packagekit and various tools)
to this transliteration.
Most of us can understand SMS-speak. SMS-speak has various properties
that make it easier to type in some conditions. Would we like someone
to change our name to its SMS variant to make it easier on SMS users?
I guess not.
Transliteration should be strictly limited to a failover for users
that for some reason use the CLI and can not type the upstream name of
a package they need.
Capitalization is something else entirely. Capitalization is common
editorial work that makes a whole more pleasant by applying the same
conventions everywhere. It's a neutral transformation. Transliteration
to ASCII OTOH is a very violent transform, and probably offensive to
an upstream that chose an non-ASCII name.
And as Simo I don't buy the argument that since it's broken, it must
not be fixed.
--
Nicolas Mailhot
1:04 p.m.
Axel Thimm wrote:
On Tue, Feb 26, 2008 at 06:52:24PM -0800, Toshio Kuratomi wrote:
> Package names should follow upstream since attempting to transliterate or
> translate upstream names can't be done sanely on our side. For things that
> map easily into the ASCii set (diacritic/accented characters, for instance,
> as found in latin-1) a transliterated Provides can be added to make
> installation easier for ASCii-conditioned users but carrying this on to
> other scripts is a losing proposition.
We violate this rule with capitalization,
python/perl/php/.... packages and what not and we'll stick with an
upstream's name that seems to want to ship its project only is certain
locales?
You raise some good points. Why do we change upstream WRT
capitalization? Probably usability. What will we do if capitalization
matters? (ie: foobar and FooBar are separate projects) Not approve a
package that has a conflicting name and try to get either or both
projects to rename upstream. If the packages weren't changed upstream
(because, says upstream, FooBar and foobar are plainly different names)
what would we do? This is an unknown that seems very relevant here.
OTOH, there is a single straightforward, non-controversial mapping from
uppercase ASCii to lowercase ASCii. Transliteration and transcription
from other scripts is not so blessed. So the rationale for wanting to
ban Unicode could be the same as wanting to ban capitalization but the
ramifications are very different.
Changing module names is interesting. Speaking for python we do two things:
1) We use "import foo" to determine what the package's upstream name is
rather than the name of the tarball. I don't consider this changing the
name as there are several equally valid ways to determine what the
upstream name is so we've just standardized on one.
2) We prepend "python-" to the name in most cases. This is partially
categorization and partially a namespacing issue. Categorization is
changing the name to make it easier for users to recognize the purpose
of a package. python-turboflot doesn't need the "python-" for anything
other than making *people* aware that turboflot is a python module.
Namespacing is a valid issue, though. python-json != php-json !=
perl-json even if they all use the equivalent of "import json" and
distributed their binaries in tarballs called json-1.0.tar.gz.
I think justifying banning unicode with the module namespace as a
precedent is illogical. If anything, converting unicode characters in a
user's chosen script to a ASCii-ization is removing a means of
categorizing the package by sight. It will also lead to more namespace
collisions rather than less.
I would instead propose a rule that says "if the transliteration
to
ASCII can't be done by the packager, he should contact upstream to
provide one and use that".
If you said that upstream should always be in charge of transliterating
I think this rule would be better. To use an example that people might
know from non-computer life what if upstream named their package 北京?
One distribution might feel perfectly confident transliterating that as
beijing while another one uses peking. Having upstream manage
transliteration pushes the decision to the correct level to coordinate
and avoid confusion.
-Toshio
3:34 p.m.
On Wed, Feb 27, 2008 at 11:04:31AM -0800, Toshio Kuratomi wrote:
> I would instead propose a rule that says "if the
transliteration to
> ASCII can't be done by the packager, he should contact upstream to
> provide one and use that".
>
If you said that upstream should always be in charge of transliterating I
think this rule would be better.
I just want to cover the case where upstream doesn't care and doesn't
reply to a packager's question on how to name the package. "Always in
charge" would mean that if there is no feedback there is no name.
To use an example that people might know from non-computer life what
if upstream named their package 北京? One distribution might feel
perfectly confident transliterating that as beijing while another
one uses peking. Having upstream manage transliteration pushes the
decision to the correct level to coordinate and avoid confusion.
"Ask upstream and if it doesn't respond take the best guess of the
packager"?
--
Axel.Thimm at ATrpms.net
9:51 p.m.
On Tue, 2008-02-26 at 10:25 -0800, Toshio Kuratomi wrote:
(09:38:59 AM) abadger1999: Yum bug:
https://bugzilla.redhat.com/show_bug.cgi?id=261961
(09:39:01 AM) buggbot: Bug 261961: low, medium, ---, Jeremy Katz, NEW ,
Yum does not like non-ascii package names
I was talking with Tim about this some today since he recently applied a
patch for a similar (and somewhat related) problem. We need to make sure
we catch it in a number of places but it is something we want to do.
there's no institutional resistance to fixing this.
-sv
5888
days inactive
5897
days old
packaging@lists.fedoraproject.org
32 comments
14 participants
participants (14)
-
Axel Thimm -
Bill Nottingham -
Dominik 'Rathann' Mierzejewski -
Jesse Keating -
Matthias Saou -
Nicolas Mailhot -
Patrice Dumas -
Peter Gordon -
Ralf Corsepius -
Seth Vidal -
Simo Sorce -
Stephen John Smoogen -
Toshio Kuratomi -
Ville Skyttä