[Bug 485566] New: Fontconfig should select DejaVu font for rendering English, even in Japanese locale.

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

Summary: Fontconfig should select DejaVu font for rendering English, even in Japanese locale.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

Summary: Fontconfig should select DejaVu font for rendering
                    English, even in Japanese locale.
           Product: Fedora
           Version: 10
          Platform: All
        OS/Version: Linux
            Status: NEW
          Severity: high
          Priority: medium
         Component: fontconfig
        AssignedTo: [e-mail]
        ReportedBy: [e-mail]
         QAContact: [e-mail]
                CC: [e-mail], [e-mail]
    Classification: Fedora

Description of problem:

As you know, fontconfig finds proper fonts for text rendering, 
depending on locales of process and text to render.
It works properly in the most cases, but there are some sever cases.

For example, if you type only "@" or "}" in gedit, 
it uses glyph of VLGothic in Japanese locale.
However, if you type "a" after the first character, 
it changes one's mind to choose glyphs of DejaVu fonts.
It's very annoying for Japanese users.

In the worst cases, the width of the indentations of sourcecode seems
different.
(As the width of the " " glyphs are differ between VLGothic and DejaVu)

How reproducible:

Steps to Reproduce:
1. Login X11 as a Japanese.
2. Open gedit and change the configuration to use "Monospace".
3. Type "@" on the first line, and "@a" on the first line.

Actual results:
"@" seems differ between the two lines.

Expected results:
"@" seems the same between the two lines.
The same glyph should be used for rendering the two "@".

[Bug 485566] Fontconfig should select DejaVu font for rendering English, even in Japanese locale.

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

--- Comment #1 from Ryo Dairiki   2009-02-14 09:34:23 EDT ---
Created an attachment (id=331924)
 --> (https://bugzilla.redhat.com/attachment.cgi?id=331924)
A new configuration file to force fontconfig to prefer DejaVu glyphs to
Japanese glyphs.

This configuration make fontconfig prefer DejaVu glyphs to Japanese glyphs in
rendering English.
When you apply this, fontconfig uses DejaVu glyphs for rendering "@" in
Japanese locale.
Note, this configuration will not affect on font selection order for Japanese
characters.

[Bug 485566] Fontconfig should select DejaVu font for rendering English, even in Japanese locale.

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

Ryo Dairiki  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
 Attachment #331924|0                           |1
        is obsolete|                            |

--- Comment #2 from Ryo Dairiki   2009-02-14 10:24:22 EDT ---
Created an attachment (id=331925)
 --> (https://bugzilla.redhat.com/attachment.cgi?id=331925)
A new configuration file to force fontconfig to prefer DejaVu glyphs to
Japanese glyphs.

A new configuration file to force fontconfig to prefer DejaVu glyphs to
Japanese glyphs.
After installing this configuration, fontconfig uses DejaVu glyph as far as
possible 
even in Japanese locale.

[Bug 485566] Fontconfig should select DejaVu font for rendering English, even in Japanese locale.

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

Akihiro Nomura  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #3 from Akihiro Nomura   2009-04-28 23:44:11 EDT ---
I tried this patch to vlgothic-* fonts, into
/usr/share/fontconfig/conf.avail/66-vlgothic-*.conf.
This patch seems to resolve following issues (or make them to original state),
which have been occurred since new fontconfig rules applied to VLGothic*.
1. In monospace, "@" glyph is rendered sometimes in vlgothic or sometimes in
dejavu mono and
   their width is different. (This bug)
2. Some English alphabet and numbers in the applets in gnome-panel looks like
monospace and hard to read.
   I don't know why this occur and is resolved by this patch.
3. Some ambiguous width character (like □) in gnome-terminal is rendered as
fill-width but treated as half-width,
   I mentioned this issue in Bug #487061.

[Bug 485566] Fontconfig should select DejaVu font for rendering English, even in Japanese locale.

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

--- Comment #4 from Jens Petersen   2009-07-13 19:02:35 EDT ---
I think we need someone in fedora-i18n to help get these kinds of fixes in?

[Bug 485566] Fontconfig should select DejaVu font for rendering English, even in Japanese locale.

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

--- Comment #5 from Nicolas Mailhot   2009-07-14 04:40:44 EDT ---
This is not really a fix, this is an ugly hack to workaround fontconfig default
behaviour. It only works in very specific instances, when specific fonts are
installed, and will probably have nasty side-effects somewhere.

The correct fix is either to get upstream to drop latin glyphs from VLGothic,
or enhance fontconfig so you can blacklist parts of a font (as people have been
requesting for years).

I'm sure Behdad would appreciate help in doing 2. though

[Bug 485566] Fontconfig should select DejaVu font for rendering English, even in Japanese locale.

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

--- Comment #6 from Nicolas Mailhot   2009-07-14 04:52:16 EDT ---
Also, apart from those local fixes, the core of the problem is pango has
usually no idea what language you're trying to type, and tries to guess from
what you're typing. Sometimes it guesses wrong, or changes its mind when you
type more text that gaves it more info.

To put an end to this class of bugs (which are not limited to VLGothic or
Japanese) you need a language switcher applet that, in addition to managing
input methods, tells explicitely apps what's being typed.

[Bug 485566] Fontconfig should select DejaVu font for rendering English, even in Japanese locale.

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

--- Comment #7 from Akira TAGOH   2009-07-14 05:44:35 EDT ---
(In reply to comment #5)
> The correct fix is either to get upstream to drop latin glyphs from VLGothic,

I'm opposed to this idea. nothing wrong in VLGothic and it introduces another
bug in applications using any other rendering/multilingual framework libraries
doesn't supports a fallback fonts. like Bug #434753.

(In reply to comment #6)
> Also, apart from those local fixes, the core of the problem is pango has
> usually no idea what language you're trying to type, and tries to guess from
> what you're typing.

Does it? though Pango sets a default PangoLanguage from current locale when
it's starting. why doesn't Pango just has a priority to pick up the font
according to that?
> To put an end to this class of bugs (which are not limited to VLGothic or
> Japanese) you need a language switcher applet that, in addition to managing
> input methods, tells explicitely apps what's being typed.

If we want any framework other than locale system to determine the input
language etc, I think it may be better adding a kind of feature to the input
method right rather than having new one and IM running at the same time.

[Bug 485566] Fontconfig should select DejaVu font for rendering English, even in Japanese locale.

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

--- Comment #8 from Nicolas Mailhot   2009-07-14 06:46:39 EDT ---
(In reply to comment #7)
> (In reply to comment #5)
> > The correct fix is either to get upstream to drop latin glyphs from VLGothic,
>
> I'm opposed to this idea. nothing wrong in VLGothic and it introduces another
> bug in applications using any other rendering/multilingual framework libraries
> doesn't supports a fallback fonts. like Bug #434753.

Anything that does not use fontconfig nowadays with its built-in font
substitution is a bug period. We barely have the energy to get our main font
system sort of working without having to work with others.
> (In reply to comment #6)
> > Also, apart from those local fixes, the core of the problem is pango has
> > usually no idea what language you're trying to type, and tries to guess from
> > what you're typing.
>
> Does it? though Pango sets a default PangoLanguage from current locale when
> it's starting. why doesn't Pango just has a priority to pick up the font
> according to that?

Because there is no strict equivalence between the language of your desktop
(the locale) and the language you are typing (what pango needs to process). It
is very common to type in a foreign language, and anyway in lots of regions
people use the en_US locale because theirs is not available or the translations
are incomplete or bad
> > To put an end to this class of bugs (which are not limited to VLGothic or
> > Japanese) you need a language switcher applet that, in addition to managing
> > input methods, tells explicitely apps what's being typed.
>
> If we want any framework other than locale system to determine the input
> language etc, I think it may be better adding a kind of feature to the input
> method right rather than having new one and IM running at the same time.

That assumes there is one true IM system that works for all languages. So far
that hasn't been the case. And even if there was one system, you need to
distinguish between the input method selection and the language selection
because there is no 1:1 relashionship between them (any latin layout will be
able to type English even if it's not an English layout, that's probably also
true for russian and cyrillic layouts, etc)

Accurate language selection gets you more accurate text rendering, correct
spellchecker selection, etc

[Bug 485566] Fontconfig should select DejaVu font for rendering English, even in Japanese locale.

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

fujiwara  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #9 from fujiwara   2009-07-14 22:55:43 EDT ---
> A new configuration file to force fontconfig to prefer DejaVu glyphs to

Japanese glyphs.

Since the Japanese font has the latin script, should it always use the Japanese
glyphs instead of the DejaVu glyphs?
So I have thought the reverse way to fix this.

The following patch fixes this issue and I integrated it in the previous
company.
http://src.opensolaris.org/source/xref/jds/spec-files/trunk/patches/pango-03...

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

Jens Petersen  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|10                          |rawhide
            Summary|Fontconfig should select    |[CJK] Fontconfig should
                   |DejaVu font for rendering   |select DejaVu font for
                   |English, even in Japanese   |rendering ASCII glyphs
                   |locale.                     |

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

Jens Petersen  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #10 from Jens Petersen   2009-10-19 04:09:59 EDT ---
*** Bug 529594 has been marked as a duplicate of this bug. ***

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

--- Comment #11 from Jens Petersen   2009-10-19 04:15:22 EDT ---
Some simple testcases from bug 529594:

LANG=zh_CN.UTF-8 pango-view --font="Monospace" <(echo -e "my@ip-1-2-3 ~$\n  @@
1 2 3")

LANG=ja_JP.UTF-8 pango-view --font="Monospace" <(echo -e "my@ip-1-2-3 ~$\n  @@
1 2 3")

LANG=ko_KR.UTF-8 pango-view --font="Monospace" <(echo -e "my@ip-1-2-3 ~$\n  @@
1 2 3")

also good without --font option.

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

--- Comment #13 from Bug Zapper  2010-11-04 07:31:03 EDT ---

This message is a reminder that Fedora 12 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 12.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '12'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 12's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 12 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

Akira TAGOH  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|12                          |14
               Flag|needinfo?(ryo-dairiki@users |
                   |.sourceforge.net)           |

--- Comment #14 from Akira TAGOH  2010-11-18 08:35:46 EST ---
This is still present in even f14.

I'm a bit thinking of a feature to get rid of the specific code point from the
charset tables for the certain font cache like:

<match target="scan">
  <test name="family">
    <string>VL Gothic</string>
  </test>
  <edit name="charset" mode="remove">
    <range>
      <string>U+0021</string>
      <string>U+00FF</string>
    </range>
  </edit>
</match>

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

Akira TAGOH  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
    External Bug ID|                            |FreeDesktop.org 31969

--- Comment #15 from Akira TAGOH  2010-12-06 01:51:35 EST ---
proposed changes for this issue:
http://bugs.freedesktop.org/show_bug.cgi?id=31969

we also need to update the config file for vlgothic-fonts.

<match target="scan">
  <test name="family">
    <string>VL Gothic</string>
  </test>
  <edit name="charset" mode="assign">
    <minus>
      <name>charset</name>
      <range>
        <int>0x0021</int>
        <int>0x00FF</int>
      </range>
    </minus>
  </edit>
</match>

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

--- Comment #16 from Akira TAGOH  2011-01-07 05:16:16 EST ---
Any chance to get a snapshot or new release of fontconfig for the above
upstream bug?

I could remove ASCII characters from charset in vlgothic-fonts perhaps then.

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

--- Comment #17 from Behdad Esfahbod  2011-01-07 20:28:20 EST ---
I'll try.

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

--- Comment #18 from Akira TAGOH  2011-01-27 02:51:35 EST ---
That would be nice if we can get new version of fontconfig in f15 alpha or
earlier. then we can work for some fixes in the font packages with it by alpha
or beta.

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

Akira TAGOH  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|14                          |rawhide

--- Comment #19 from Akira TAGOH  2011-11-21 03:06:19 EST ---
still waiting for new feature in fontconfig not yet released.

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

Akira TAGOH  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #20 from Akira TAGOH  2011-12-09 00:44:04 EST ---
*** Bug 765675 has been marked as a duplicate of this bug. ***

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

Fedora Admin XMLRPC Client  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|[e-mail]    |[e-mail]

--- Comment #21 from Fedora Admin XMLRPC Client  2012-01-10 10:42:39 EST ---
This package has changed ownership in the Fedora Package Database.  Reassigning
to the new owner of this component.

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug.

https://bugzilla.redhat.com/show_bug.cgi?id=485566

Fedora Admin XMLRPC Client  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|[e-mail]          |[e-mail]

--- Comment #22 from Fedora Admin XMLRPC Client  2012-03-21 21:37:51 EDT ---
This package has changed ownership in the Fedora Package Database.  Reassigning
to the new owner of this component.

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=485566

Peng Wu  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #23 from Peng Wu  ---
Recently I also en-countered this bug in Chinese locale.

And I find a work around that if I customize the font to some English font in
gedit preference dialog, the problems are gone.
For more details in http://pwu.fedorapeople.org/pango/README.

And I tried several tweaks in Chinese fontconfig confs to simulate the above
changes.
See http://pwu.fedorapeople.org/pango/.

Then I spend a long time on investigating it, and finally find its reasons.
See http://pwu.fedorapeople.org/pango/REASONS.

How about we restrict this change only for monospace font.

PS:
The problem is shown in http://pwu.fedorapeople.org/pango/before%20change.png.
And the fixes is shown in http://pwu.fedorapeople.org/pango/after%20change.png.

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=485566

--- Comment #24 from Akira TAGOH  ---
Sure. that is actually same to what being proposed at comment#2 and
disadvantages on this way is:

* if applications doesn't use any rendering library which doesn't support the
fallback mechanism, you'll miss Chinese characters on display.

* it may still gives you different looks when the default Latin font will not
be DejaVu anymore and need to fix it yourself.

just to clarify, my suggestion at comment#15 also has a disadvantage...

* if applications doesn't use any rendering library which doesn't support the
fallback mechanism, one will miss ASCII characters on display as it can be seen
with Lohit now.

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=485566

--- Comment #25 from Peng Wu  ---
As this bug can be work around by manually choose "DejaVu Sans Mono" font in
gedit preference dialog, I wrote a small fontconfig file to simulate this
change, see 32-dejavu.conf.

How about we add one fontconfig file to the default Latin font?

URL: http://pwu.fedorapeople.org/pango/32-dejavu.conf

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=485566

--- Comment #26 from Peng Wu  ---
I also write a small script to allow user to disable this feature, if he meets
some problems with this new fontconfig conf file.

See http://pwu.fedorapeople.org/pango/dejavuset

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

Product: Fedora
https://bugzilla.redhat.com/show_bug.cgi?id=485566

Peng Wu  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://bugzilla.redhat.com
                   |                            |/show_bug.cgi?id=919861

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=485566

--- Comment #28 from Fedora End Of Life  ---
This message is a notice that Fedora 19 is now at end of life. Fedora 
has stopped maintaining and issuing updates for Fedora 19. It is 
Fedora's policy to close all bug reports from releases that are no 
longer maintained. Approximately 4 (four) weeks from now this bug will
be closed as EOL if it remains open with a Fedora 'version' of '19'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 19 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

[Bug 485566] [CJK] Fontconfig should select DejaVu font for rendering ASCII glyphs

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=485566

Akira TAGOH  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |FutureFeature
            Version|19                          |rawhide

Red Hat Bugzilla  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Doc Type|Bug Fix                     |Enhancement