[rhel7-branch] Rebase our language/locale handling code

[Vratislav Podzimek]

There has been major changes of the language/locale handling code done in the
F20 cycle which significantly improve user experience of the installer and also
make our code cleaner, more deterministic and better documented.

On behalf of the bug #1006458 [1] all these 14 patches should be ported from
master to rhel7-branch. They contain a lot of changes, but they are just ports
of the master patches, so they should be okay.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1006458

Vratislav Podzimek (14):
  Rewrite the localization module
  Adapt to the new localization module
  Fix the langcode parsing regexp
  Specify also query script when getting locale's native name
  Remove an unused argument of get_available_translations
  Allow seting up locale without modifying ksdata
  Rework the Welcome spoke to allow users choose from all locales
  Get rid of the non-deterministic expand_langs and its usage
  Match langs with stripped accents when filtering languages
  Rework the Langsupport spoke to work with all locales
  Share code between the Welcome and Langsupport spokes
  More robust parsing of the layout and variant string specification
  Normalize keyboard layout and variant strings from langtable
  Center the Langsupport spoke's description

anaconda                                           |  38 +-
 anaconda.spec.in                                   |   3 +-
 pyanaconda/keyboard.py                             |  59 +-
 pyanaconda/localization.py                         | 716 ++++++++++-----------
 pyanaconda/packaging/yumpayload.py                 |   9 +-
 pyanaconda/timezone.py                             |  36 +-
 pyanaconda/ui/gui/hubs/progress.py                 |  53 +-
 pyanaconda/ui/gui/spokes/langsupport.glade         | 172 +++--
 pyanaconda/ui/gui/spokes/langsupport.py            | 204 +++---
 .../ui/gui/spokes/lib/lang_locale_handler.py       | 161 +++++
 pyanaconda/ui/gui/spokes/welcome.glade             | 336 +++++-----
 pyanaconda/ui/gui/spokes/welcome.py                | 260 +++-----
 pyanaconda/ui/gui/utils.py                         |  40 ++
 13 files changed, 1108 insertions(+), 979 deletions(-)
 create mode 100644 pyanaconda/ui/gui/spokes/lib/lang_locale_handler.py

[PATCH 01/14] Rewrite the localization module

[Vratislav Podzimek]

We can now use the langtable package for a lot of things and the old version of
the module was very cryptic and too complex for what we need of it.

Related: rhbz#1006458
(cherry-picked aabed26dfcb8e57b0750afe4706ae728841df231 from master)

Signed-off-by: Vratislav Podzimek 
---
 anaconda.spec.in           |   3 +-
 pyanaconda/localization.py | 619 +++++++++++++++++++--------------------------
 2 files changed, 267 insertions(+), 355 deletions(-)

diff --git a/anaconda.spec.in b/anaconda.spec.in
index 5383ab6..615337b 100644
--- a/anaconda.spec.in
+++ b/anaconda.spec.in
@@ -40,7 +40,7 @@ Source0: %{name}-%{version}.tar.bz2
 %define iscsiver 6.2.0.870-3
 %define rpmver 4.10.0
 %define libarchivever 3.0.4
-%define langtablever 0.0.5-1
+%define langtablever 0.0.7-1

BuildRequires: audit-libs-devel
 BuildRequires: gettext >= %{gettextver}
@@ -92,7 +92,6 @@ Requires: yum >= %{yumver}
 Requires: python-urlgrabber >= %{pythonurlgrabberver}
 Requires: system-logos
 Requires: pykickstart >= %{pykickstartver}
-Requires: python-babel
 Requires: langtable-data >= %{langtablever}
 Requires: langtable-python >= %{langtablever}
 Requires: libuser-python
diff --git a/pyanaconda/localization.py b/pyanaconda/localization.py
index f9d0037..3611c87 100644
--- a/pyanaconda/localization.py
+++ b/pyanaconda/localization.py
@@ -20,28 +20,22 @@
 #                    Vratislav Podzimek 
 #

-from collections import defaultdict, deque
 import gettext
-import locale as locale_mod
 import os
 import re
-import logging
-log = logging.getLogger("anaconda")
-
-import babel
 import langtable
+import glob

-from pyanaconda.constants import DEFAULT_LANG
-
-LOCALE_PREFERENCES = {}
+import logging
+log = logging.getLogger("anaconda")

LOCALE_CONF_FILE_PATH = "/etc/locale.conf"

#e.g. 'SR_RS.UTF-8@latin'
 LANGCODE_RE = re.compile(r'(?P<language>[A-Za-z]+)'
                          r'(_(?P<territory>[A-Za-z]+))?'
-                         r'(.(?P<codeset>[-\w]+))?'
-                         r'(@(?P<modifier>[-\w]+))?')
+                         r'(.(?P<encoding>[-\w]+))?'
+                         r'(@(?P<script>[-\w]+))?')

class LocalizationConfigError(Exception):
     """Exception class for localization configuration related problems"""
@@ -53,178 +47,26 @@ class InvalidLocaleSpec(LocalizationConfigError):

pass

-class LocaleInfo(object):
+def _upcase_first_letter(string):
     """
-    Class unifying the glibc-like and CLDR-like locales by providing a single
-    API based on properties.
-
-    """
-
-    def __init__(self, localespec):
-        """
-        Constructor for the LocaleInfo class.
-
-        :param localespec: locale string such as "sr_RS.UTF-8@latin"
-                           (glibc-like format) or "sr_Latn_RS" (CLDR-format)
-        :type localespec: str
-
-        """
-
-        # get the basic values
-        parsed_values = parse_langcode(localespec)
-        if parsed_values:
-            # successfully parsed -> a glibc-like locale, try to get some more
-            # information by querying langtable
-
-            # language must be specified
-            language = parsed_values["language"]
-            territory = parsed_values.get("territory", "")
-            script = parsed_values.get("modifier", "")
-
-            locales = langtable.list_locales(languageId=language,
-                                             territoryId=territory,
-                                             scriptId=script)
-        else:
-            # not in the glibc-like format, use langtable to get it
-            locales = langtable.list_locales(languageId=localespec)
-
-        if not locales:
-            msg = "'%s' is not a valid locale" % localespec
-            raise InvalidLocaleSpec(msg)
-
-        # locales are sorted from the most probable to the least probable
-        self._locale = locales[0]
-
-        parsed_values = parse_langcode(self._locale)
-
-        # language must be specified
-        self._language = parsed_values["language"]
-
-        # try to get the other values
-        self._territory = parsed_values.get("territory", "")
-        self._encoding = parsed_values.get("codeset", "")
-        self._script = parsed_values.get("modifier", "")
-
-        # TODO: never appeared so far, needs to be parsed and stored?
-        self._variant = None
-
-    @property
-    def language(self):
-        return self._language
-
-    @property
-    def territory(self):
-        return self._territory
-
-    @property
-    def script(self):
-        return self._script
-
-    @property
-    def variant(self):
-        return self._variant
-
-    @property
-    def encoding(self):
-        return self._encoding
-
-    @property
-    def locale(self):
-        return self._locale
-
-    @property
-    def english_name(self):
-        name = langtable.language_name(languageId=self.language,
-                                       territoryId=self.territory,
-                                       scriptId=self.script,
-                                       languageIdQuery="en")
-        # some languages don't start with a capital letter
-        return name.title()
-
-    @property
-    def display_name(self):
-        name = langtable.language_name(languageId=self.language,
-                                       territoryId=self.territory,
-                                       scriptId=self.script)
-        # some languages don't start with a capital letter
-        return name.title()
-
-    @property
-    def short_name(self):
-        return self.__repr__()
-
-    def __repr__(self):
-        formatstr = '{0.language}'
-        if self.territory is not None:
-            formatstr += '_{0.territory}'
-        if self.encoding:
-            formatstr += '.{0.encoding}'
-        if self.script:
-            formatstr += '@{0.script}'
-        if self.variant is not None:
-            formatstr += '#{0.variant}'
-
-        return formatstr.format(self)
-
-    def __str__(self):
-        return self.english_name.encode('ascii', 'replace')
-
-    def __unicode__(self):
-        return self.english_name
-
-    def __eq__(self, other):
-        return repr(self) == repr(other)
-
-# XXX this should probably be somewhere else
-def partition(seq, func=bool, func_range=(True, False)):
-    buffers = dict(((x, deque()) for x in func_range))
-
-    def values(x, seq=iter(seq)):
-        while True:
-            while not buffers[x]:
-                item = seq.next()
-                buffers[func(item)].append(item)
-
-            yield buffers[x].popleft()
-
-    return tuple(values(x) for x in func_range)
-
-
-def get_all_locales():
-    localeset = set()
-    for localename in sorted(babel.localedata.list()):
-        try:
-            locale = LocaleInfo(localename)
-        except InvalidLocaleSpec:
-            continue
-
-        if repr(locale) not in localeset:
-            localeset.add(repr(locale))
-            yield locale
+    Helper function that upcases the first letter of the string. Python's
+    standard string.capitalize() not only upcases the first letter but also
+    lowercases all the others. string.title() capitalizes all words in the
+    string.

+    :type string: either a str or unicode object
+    :return: the given string with the first letter upcased
+    :rtype: str or unicode (depends on the input)

-def get_available_translations(domain=None, localedir=None):
-    localeset = set()
-    domain = domain or gettext._current_domain
-    localedir = localedir or gettext._default_localedir
-
-    langs = babel.localedata.list()
-    messagefiles = gettext.find(domain, localedir, langs, all=True)
-    languages = [path.split(os.path.sep)[-3] for path in messagefiles]
-
-    # usually there are no message files for en
-    if 'en' not in languages:
-        languages.append('en')
-
-    for langcode in languages:
-        try:
-            locale = LocaleInfo(langcode)
-        except InvalidLocaleSpec:
-            continue
+    """

-        if repr(locale) not in localeset:
-            localeset.add(repr(locale))
-            yield locale
+    if not string:
+        # cannot change anything
+        return string
+    elif len(string) == 1:
+        return string.upper()
+    else:
+        return string[0].upper() + string[1:]

def parse_langcode(langcode):
     """
@@ -265,7 +107,7 @@ def expand_langs(astring):
         return list(langs)

base, loc, enc, script = [lang_dict[key] for key in ("language",
-                                      "territory", "codeset", "modifier")]
+                                      "territory", "encoding", "script")]

if not base:
         return list(langs)
@@ -288,6 +130,194 @@ def expand_langs(astring):

return list(langs)

+def is_supported_locale(locale):
+    """
+    Function that tells if the given locale is supported by the Anaconda or
+    not. We consider locales supported by the langtable as supported by the
+    Anaconda.
+
+    :param locale: locale to test
+    :type locale: str
+    :return: whether the given locale is supported or not
+    :rtype: bool
+    :raise InvalidLocaleSpec: if an invalid locale is given (see LANGCODE_RE)
+
+    """
+
+    en_name = get_english_name(locale)
+    return bool(en_name)
+
+def setup_locale(locale, lang):
+    """
+    Procedure setting the system to use the given locale and store it in to the
+    ksdata.lang object. DOES NOT PERFORM ANY CHECKS OF THE GIVEN LOCALE.
+
+    :param locale: locale to setup
+    :type locale: str
+    :param lang: ksdata.lang object
+    :return: None
+    :rtype: None
+
+    """
+
+    lang.lang = locale
+    os.environ["LANG"] = locale
+
+def get_english_name(locale):
+    """
+    Function returning english name for the given locale.
+
+    :param locale: locale to return english name for
+    :type locale: str
+    :return: english name for the locale or empty string if unknown
+    :rtype: st
+    :raise InvalidLocaleSpec: if an invalid locale is given (see LANGCODE_RE)
+
+    """
+
+    parts = parse_langcode(locale)
+    if "language" not in parts:
+        raise InvalidLocaleSpec("'%s' is not a valid locale" % locale)
+
+    name = langtable.language_name(languageId=parts["language"],
+                                   territoryId=parts.get("territory", ""),
+                                   scriptId=parts.get("script", ""),
+                                   languageIdQuery="en")
+
+    return _upcase_first_letter(name)
+
+def get_native_name(locale):
+    """
+    Function returning native name for the given locale.
+
+    :param locale: locale to return native name for
+    :type locale: str
+    :return: english name for the locale or empty string if unknown
+    :rtype: st
+    :raise InvalidLocaleSpec: if an invalid locale is given (see LANGCODE_RE)
+
+    """
+
+    parts = parse_langcode(locale)
+    if "language" not in parts:
+        raise InvalidLocaleSpec("'%s' is not a valid locale" % locale)
+
+    name = langtable.language_name(languageId=parts["language"],
+                                   territoryId=parts.get("territory", ""),
+                                   scriptId=parts.get("script", ""),
+                                   languageIdQuery=parts["language"])
+
+    return _upcase_first_letter(name)
+
+def get_available_translations(domain=None, localedir=None):
+    """
+    Method that generates (i.e. returns a generator) available translations for
+    the given domain and localedir.
+
+    :type domain: str
+    :type localedir: str
+    :return: generator yielding available translations
+    :rtype: generator yielding strings
+
+    """
+
+    domain = domain or gettext._current_domain
+    localedir = localedir or gettext._default_localedir
+
+    messagefiles = sorted(glob.glob(localedir + "/*/LC_MESSAGES/anaconda.mo"))
+    trans_gen = (path.split(os.path.sep)[-3] for path in messagefiles)
+
+    # usually there are no message files for en
+    langs = {"en"}
+    yield "en_US.UTF-8"
+
+    for trans in trans_gen:
+        parts = parse_langcode(trans)
+        lang = parts.get("language", "")
+        if lang and lang not in langs:
+            langs.add(lang)
+            locales = get_language_locales(lang)
+            if not locales:
+                continue
+
+            # take the first locale (with highest rank) for the language
+            yield locales[0]
+
+def get_language_locales(lang):
+    """
+    Function returning all locales available for the given language.
+
+    :param lang: language to get available locales for
+    :type lang: str
+    :return: a list of available locales
+    :rtype: list of strings
+    :raise InvalidLocaleSpec: if an invalid locale is given (see LANGCODE_RE)
+
+    """
+
+    parts = parse_langcode(lang)
+    if "language" not in parts:
+        raise InvalidLocaleSpec("'%s' is not a valid language" % lang)
+
+    return langtable.list_locales(languageId=parts["language"],
+                                  territoryId=parts.get("territory", ""),
+                                  scriptId=parts.get("script", ""))
+
+def get_territory_locales(territory):
+    """
+    Function returning list of locales for the given territory. The list is
+    sorted from the most probable locale to the least probable one (based on
+    langtable's ranking.
+
+    :param territory: territory to return locales for
+    :type territory: str
+    :return: list of locales
+    :rtype: list of strings
+
+    """
+
+    return langtable.list_locales(territoryId=territory)
+
+def get_locale_keyboards(locale):
+    """
+    Function returning preferred keyboard layouts for the given locale.
+
+    :param locale: locale string (see LANGCODE_RE)
+    :type locale: str
+    :return: list of preferred keyboard layouts
+    :rtype: list of strings
+    :raise InvalidLocaleSpec: if an invalid locale is given (see LANGCODE_RE)
+
+    """
+
+    parts = parse_langcode(locale)
+    if "language" not in parts:
+        raise InvalidLocaleSpec("'%s' is not a valid locale" % locale)
+
+    return langtable.list_keyboards(languageId=parts["language"],
+                                    territoryId=parts.get("territory", ""),
+                                    scriptId=parts.get("script", ""))
+
+def get_locale_timezones(locale):
+    """
+    Function returning preferred timezones for the given locale.
+
+    :param locale: locale string (see LANGCODE_RE)
+    :type locale: str
+    :return: list of preferred timezones
+    :rtype: list of strings
+    :raise InvalidLocaleSpec: if an invalid locale is given (see LANGCODE_RE)
+
+    """
+
+    parts = parse_langcode(locale)
+    if "language" not in parts:
+        raise InvalidLocaleSpec("'%s' is not a valid locale" % locale)
+
+    return langtable.list_timezones(languageId=parts["language"],
+                                    territoryId=parts.get("territory", ""),
+                                    scriptId=parts.get("script", ""))
+
 def write_language_configuration(lang, root):
     """
     Write language configuration to the $root/etc/locale.conf file.
@@ -305,176 +335,59 @@ def write_language_configuration(lang, root):
         msg = "Cannot write language configuration file: %s" % ioerr.strerror
         raise LocalizationConfigError(msg)

-class PreferredLocale(object):
-
-    @staticmethod
-    def from_language(language):
-        locales = defaultdict(set)
-        for locale in get_all_locales():
-            locales[repr(locale)].add(locale)
-            locales[locale.language].add(locale)
-
-        return PreferredLocale(locales[language])
-
-    @staticmethod
-    def from_territory(territory):
-        locales = defaultdict(set)
-        for locale in get_all_locales():
-            locales[locale.territory].add(locale)
-
-        return PreferredLocale(locales[territory])
-
-    def __init__(self, localeset):
-        self._localedict = {repr(locale):locale for locale in localeset}
-
-    def get_all_locales(self, preferences=None):
-        if preferences is None:
-            preferences = []
-        preferences = filter(self._localedict.__contains__, preferences)
-        inside, outside = partition(self._localedict.keys(),
-                                    func=lambda x: x in preferences)
-        sorted_locales = [self._localedict[localename]
-                          for localename in list(inside) + list(outside)]
-        return sorted_locales
-
-    def get_preferred_locale(self, preferences=None):
-        if preferences is None:
-            preferences = []
-        try:
-            return self.get_all_locales(preferences)[0]
-        except IndexError:
-            return None
-
-
-class Language(object):
-
-    def __init__(self, preferences=None, territory=None):
-        if preferences is None:
-            preferences = {}
-
-        self.translations = {repr(locale):locale
-                             for locale in get_available_translations()}
-        self.locales = {repr(locale):locale for locale in get_all_locales()}
-
-        self.system_lang = self._get_firmware_language()
-        self.install_lang = self.system_lang
-        self.preferred_translation = self.translations[self.system_lang]
-        self.preferred_locales = [self.locales[self.system_lang]]
-        self.preferred_locale = self.preferred_locales[0]
-
-        self.all_preferences = preferences
-        self.preferences = self.all_preferences.get(territory, [])
-        self.territory = territory
-        if self.territory:
-            self._get_preferred_translation_and_locales()
-
-    def _get_firmware_language(self):
-        try:
-            n = "/sys/firmware/efi/efivars/PlatformLang-8be4df61-93ca-11d2-aa0d-00e098032b8c"
-            d = open(n, 'r', 0).read()
-        except:
-            return DEFAULT_LANG
-
-        try:
-            # the contents of the file are:
-            # 4-bytes of attribute data that we don't care about
-            # NUL terminated ASCII string like 'en-US'.
-            if len(d) < 10:
-                log.debug("PlatformLang was too short")
-                raise ValueError
-            d = d[4:]
-            if d[2] != '-':
-                log.debug("PlatformLang was malformed")
-                raise ValueError
-
-            # they use - and we use _, so fix it...
-            d = d[:2] + '_' + d[3:-1]
-
-            # UEFI 2.3.1 Errata C specifies 2 aliases in common use that
-            # aren't part of RFC 4646, but are allowed in PlatformLang.
-            # Because why make anything simple?
-            if d.startswith('zh_chs'):
-                d = 'zh_Hans'
-            elif d.startswith('zh_cht'):
-                d = 'zh_Hant'
-            d += '.UTF-8'
-
-            if not self.translations.has_key(d):
-                log.debug("PlatformLang was "%s", which is unsupported." % d)
-                raise ValueError
-            log.debug("Using UEFI PlatformLang "%s" as our language." % d)
-            return d
-        except ValueError:
-            return DEFAULT_LANG
-
-    def _get_preferred_translation_and_locales(self):
-        # get locales from territory
-        locales_from_territory = PreferredLocale.from_territory(self.territory)
-        all_locales = locales_from_territory.get_all_locales(self.preferences)
-
-        # get preferred translation
-        for locale in all_locales:
-            if locale.language in self.translations:
-                self.preferred_translation = self.translations[locale.language]
-                break
-
-        for locale in all_locales:
-            if locale.short_name in self.translations:
-                self.preferred_translation = self.translations[locale.short_name]
-                break
-
-        self.preferred_locales = all_locales
-
-    def select_translation(self, translation):
-        translation = self.translations[translation]
-        self.preferences.extend(self.all_preferences.get(translation.language, []))
-
-        # get locales from translation
-        locales_from_language = PreferredLocale.from_language(translation.short_name)
-        all_locales = locales_from_language.get_all_locales(self.preferences)
-
-        # get preferred locale
-        for locale in all_locales:
-            if locale in self.preferred_locales:
-                self.preferred_locale = locale
-                break
-        else:
-            try:
-                self.preferred_locale = all_locales[0]
-            except IndexError:
-                self.preferred_locale = self.preferred_locales[0]
-
-        # add the preferred locale to the beginning of locales
-        if self.preferred_locale in self.preferred_locales:
-            self.preferred_locales.remove(self.preferred_locale)
-        self.preferred_locales.insert(0, self.preferred_locale)
-
-        # if territory is not set, use the one from preferred locale
-        self.territory = self.territory or self.preferred_locale.territory
-
-    @property
-    def install_lang_as_dict(self):
-        parse_langcode(self.install_lang)
-
-    @property
-    def system_lang_as_dict(self):
-        parse_langcode(self.system_lang)
-
-    def set_install_lang(self, langcode):
-        self.install_lang = langcode
-
-        os.environ['LANG'] = langcode
-        os.environ['LC_NUMERIC'] = 'C'
-
-        try:
-            locale_mod.setlocale(locale_mod.LC_ALL, '')
-        except locale_mod.Error:
-            pass
-
-        # XXX this is the sort of thing which you should never do,
-        # but we switch languages at runtime and thus need to invalidate
-        # the set of languages/mofiles which gettext knows about
-        gettext._translations = {}
-
-    def set_system_lang(self, langcode):
-        self.system_lang = langcode
+def load_firmware_language(lang):
+    """
+    Procedure that loads firmware language information (if any). It stores the
+    information in the given ksdata.lang object and sets the $LANG environment
+    variable.
+
+    :param lang: ksdata.lang object
+    :return: None
+    :rtype: None
+
+    """
+
+    if lang.lang and lang.seen:
+        # set in kickstart, do not override
+        return
+
+    try:
+        n = "/sys/firmware/efi/efivars/PlatformLang-8be4df61-93ca-11d2-aa0d-00e098032b8c"
+        d = open(n, 'r', 0).read()
+    except:
+        return
+
+    # the contents of the file are:
+    # 4-bytes of attribute data that we don't care about
+    # NUL terminated ASCII string like 'en-US'.
+    if len(d) < 10:
+        log.debug("PlatformLang was too short")
+        return
+    d = d[4:]
+    if d[2] != '-':
+        log.debug("PlatformLang was malformed")
+        return
+
+    # they use - and we use _, so fix it...
+    d = d[:2] + '_' + d[3:-1]
+
+    # UEFI 2.3.1 Errata C specifies 2 aliases in common use that
+    # aren't part of RFC 4646, but are allowed in PlatformLang.
+    # Because why make anything simple?
+    if d.startswith('zh_chs'):
+        d = 'zh_Hans'
+    elif d.startswith('zh_cht'):
+        d = 'zh_Hant'
+    d += '.UTF-8'
+
+    if not is_supported_locale(d):
+        log.debug("PlatformLang was '%s', which is unsupported." % d)
+        return
+
+    locales = get_language_locales(d)
+    if not locales:
+        log.debug("No locales found for the PlatformLang '%s'." % d)
+        return
+
+    log.debug("Using UEFI PlatformLang '%s' ('%s') as our language." % (d, locales[0]))
+    setup_locale(locales[0], lang)

[PATCH 02/14] Adapt to the new localization module

[Vratislav Podzimek]

Things get different but much simpler with the new version of the module.

Related: rhbz#1006458
(cherry-picked f90ece6aa09cfe42fc93982ddd2c7a9f3c0cfdce from master)

Signed-off-by: Vratislav Podzimek 
---
 anaconda                                | 38 +++++------------
 pyanaconda/localization.py              | 18 ++++++++
 pyanaconda/timezone.py                  | 36 ++++------------
 pyanaconda/ui/gui/spokes/langsupport.py | 52 ++++++++++++-----------
 pyanaconda/ui/gui/spokes/welcome.py     | 73 ++++++++++++---------------------
 5 files changed, 90 insertions(+), 127 deletions(-)

diff --git a/anaconda b/anaconda
index dad8a7b..f61422a 100755
--- a/anaconda
+++ b/anaconda
@@ -954,32 +954,15 @@ if __name__ == "__main__":
         else:
             log.error("Unknown method: %s", (anaconda.methodstr,))

+    from pyanaconda import localization
     # Set the language before loading an interface, when it may be too late.
     if opts.lang:
-        from pyanaconda.localization import Language, LOCALE_PREFERENCES, expand_langs
-
-        langObj = Language(LOCALE_PREFERENCES, territory=None)
-
-        # Given something other than the long format we prefer?  We need to
-        # dig through supported translations to figure out what the user
-        # meant.
-        foundLang = False
-        if not opts.lang in langObj.translations:
-
-            for trans in langObj.translations.keys():
-                if opts.lang in expand_langs(trans):
-                    opts.lang = trans
-                    foundLang = True
-                    break
-
-            if not foundLang:
-                opts.lang = constants.DEFAULT_LANG
-
-        langObj.set_install_lang(opts.lang)
-        ksdata.lang.lang = opts.lang
-        # report if a language specified by the user
-        # or a default was used
-        ksdata.lang.seen = foundLang
+        locales = localization.get_language_locales(opts.lang)
+        if locales:
+            localization.setup_locale(locales[0], ksdata.lang)
+            ksdata.lang.seen = True
+        else:
+            log.error("Invalid locale '%s' given on command line" % opts.lang)

import blivet
     blivet.enable_installer_mode()
@@ -1024,10 +1007,6 @@ if __name__ == "__main__":
                                          exception.test_exception_handling())
     signal.signal(signal.SIGUSR2, lambda signum, frame: anaconda.dumpState())

-    if opts.lang:
-        # this is lame, but make things match what we expect (#443408)
-        opts.lang = opts.lang.replace(".utf8", ".UTF-8")
-
     from blivet import storageInitialize
     from pyanaconda.packaging import payloadInitialize
     from pyanaconda.network import networkInitialize, wait_for_connecting_NM_thread
@@ -1084,6 +1063,9 @@ if __name__ == "__main__":
         if not anaconda.ksdata.timezone.nontp:
             iutil.start_service("chronyd")

+    # try to load firmware language
+    localization.load_firmware_language(ksdata.lang)
+
     # FIXME:  This will need to be made cleaner once this file starts to take
     # shape with the new UI code.
     anaconda._intf.setup(ksdata)
diff --git a/pyanaconda/localization.py b/pyanaconda/localization.py
index 3611c87..08a9ec6 100644
--- a/pyanaconda/localization.py
+++ b/pyanaconda/localization.py
@@ -318,6 +318,24 @@ def get_locale_timezones(locale):
                                     territoryId=parts.get("territory", ""),
                                     scriptId=parts.get("script", ""))

+def get_locale_territory(locale):
+    """
+    Function returning locale's territory.
+
+    :param locale: locale string (see LANGCODE_RE)
+    :type locale: str
+    :return: territory or None
+    :rtype: str or None
+    :raise InvalidLocaleSpec: if an invalid locale is given (see LANGCODE_RE)
+
+    """
+
+    parts = parse_langcode(locale)
+    if "language" not in parts:
+        raise InvalidLocaleSpec("'%s' is not a valid locale" % locale)
+
+    return parts.get("territory", None)
+
 def write_language_configuration(lang, root):
     """
     Write language configuration to the $root/etc/locale.conf file.
diff --git a/pyanaconda/timezone.py b/pyanaconda/timezone.py
index 1dea80b..5dca6fc 100644
--- a/pyanaconda/timezone.py
+++ b/pyanaconda/timezone.py
@@ -26,9 +26,9 @@ configuration, valid timezones recognition etc.

import os
 import pytz
+import langtable
 from collections import OrderedDict

-from pyanaconda import localization
 from pyanaconda import iutil
 from pyanaconda.constants import THREAD_STORAGE
 from pyanaconda.threads import threadMgr
@@ -157,42 +157,24 @@ def save_hw_clock(timezone):
     iutil.execWithRedirect(cmd, args)

-def get_all_territory_timezones(territory):
-    """
-    Return the list of timezones for a given territory.
-
-    :param territory: either localization.LocaleInfo or territory
-
-    """
-
-    if isinstance(territory, localization.LocaleInfo):
-        territory = territory.territory
-
-    try:
-        timezones = pytz.country_timezones(territory)
-    except KeyError:
-        timezones = list()
-
-    timezones = [zone.encode("utf-8") for zone in timezones]
-    return timezones
-
-
 def get_preferred_timezone(territory):
     """
     Get the preferred timezone for a given territory. Note that this function
     simply returns the first timezone in the list of timezones for a given
     territory.

-    :param territory: either localization.LocaleInfo or territory
+    :param territory: territory to get preferred timezone for
+    :type territory: str
+    :return: preferred timezone for the given territory or None if no found
+    :rtype: str or None

"""

-    try:
-        timezone = get_all_territory_timezones(territory)[0]
-    except IndexError:
-        timezone = None
+    timezones = langtable.list_timezones(territoryId=territory)
+    if not timezones:
+        return None

-    return timezone
+    return timezones[0]

def get_all_regions_and_timezones():
     """
diff --git a/pyanaconda/ui/gui/spokes/langsupport.py b/pyanaconda/ui/gui/spokes/langsupport.py
index b071c01..66dbad2 100644
--- a/pyanaconda/ui/gui/spokes/langsupport.py
+++ b/pyanaconda/ui/gui/spokes/langsupport.py
@@ -19,12 +19,13 @@
 # Red Hat Author(s): Radek Vykydal 
 #

+# pylint: disable-msg=E0611
 from gi.repository import Gtk, Pango
 from pyanaconda.flags import flags
 from pyanaconda.i18n import _, N_
 from pyanaconda.ui.gui.spokes import NormalSpoke
 from pyanaconda.ui.gui.categories.localization import LocalizationCategory
-from pyanaconda.localization import Language, LOCALE_PREFERENCES, expand_langs
+from pyanaconda import localization

import re

@@ -65,14 +66,16 @@ class LangsupportSpoke(NormalSpoke):
             renderer = self.builder.get_object(rend)
             column.set_cell_data_func(renderer, self._mark_selected_bold, idx)

-        language = Language(LOCALE_PREFERENCES, territory=None)
         # source of lang code <-> UI name mapping
-        self.locale_infos_for_data = language.translations
-        self.locale_infos_for_ui = language.translations
-
-        for code, info in sorted(self.locale_infos_for_ui.items()):
-            self._add_language(self._langsupportStore, info.display_name,
-                               info.english_name, info.short_name,
+        # (localization.get_available_translations() returns a generator)
+        self.locale_infos_for_data = list(localization.get_available_translations())
+        self.locale_infos_for_ui = self.locale_infos_for_data[:]
+
+        for locale in sorted(self.locale_infos_for_ui):
+            self._add_language(self._langsupportStore,
+                               localization.get_native_name(locale),
+                               localization.get_english_name(locale),
+                               locale,
                                False, True)

self._select_language(self._langsupportStore, self.data.lang.lang)
@@ -85,23 +88,22 @@ class LangsupportSpoke(NormalSpoke):
     def refresh(self):
         self._langsupportEntry.set_text("")
         lang_infos = self._find_localeinfos_for_code(self.data.lang.lang, self.locale_infos_for_ui)
-        lang_short_names = [info.short_name for info in lang_infos]
-        if len(lang_short_names) > 1:
+        if len(lang_infos) > 1:
             log.warning("Found multiple locales for lang %s: %s, picking first" %
-                        (self.data.lang.lang, lang_short_names))
+                        (self.data.lang.lang, lang_infos))
         # Just take the first found
         # TODO - for corner cases choose the one that is common prefix
-        lang_short_names = lang_short_names[:1]
+        lang_infos = lang_infos[:1]

-        addsupp_short_names = []
+        addsupp_infos = []
         for code in self.data.lang.addsupport:
             code_infos = self._find_localeinfos_for_code(code, self.locale_infos_for_ui)
-            addsupp_short_names.extend(info.short_name for info in code_infos)
+            addsupp_infos.extend(code_infos)

for row in self._langsupportStore:
-            if row[COL_LANG_SETTING] in addsupp_short_names:
+            if row[COL_LANG_SETTING] in addsupp_infos:
                 row[COL_SELECTED] = True
-            if row[COL_LANG_SETTING] in lang_short_names:
+            if row[COL_LANG_SETTING] in lang_infos:
                 row[COL_SELECTED] = True
                 row[COL_IS_ADDITIONAL] = False

@@ -117,7 +119,7 @@ class LangsupportSpoke(NormalSpoke):
             for info in self._find_localeinfos_for_code(code, self.locale_infos_for_data):
                 if info not in infos:
                     infos.append(info)
-        return ", ".join(info.english_name for info in infos)
+        return ", ".join(localization.get_english_name(info) for info in infos)

@property
     def mandatory(self):
@@ -128,13 +130,13 @@ class LangsupportSpoke(NormalSpoke):
         return True

def _find_localeinfos_for_code(self, code, infos):
-        try:
-            retval = [infos[code]]
-        except KeyError:
-            retval = [info for _code, info in infos.items()
-                      if code in expand_langs(_code)]
-            log.debug("locale info found for %s: %s" % (code, retval))
-        return retval
+        if code in infos:
+            return [code]
+        else:
+            retval = [info for info in infos
+                      if code in localization.expand_langs(info)]
+            log.debug("locale infos found for %s: %s" % (code, retval))
+            return retval

def _add_language(self, store, native, english, setting, selected, additional):
         store.append(['<span lang="%s">%s</span>' % (re.sub(r'..*', '', setting), native),
@@ -142,7 +144,7 @@ class LangsupportSpoke(NormalSpoke):

def _select_language(self, store, language):
         itr = store.get_iter_first()
-        while itr and language not in expand_langs(store[itr][COL_LANG_SETTING]):
+        while itr and language not in localization.expand_langs(store[itr][COL_LANG_SETTING]):
             itr = store.iter_next(itr)

# If we were provided with an unsupported language, just use the default.
diff --git a/pyanaconda/ui/gui/spokes/welcome.py b/pyanaconda/ui/gui/spokes/welcome.py
index 18c5109..5d982a4 100644
--- a/pyanaconda/ui/gui/spokes/welcome.py
+++ b/pyanaconda/ui/gui/spokes/welcome.py
@@ -28,7 +28,7 @@ from pyanaconda.ui.gui.hubs.summary import SummaryHub
 from pyanaconda.ui.gui.spokes import StandaloneSpoke
 from pyanaconda.ui.gui.utils import enlightbox

-from pyanaconda.localization import Language, LOCALE_PREFERENCES, expand_langs
+from pyanaconda import localization
 from pyanaconda.product import distributionText, isFinal, productName, productVersion
 from pyanaconda import keyboard
 from pyanaconda import timezone
@@ -59,9 +59,8 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
         selected = self.builder.get_object("languageViewSelection")
         (store, itr) = selected.get_selected()

-        lang = store[itr][2]
-        self.language.select_translation(lang)
-        self.data.lang.lang = lang
+        locale = store[itr][2]
+        localization.setup_locale(locale, self.data.lang)

# Skip timezone and keyboard default setting for kickstart installs.
         # The user may have provided these values via kickstart and if not, we
@@ -69,22 +68,18 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
         if flags.flags.automatedInstall:
             return

-        lang_timezone = None
-        # check if the geolocation lookup returned a time zone
-        # (the geolocation module makes sure that the returned timezone is
-        # either a valid timezone or None)
         geoloc_timezone = geoloc.get_timezone()
+        loc_timezones = localization.get_locale_timezones(self.data.lang.lang)
         if geoloc_timezone:
-            lang_timezone = geoloc_timezone
-        # if no data is provided by Geolocation,
-        # try to get timezone from the current language
-        elif self.language.territory and not self.data.timezone.timezone:
-            lang_timezone = timezone.get_preferred_timezone(self.language.territory)
-
-        if lang_timezone:
-            self.data.timezone.timezone = lang_timezone
-
-        lang_country = self.language.preferred_locale.territory
+            # (the geolocation module makes sure that the returned timezone is
+            # either a valid timezone or None)
+            self.data.timezone.timezone = geoloc_timezone
+        elif loc_timezones and not self.data.timezone.timezone:
+            # no data is provided by Geolocation, try to get timezone from the
+            # current language
+            self.data.timezone.timezone = loc_timezones[0]
+
+        lang_country = localization.get_locale_territory(self.data.lang.lang)
         self._set_keyboard_defaults(store[itr][1], lang_country)

def _set_keyboard_defaults(self, lang_name, country):
@@ -158,24 +153,17 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
         # We can use the territory from geolocation here
         # to preselect the translation, when it's available.
         territory = geoloc.get_territory_code()
-        self.language = Language(LOCALE_PREFERENCES, territory=territory)
-
-        # check if there is one and only one locale for the territory
-        if len(self.language.preferred_locales) != 1:
-            log.info("Didn't get a single locale from Geolocation,"
-                        " falling back to default locale.")
-            self.language = Language(LOCALE_PREFERENCES, territory=None)
-            # Explanation:
-            # Some territories have multiple locales,
-            # for example, the Switzerland has:
-            # de_CH, it_CH and fr_CH
-            # As there is no clear order of preference for them,
-            # it is safer to just fall back to the default locale
+
+        locales = localization.get_territory_locales(territory)
+        if locales and not (self.data.lang.lang and self.data.lang.seen):
+            # get something from the GeoIP lookup and not set in/on the
+            # kickstart/command line
+            localization.setup_locale(locales[0], self.data.lang)

# fill the list with available translations
-        for _code, trans in sorted(self.language.translations.items()):
-            self._addLanguage(store, trans.display_name,
-                              trans.english_name, trans.short_name)
+        for locale in localization.get_available_translations():
+            self._addLanguage(store, localization.get_native_name(locale),
+                              localization.get_english_name(locale), locale)

# Move the default language (whatever was provided on the command line,
         # or by kickstart, or by geoip, or English if nothing else) to the top
@@ -183,16 +171,7 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
         # dropped into the middle of a scrollable list.
         (store, itr) = self._selection.get_selected()
         if not itr:
-            # check if a language was set by kickstart
-            # NOTE: seen means the language was "seen" in
-            # kickstart or boot option, so it overrides
-            # the language detected by geolocation
-            if self.data.lang.lang and self.data.lang.seen:
-                lang = self.data.lang.lang
-            else:
-                lang = self.language.preferred_translation.short_name
-
-            itr = self._selectLanguage(lang)
+            itr = self._selectLanguage(self.data.lang.lang)

# store is the filtered store, and itr is an iter on it.  We need to
         # convert to an iter on the underlying store.
@@ -290,7 +269,8 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
         itr = store.get_iter_first()
         # store[itr][3] is True if this row is a separator in the view, so we
         # want to skip those.
-        while itr and not store[itr][3] and language not in expand_langs(store[itr][2]):
+        while itr and not store[itr][3] \
+                and language not in localization.expand_langs(store[itr][2]):
             itr = store.iter_next(itr)

# If we were provided with an unsupported language, just use the default.
@@ -326,8 +306,7 @@ class WelcomeLanguageSpoke(StandaloneSpoke):

if selected:
             lang = store[selected[0]][2]
-            self.language.set_install_lang(lang)
-            self.language.set_system_lang(lang)
+            localization.setup_locale(lang, self.data.lang)
             self.retranslate(lang)

def on_clear_icon_clicked(self, entry, icon_pos, event):

[PATCH 03/14] Fix the langcode parsing regexp

[Vratislav Podzimek]

Obviously \w matches the underscore. Also fix the docstring of the
parse_langcode function.

Related: rhbz#1006458
(cherry-picked 34b493ec13d97e54a235c9910afff295d508e674 from master)

Signed-off-by: Vratislav Podzimek 
---
 pyanaconda/localization.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/pyanaconda/localization.py b/pyanaconda/localization.py
index 08a9ec6..fe6229f 100644
--- a/pyanaconda/localization.py
+++ b/pyanaconda/localization.py
@@ -34,8 +34,8 @@ LOCALE_CONF_FILE_PATH = "/etc/locale.conf"
 #e.g. 'SR_RS.UTF-8@latin'
 LANGCODE_RE = re.compile(r'(?P<language>[A-Za-z]+)'
                          r'(_(?P<territory>[A-Za-z]+))?'
-                         r'(.(?P<encoding>[-\w]+))?'
-                         r'(@(?P<script>[-\w]+))?')
+                         r'(.(?P<encoding>[-A-Za-z0-9]+))?'
+                         r'(@(?P<script>[-A-Za-z0-9]+))?')

class LocalizationConfigError(Exception):
     """Exception class for localization configuration related problems"""
@@ -75,8 +75,8 @@ def parse_langcode(langcode):

'language' : 'SR'
     'territory' : 'RS'
-    'codeset' : 'UTF-8'
-    'modifier' : 'latin'
+    'encoding' : 'UTF-8'
+    'script' : 'latin'

or None if the given string doesn't match the LANGCODE_RE.

[PATCH 04/14] Specify also query script when getting locale's native name

[Vratislav Podzimek]

When getting native name for some locale we want to get it for the right
language, right territory and also script, so that we get e.g. 'Srpski
(Latinica)' for 'sr_RS@latin' instead of the cyrilic (default) version.

Related: rhbz#1006458
(cherry-picked 10d29e1d11f9238e4772b03a67a787964d60a134 from master)

Signed-off-by: Vratislav Podzimek 
---
 pyanaconda/localization.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/pyanaconda/localization.py b/pyanaconda/localization.py
index fe6229f..f3ef182 100644
--- a/pyanaconda/localization.py
+++ b/pyanaconda/localization.py
@@ -205,7 +205,8 @@ def get_native_name(locale):
     name = langtable.language_name(languageId=parts["language"],
                                    territoryId=parts.get("territory", ""),
                                    scriptId=parts.get("script", ""),
-                                   languageIdQuery=parts["language"])
+                                   languageIdQuery=parts["language"],
+                                   scriptIdQuery=parts.get("script", ""))

return _upcase_first_letter(name)

[PATCH 05/14] Remove an unused argument of get_available_translations

[Vratislav Podzimek]

Related: rhbz#1006458
(cherry-picked ac04c8b0637635babfdc7877643183dde33fcdf6 from master)

Signed-off-by: Vratislav Podzimek 
---
 pyanaconda/localization.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/pyanaconda/localization.py b/pyanaconda/localization.py
index f3ef182..94da80c 100644
--- a/pyanaconda/localization.py
+++ b/pyanaconda/localization.py
@@ -210,19 +210,17 @@ def get_native_name(locale):

return _upcase_first_letter(name)

-def get_available_translations(domain=None, localedir=None):
+def get_available_translations(localedir=None):
     """
     Method that generates (i.e. returns a generator) available translations for
     the given domain and localedir.

-    :type domain: str
     :type localedir: str
     :return: generator yielding available translations
     :rtype: generator yielding strings

"""

-    domain = domain or gettext._current_domain
     localedir = localedir or gettext._default_localedir

messagefiles = sorted(glob.glob(localedir + "/*/LC_MESSAGES/anaconda.mo"))

[PATCH 06/14] Allow seting up locale without modifying ksdata

[Vratislav Podzimek]

This is useful for switching between locales on the welcome screen to make
retranslate methods work and not modify ksdata with temporary values.

Related: rhbz#1006458
(cherry-picked 367e9ec8cbd4a743fa9c690cd180d96911314063 from master)

Signed-off-by: Vratislav Podzimek 
---
 pyanaconda/localization.py | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/pyanaconda/localization.py b/pyanaconda/localization.py
index 94da80c..8218ea9 100644
--- a/pyanaconda/localization.py
+++ b/pyanaconda/localization.py
@@ -147,20 +147,23 @@ def is_supported_locale(locale):
     en_name = get_english_name(locale)
     return bool(en_name)

-def setup_locale(locale, lang):
+def setup_locale(locale, lang=None):
     """
     Procedure setting the system to use the given locale and store it in to the
-    ksdata.lang object. DOES NOT PERFORM ANY CHECKS OF THE GIVEN LOCALE.
+    ksdata.lang object (if given). DOES NOT PERFORM ANY CHECKS OF THE GIVEN
+    LOCALE.

:param locale: locale to setup
     :type locale: str
-    :param lang: ksdata.lang object
+    :param lang: ksdata.lang object or None
     :return: None
     :rtype: None

"""

-    lang.lang = locale
+    if lang:
+        lang.lang = locale
+
     os.environ["LANG"] = locale

def get_english_name(locale):

[PATCH 07/14] Rework the Welcome spoke to allow users choose from all locales

[Vratislav Podzimek]

What we actually need from users is a locale, value that can be set in the $LANG
environment variable. However, what we get from the files with translations are
usually languages (en, cs, es, de, ...). Instead of picking up one locale for
each such language, we may offer user all locales for all languages we have
translations for and let them choose the exact locale they want to use. By
setting it as the value of the $LANG variable, we then get the right
translations.

Resolves: rhbz#1006458
(cherry-picked 44f9c89ef467c3acd4d53dbb84a8c275cbcbe951 from master)

Signed-off-by: Vratislav Podzimek 
---
 pyanaconda/localization.py             |  16 +-
 pyanaconda/ui/gui/spokes/welcome.glade | 336 ++++++++++++++++-----------------
 pyanaconda/ui/gui/spokes/welcome.py    | 224 ++++++++++++----------
 pyanaconda/ui/gui/utils.py             |  40 ++++
 4 files changed, 337 insertions(+), 279 deletions(-)

diff --git a/pyanaconda/localization.py b/pyanaconda/localization.py
index 8218ea9..349b47d 100644
--- a/pyanaconda/localization.py
+++ b/pyanaconda/localization.py
@@ -216,34 +216,34 @@ def get_native_name(locale):
 def get_available_translations(localedir=None):
     """
     Method that generates (i.e. returns a generator) available translations for
-    the given domain and localedir.
+    the installer in the given localedir.

:type localedir: str
-    :return: generator yielding available translations
+    :return: generator yielding available translations (languages)
     :rtype: generator yielding strings

"""

localedir = localedir or gettext._default_localedir

-    messagefiles = sorted(glob.glob(localedir + "/*/LC_MESSAGES/anaconda.mo"))
+    # usually there are no message files for en
+    messagefiles = sorted(glob.glob(localedir + "/*/LC_MESSAGES/anaconda.mo") +
+                          ["blob/en/blob/blob"])
     trans_gen = (path.split(os.path.sep)[-3] for path in messagefiles)

-    # usually there are no message files for en
-    langs = {"en"}
-    yield "en_US.UTF-8"
+    langs = set()

for trans in trans_gen:
         parts = parse_langcode(trans)
         lang = parts.get("language", "")
         if lang and lang not in langs:
             langs.add(lang)
+            # check if there are any locales for the language
             locales = get_language_locales(lang)
             if not locales:
                 continue

-            # take the first locale (with highest rank) for the language
-            yield locales[0]
+            yield lang

def get_language_locales(lang):
     """
diff --git a/pyanaconda/ui/gui/spokes/welcome.glade b/pyanaconda/ui/gui/spokes/welcome.glade
index 1f44cab..915bf5f 100644
--- a/pyanaconda/ui/gui/spokes/welcome.glade
+++ b/pyanaconda/ui/gui/spokes/welcome.glade
@@ -126,6 +126,29 @@
       <action-widget response="1">uhdContinueButton</action-widget>
     </action-widgets>
   </object>
+  <object class="GtkListStore" id="languageStore">
+    <columns>
+      <!-- column-name nativeName -->
+      <column type="gchararray"/>
+      <!-- column-name englishName -->
+      <column type="gchararray"/>
+      <!-- column-name langSetting -->
+      <column type="gchararray"/>
+      <!-- column-name separator -->
+      <column type="gboolean"/>
+    </columns>
+  </object>
+  <object class="GtkListStore" id="localeStore">
+    <columns>
+      <!-- column-name nativeName -->
+      <column type="gchararray"/>
+      <!-- column-name locale -->
+      <column type="gchararray"/>
+    </columns>
+  </object>
+  <object class="GtkTreeModelFilter" id="languageStoreFilter">
+    <property name="child_model">languageStore</property>
+  </object>
   <object class="GtkDialog" id="betaWarnDialog">
     <property name="can_focus">False</property>
     <property name="border_width">5</property>
@@ -254,24 +277,29 @@ If you do not understand or accept the risks, then please exit this program by c
       <action-widget response="1">continueButton</action-widget>
     </action-widgets>
   </object>
-  <object class="AnacondaSpokeWindow" id="languageSpokeWindow">
+  <object class="AnacondaStandaloneWindow" id="welcomeWindow">
     <property name="startup_id">filler</property>
     <property name="can_focus">False</property>
     <property name="startup_id">filler</property>
-    <signal name="button-clicked" handler="on_back_clicked" swapped="no"/>
+    <property name="mnemonics_visible">False</property>
+    <property name="focus_visible">False</property>
+    <property name="window_name"></property>
     <child internal-child="main_box">
-      <object class="GtkBox" id="AnacondaSpokeWindow-main_box1">
+      <object class="GtkBox" id="AnacondaStandaloneWindow-main_box1">
         <property name="can_focus">False</property>
         <property name="orientation">vertical</property>
-        <property name="spacing">6</property>
         <child internal-child="nav_box">
-          <object class="GtkEventBox" id="AnacondaSpokeWindow-nav_box1">
+          <object class="GtkEventBox" id="AnacondaStandaloneWindow-nav_box1">
+            <property name="can_focus">False</property>
             <child internal-child="nav_area">
-              <object class="GtkGrid" id="AnacondaSpokeWindow-nav_area1">
+              <object class="GtkGrid" id="AnacondaStandaloneWindow-nav_area1">
                 <property name="can_focus">False</property>
-                <property name="margin_left">6</property>
-                <property name="margin_right">6</property>
-                <property name="margin_top">6</property>
+                <child>
+                  <placeholder/>
+                </child>
+                <child>
+                  <placeholder/>
+                </child>
               </object>
             </child>
           </object>
@@ -282,88 +310,11 @@ If you do not understand or accept the risks, then please exit this program by c
           </packing>
         </child>
         <child internal-child="alignment">
-          <object class="GtkAlignment" id="AnacondaSpokeWindow-alignment1">
-            <property name="can_focus">False</property>
-            <property name="yalign">0</property>
-            <property name="xscale">1</property>
-            <property name="yscale">1</property>
-            <property name="bottom_padding">48</property>
-            <property name="left_padding">24</property>
-            <property name="right_padding">24</property>
-            <child internal-child="action_area">
-              <object class="GtkBox" id="languageSpokeWindowContentBox">
-                <property name="can_focus">False</property>
-                <property name="orientation">vertical</property>
-                <property name="spacing">6</property>
-                <child>
-                  <placeholder/>
-                </child>
-              </object>
-            </child>
-          </object>
-          <packing>
-            <property name="expand">True</property>
-            <property name="fill">True</property>
-            <property name="position">1</property>
-          </packing>
-        </child>
-        <child>
-          <placeholder/>
-        </child>
-      </object>
-    </child>
-  </object>
-  <object class="GtkListStore" id="languageStore">
-    <columns>
-      <!-- column-name nativeName -->
-      <column type="gchararray"/>
-      <!-- column-name englishName -->
-      <column type="gchararray"/>
-      <!-- column-name langSetting -->
-      <column type="gchararray"/>
-      <!-- column-name separator -->
-      <column type="gboolean"/>
-    </columns>
-  </object>
-  <object class="GtkTreeModelFilter" id="languageStoreFilter">
-    <property name="child_model">languageStore</property>
-  </object>
-  <object class="AnacondaStandaloneWindow" id="welcomeWindow">
-    <property name="startup_id">filler</property>
-    <property name="can_focus">False</property>
-    <property name="startup_id">filler</property>
-    <property name="mnemonics_visible">False</property>
-    <property name="focus_visible">False</property>
-    <property name="window_name"/>
-    <child internal-child="main_box">
-      <object class="GtkBox" id="AnacondaStandaloneWindow-main_box1">
-        <property name="can_focus">False</property>
-        <property name="orientation">vertical</property>
-        <child internal-child="nav_box">
-           <object class="GtkEventBox" id="AnacondaStandaloneWindow-nav_box1">
-              <child internal-child="nav_area">
-                <object class="GtkGrid" id="AnacondaStandaloneWindow-nav_area1">
-                  <property name="can_focus">False</property>
-                  <child>
-                    <placeholder/>
-                  </child>
-                </object>
-              </child>
-           </object>
-           <packing>
-             <property name="expand">False</property>
-             <property name="fill">False</property>
-             <property name="position">0</property>
-           </packing>
-        </child>
-        <child internal-child="alignment">
           <object class="GtkAlignment" id="AnacondaStandaloneWindow-alignment1">
             <property name="can_focus">False</property>
             <property name="hexpand">True</property>
             <property name="vexpand">True</property>
             <property name="yalign">0</property>
-            <property name="xscale">1</property>
-            <property name="yscale">1</property>
             <property name="bottom_padding">48</property>
             <property name="left_padding">24</property>
             <property name="right_padding">24</property>
@@ -391,7 +342,7 @@ If you do not understand or accept the risks, then please exit this program by c
                 </child>
                 <child>
                   <object class="GtkLabel" id="pickLanguageLabel">
-                   <property name="height_request">45</property>
+                    <property name="height_request">45</property>
                     <property name="visible">True</property>
                     <property name="can_focus">False</property>
                     <property name="valign">start</property>
@@ -408,128 +359,167 @@ If you do not understand or accept the risks, then please exit this program by c
                   </packing>
                 </child>
                 <child>
-                  <object class="GtkAlignment" id="languageAlignment">
+                  <object class="GtkGrid" id="mainGrid">
                     <property name="visible">True</property>
                     <property name="can_focus">False</property>
+                    <property name="halign">center</property>
                     <property name="hexpand">True</property>
                     <property name="vexpand">True</property>
+                    <property name="row_spacing">6</property>
+                    <property name="column_spacing">6</property>
                     <child>
-                      <object class="GtkBox" id="box1">
+                      <object class="GtkScrolledWindow" id="languageWindow">
                         <property name="visible">True</property>
-                        <property name="can_focus">False</property>
+                        <property name="can_focus">True</property>
                         <property name="hexpand">True</property>
                         <property name="vexpand">True</property>
-                        <property name="orientation">vertical</property>
+                        <property name="hscrollbar_policy">never</property>
+                        <property name="shadow_type">in</property>
                         <child>
-                          <object class="GtkScrolledWindow" id="languageWindow">
+                          <object class="GtkTreeView" id="languageView">
                             <property name="visible">True</property>
                             <property name="can_focus">True</property>
-                            <property name="has_focus">True</property>
                             <property name="hexpand">True</property>
                             <property name="vexpand">True</property>
-                            <property name="hscrollbar_policy">never</property>
-                            <property name="shadow_type">in</property>
+                            <property name="model">languageStoreFilter</property>
+                            <property name="headers_visible">False</property>
+                            <property name="enable_search">False</property>
+                            <property name="search_column">0</property>
+                            <child internal-child="selection">
+                              <object class="GtkTreeSelection" id="languageViewSelection">
+                                <signal name="changed" handler="on_lang_selection_changed" swapped="no"/>
+                              </object>
+                            </child>
                             <child>
-                              <object class="GtkTreeView" id="languageView">
-                                <property name="visible">True</property>
-                                <property name="can_focus">True</property>
-                                <property name="hexpand">True</property>
-                                <property name="vexpand">True</property>
-                                <property name="model">languageStoreFilter</property>
-                                <property name="headers_visible">False</property>
-                                <property name="enable_search">False</property>
-                                <property name="search_column">0</property>
-                                <child internal-child="selection">
-                                  <object class="GtkTreeSelection" id="languageViewSelection">
-                                    <signal name="changed" handler="on_selection_changed" swapped="no"/>
+                              <object class="GtkTreeViewColumn" id="nativeName">
+                                <property name="title" translatable="yes">nativeName</property>
+                                <property name="expand">True</property>
+                                <property name="clickable">True</property>
+                                <property name="sort_column_id">0</property>
+                                <child>
+                                  <object class="GtkCellRendererText" id="nativeNameRenderer">
+                                    <property name="xalign">0.89999997615814209</property>
+                                    <property name="font">Cantarell 12</property>
                                   </object>
+                                  <attributes>
+                                    <attribute name="markup">0</attribute>
+                                  </attributes>
                                 </child>
+                              </object>
+                            </child>
+                            <child>
+                              <object class="GtkTreeViewColumn" id="englishName">
+                                <property name="title" translatable="yes">englishName</property>
+                                <property name="expand">True</property>
+                                <property name="clickable">True</property>
+                                <property name="sort_column_id">1</property>
                                 <child>
-                                  <object class="GtkTreeViewColumn" id="nativeName">
-                                    <property name="title" translatable="yes">nativeName</property>
-                                    <property name="expand">True</property>
-                                    <property name="clickable">True</property>
-                                    <property name="sort_column_id">0</property>
-                                    <child>
-                                      <object class="GtkCellRendererText" id="nativeNameRenderer">
-                                        <property name="font">Cantarell 12</property>
-                                      </object>
-                                      <attributes>
-                                        <attribute name="markup">0</attribute>
-                                      </attributes>
-                                    </child>
+                                  <object class="GtkCellRendererText" id="englishNameRenderer">
+                                    <property name="xalign">0.10000000149011612</property>
+                                    <property name="font">Cantarell Italic 14</property>
+                                    <property name="foreground">gray</property>
                                   </object>
+                                  <attributes>
+                                    <attribute name="text">1</attribute>
+                                  </attributes>
                                 </child>
+                              </object>
+                            </child>
+                            <child>
+                              <object class="GtkTreeViewColumn" id="langSelectedColumn">
+                                <property name="title" translatable="yes">selected</property>
                                 <child>
-                                  <object class="GtkTreeViewColumn" id="englishName">
-                                    <property name="title" translatable="yes">englishName</property>
-                                    <property name="expand">True</property>
-                                    <property name="clickable">True</property>
-                                    <property name="sort_column_id">1</property>
-                                    <child>
-                                      <object class="GtkCellRendererText" id="englishNameRenderer">
-                                        <property name="font">Cantarell Italic 14</property>
-                                        <property name="foreground">gray</property>
-                                      </object>
-                                      <attributes>
-                                        <attribute name="text">1</attribute>
-                                      </attributes>
-                                    </child>
-                                  </object>
+                                  <object class="GtkCellRendererPixbuf" id="langSelectedRenderer"/>
                                 </child>
                               </object>
                             </child>
                           </object>
-                          <packing>
-                            <property name="expand">True</property>
-                            <property name="fill">True</property>
-                            <property name="position">0</property>
-                          </packing>
                         </child>
+                      </object>
+                      <packing>
+                        <property name="left_attach">0</property>
+                        <property name="top_attach">0</property>
+                        <property name="width">1</property>
+                        <property name="height">1</property>
+                      </packing>
+                    </child>
+                    <child>
+                      <object class="GtkEntry" id="languageEntry">
+                        <property name="visible">True</property>
+                        <property name="can_focus">True</property>
+                        <property name="valign">start</property>
+                        <property name="invisible_char">●</property>
+                        <property name="invisible_char_set">True</property>
+                        <property name="secondary_icon_name">edit-clear-symbolic</property>
+                        <property name="placeholder_text">Type here to search.</property>
+                        <signal name="changed" handler="on_entry_changed" swapped="no"/>
+                        <signal name="icon-release" handler="on_clear_icon_clicked" swapped="no"/>
+                      </object>
+                      <packing>
+                        <property name="left_attach">0</property>
+                        <property name="top_attach">1</property>
+                        <property name="width">1</property>
+                        <property name="height">1</property>
+                      </packing>
+                    </child>
+                    <child>
+                      <object class="GtkScrolledWindow" id="localeWindow">
+                        <property name="height_request">150</property>
+                        <property name="visible">True</property>
+                        <property name="can_focus">True</property>
+                        <property name="hexpand">True</property>
+                        <property name="vexpand">True</property>
+                        <property name="hscrollbar_policy">never</property>
+                        <property name="shadow_type">in</property>
                         <child>
-                          <object class="GtkEntry" id="languageEntry">
+                          <object class="GtkTreeView" id="localeView">
                             <property name="visible">True</property>
                             <property name="can_focus">True</property>
-                            <property name="valign">start</property>
-                            <property name="margin_top">6</property>
-                            <property name="invisible_char">●</property>
-                            <property name="invisible_char_set">True</property>
-                            <property name="secondary_icon_name">edit-clear-symbolic</property>
-                            <property name="placeholder_text" translatable="yes">Type here to search.</property>
-                            <signal name="changed" handler="on_entry_changed" swapped="no"/>
-                            <signal name="icon-release" handler="on_clear_icon_clicked" swapped="no"/>
+                            <property name="hexpand">True</property>
+                            <property name="vexpand">True</property>
+                            <property name="model">localeStore</property>
+                            <property name="headers_visible">False</property>
+                            <property name="enable_search">False</property>
+                            <property name="search_column">0</property>
+                            <child internal-child="selection">
+                              <object class="GtkTreeSelection" id="localeViewSelection">
+                                <signal name="changed" handler="on_locale_selection_changed" swapped="no"/>
+                              </object>
+                            </child>
+                            <child>
+                              <object class="GtkTreeViewColumn" id="nativeName1">
+                                <property name="title" translatable="yes">nativeName</property>
+                                <property name="expand">True</property>
+                                <property name="clickable">True</property>
+                                <property name="sort_column_id">0</property>
+                                <child>
+                                  <object class="GtkCellRendererText" id="nativeNameRenderer1">
+                                    <property name="font">Cantarell 12</property>
+                                  </object>
+                                  <attributes>
+                                    <attribute name="markup">0</attribute>
+                                  </attributes>
+                                </child>
+                              </object>
+                            </child>
                           </object>
-
-                          <packing>
-                            <property name="expand">False</property>
-                            <property name="fill">True</property>
-                            <property name="position">1</property>
-                          </packing>
                         </child>
                       </object>
+                      <packing>
+                        <property name="left_attach">1</property>
+                        <property name="top_attach">0</property>
+                        <property name="width">1</property>
+                        <property name="height">1</property>
+                      </packing>
+                    </child>
+                    <child>
+                      <placeholder/>
                     </child>
-                  </object>
-                  <packing>
-                    <property name="expand">True</property>
-                    <property name="fill">True</property>
-                    <property name="position">2</property>
-                  </packing>
-                </child>
-                <child>
-                  <object class="GtkCheckButton" id="setKeyboardCheckButton">
-                    <property name="label" translatable="yes">Set _keyboard to default layout for selected language.</property>
-                    <property name="visible">True</property>
-                    <property name="can_focus">True</property>
-                    <property name="receives_default">False</property>
-                    <property name="use_underline">True</property>
-                    <property name="xalign">0</property>
-                    <property name="draw_indicator">True</property>
                   </object>
                   <packing>
                     <property name="expand">False</property>
                     <property name="fill">True</property>
-                    <property name="pack_type">end</property>
-                    <property name="position">3</property>
+                    <property name="position">2</property>
                   </packing>
                 </child>
               </object>
diff --git a/pyanaconda/ui/gui/spokes/welcome.py b/pyanaconda/ui/gui/spokes/welcome.py
index 5d982a4..f5b429e 100644
--- a/pyanaconda/ui/gui/spokes/welcome.py
+++ b/pyanaconda/ui/gui/spokes/welcome.py
@@ -17,25 +17,26 @@
 # Red Hat, Inc.
 #
 # Red Hat Author(s): Chris Lumens 
+#                    Vratislav Podzimek 
 #

import sys
 import re
+import langtable

-# pylint: disable-msg=E0611
-from gi.repository import AnacondaWidgets, Gtk, Pango
+from gi.repository import Gtk, Pango
 from pyanaconda.ui.gui.hubs.summary import SummaryHub
 from pyanaconda.ui.gui.spokes import StandaloneSpoke
-from pyanaconda.ui.gui.utils import enlightbox
+from pyanaconda.ui.gui.utils import enlightbox, set_treeview_selection

from pyanaconda import localization
 from pyanaconda.product import distributionText, isFinal, productName, productVersion
 from pyanaconda import keyboard
-from pyanaconda import timezone
 from pyanaconda import flags
 from pyanaconda import geoloc
-from pyanaconda.i18n import _, N_
+from pyanaconda.i18n import _
 from pyanaconda.iutil import is_unsupported_hw
+from pyanaconda.constants import DEFAULT_LANG

import logging
 log = logging.getLogger("anaconda")
@@ -45,7 +46,8 @@ __all__ = ["WelcomeLanguageSpoke"]
 class WelcomeLanguageSpoke(StandaloneSpoke):
     mainWidgetName = "welcomeWindow"
     uiFile = "spokes/welcome.glade"
-    builderObjects = ["languageStore", "languageStoreFilter", "welcomeWindow", "betaWarnDialog", "unsupportedHardwareDialog"]
+    builderObjects = ["languageStore", "languageStoreFilter", "localeStore",
+                      "welcomeWindow", "betaWarnDialog", "unsupportedHardwareDialog"]

preForHub = SummaryHub
     priority = 0
@@ -56,10 +58,9 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
         self._origStrings = {}

def apply(self):
-        selected = self.builder.get_object("languageViewSelection")
-        (store, itr) = selected.get_selected()
+        (store, itr) = self._localeSelection.get_selected()

-        locale = store[itr][2]
+        locale = store[itr][1]
         localization.setup_locale(locale, self.data.lang)

# Skip timezone and keyboard default setting for kickstart installs.
@@ -79,14 +80,18 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
             # current language
             self.data.timezone.timezone = loc_timezones[0]

-        lang_country = localization.get_locale_territory(self.data.lang.lang)
-        self._set_keyboard_defaults(store[itr][1], lang_country)
+        self._set_keyboard_defaults(self.data.lang.lang)

-    def _set_keyboard_defaults(self, lang_name, country):
+    def _set_keyboard_defaults(self, locale):
         """
         Set default keyboard settings (layouts, layout switching).

-        :param lang_name: name of the selected language (e.g. "Czech")
+        :param locale: locale string (see localization.LANGCODE_RE)
+        :type locale: str
+        :return: list of preferred keyboard layouts
+        :rtype: list of strings
+        :raise InvalidLocaleSpec: if an invalid locale is given (see
+                                  localization.LANGCODE_RE)

"""

@@ -102,23 +107,17 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
             #do not add layouts if there are any specified in the kickstart
             return

-        #get language name without any additional specifications
-        #e.g. 'English (United States)' -> 'English'
-        lang_name = lang_name.split()[0]
-
-        default_layout = self._xklwrapper.get_default_lang_country_layout(lang_name,
-                                                                          country)
-        if default_layout:
-            new_layouts = [default_layout]
+        layouts = localization.get_locale_keyboards(locale)
+        if layouts:
+            # take the first locale (with highest rank) from the list
+            new_layouts = [layouts[0]]
+            if not langtable.supports_ascii(layouts[0]):
+                # does not support typing ASCII chars, append the 'us' layout
+                new_layouts.append("us")
         else:
+            log.error("Failed to get layout for chosen locale '%s'" % locale)
             new_layouts = ["us"]

-        checkbutton = self.builder.get_object("setKeyboardCheckButton")
-        if not checkbutton.get_active() and "us" not in new_layouts:
-            #user doesn't want only the language-default layout, prepend
-            #'English (US)' layout
-            new_layouts.insert(0, "us")
-
         self.data.keyboard.x_layouts = new_layouts
         if flags.can_touch_runtime_system("replace runtime X layouts"):
             self._xklwrapper.replace_layouts(new_layouts)
@@ -129,6 +128,9 @@ class WelcomeLanguageSpoke(StandaloneSpoke):

if flags.can_touch_runtime_system("init layout switching"):
                 self._xklwrapper.set_switching_options(["grp:alt_shift_toggle"])
+                # activate the first (language-default) layout instead of the
+                # 'us' one
+                self._xklwrapper.activate_default_layout()

@property
     def completed(self):
@@ -140,15 +142,33 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
     def _row_is_separator(self, model, itr, *args):
         return model[itr][3]

+    def _render_lang_selected(self, column, renderer, model, itr, user_data=None):
+        (lang_store, sel_itr) = self._langSelection.get_selected()
+
+        if sel_itr and lang_store[sel_itr][2] == model[itr][2]:
+            renderer.set_property("pixbuf", self._right_arrow.get_pixbuf())
+        else:
+            renderer.set_property("pixbuf", None)
+
     def initialize(self):
-        store = self.builder.get_object("languageStore")
+        self._languageStore = self.builder.get_object("languageStore")
         self._languageStoreFilter = self.builder.get_object("languageStoreFilter")
         self._languageEntry = self.builder.get_object("languageEntry")
-        self._selection = self.builder.get_object("languageViewSelection")
-        self._view = self.builder.get_object("languageView")
+        self._langSelection = self.builder.get_object("languageViewSelection")
+        self._langSelectedRenderer = self.builder.get_object("langSelectedRenderer")
+        self._langSelectedColumn = self.builder.get_object("langSelectedColumn");
+        self._langView = self.builder.get_object("languageView")
+        self._localeView = self.builder.get_object("localeView")
+        self._localeStore = self.builder.get_object("localeStore")
+        self._localeSelection = self.builder.get_object("localeViewSelection")

# We need to tell the view whether something is a separator or not.
-        self._view.set_row_separator_func(self._row_is_separator, None)
+        self._langView.set_row_separator_func(self._row_is_separator, None)
+
+        # Render a right arrow for the chosen language
+        self._right_arrow = Gtk.Image.new_from_file("/usr/share/anaconda/pixmaps/right-arrow-icon.png")
+        self._langSelectedColumn.set_cell_data_func(self._langSelectedRenderer,
+                                                    self._render_lang_selected)

# We can use the territory from geolocation here
         # to preselect the translation, when it's available.
@@ -161,22 +181,28 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
             localization.setup_locale(locales[0], self.data.lang)

# fill the list with available translations
-        for locale in localization.get_available_translations():
-            self._addLanguage(store, localization.get_native_name(locale),
-                              localization.get_english_name(locale), locale)
+        for lang in localization.get_available_translations():
+            self._addLanguage(self._languageStore,
+                              localization.get_native_name(lang),
+                              localization.get_english_name(lang), lang)

# Move the default language (whatever was provided on the command line,
         # or by kickstart, or by geoip, or English if nothing else) to the top
         # of the list and select it by default.  People find it confusing to be
         # dropped into the middle of a scrollable list.
-        (store, itr) = self._selection.get_selected()
-        if not itr:
-            itr = self._selectLanguage(self.data.lang.lang)
+        lang_itr, locale_itr = self._select_locale(self.data.lang.lang)
+
+        if not lang_itr or not locale_itr:
+            log.error("Failed to select language %s, using the default %s",
+                      self.data.lang.lang, DEFAULT_LANG)
+            lang_itr, locale_itr = self._select_locale(DEFAULT_LANG)
+            self.data.lang.lang = DEFAULT_LANG

-        # store is the filtered store, and itr is an iter on it.  We need to
+        filter_store = self._languageStoreFilter
+        # filtered store and lang_itr is an iter on it.  We need to
         # convert to an iter on the underlying store.
-        itr = store.convert_iter_to_child_iter(itr)
-        store = store.get_model()
+        itr = filter_store.convert_iter_to_child_iter(lang_itr)
+        store = filter_store.get_model()
         store.move_after(itr, None)

# And then we add a separator after the default chosen language.
@@ -200,7 +226,7 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
         # Change the translations on labels and buttons that do not have
         # substitution text.
         for name in ["pickLanguageLabel", "betaWarnTitle", "betaWarnDesc",
-                     "quitButton", "continueButton", "setKeyboardCheckButton"]:
+                     "quitButton", "continueButton"]:
             self._retranslate_one(name)

# The welcome label is special - it has text that needs to be
@@ -211,7 +237,7 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
             self._origStrings[welcomeLabel] = welcomeLabel.get_label()

before = self._origStrings[welcomeLabel]
-        xlated = _(before) % (productName.upper(), productVersion)
+        xlated = _(before) % {"name" : productName.upper(), "version" : productVersion}
         welcomeLabel.set_label(xlated)

# And of course, don't forget the underlying window.
@@ -219,28 +245,17 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
         self.window.retranslate(lang)

def refresh(self):
-        self._selectLanguage(self.data.lang.lang)
-
-        # Rip the label and language selection window
-        # from where it is right now and add it to this
-        # spoke.
-        # This way we can share the dialog and code
-        # between Welcome and Language spokes
-        langLabel = self.builder.get_object("pickLanguageLabel")
-        langLabel.get_parent().remove(langLabel)
-        langAlign = self.builder.get_object("languageAlignment")
-        langAlign.get_parent().remove(langAlign)
-
-        content = self.builder.get_object("welcomeWindowContentBox")
-        content.pack_start(child = langLabel, fill = True, expand = False, padding = 0)
-        content.pack_start(child = langAlign, fill = True, expand = True, padding = 0)
-
+        self._select_locale(self.data.lang.lang)
         self._languageEntry.set_text("")
         self._languageStoreFilter.refilter()

-    def _addLanguage(self, store, native, english, setting):
-        lang = '<span lang="%s">%s</span>' % (re.sub(r'..*', '', setting), native)
-        store.append([lang, english, setting, False])
+    def _addLanguage(self, store, native, english, lang):
+        native_span = '<span lang="%s">%s</span>' % (lang, native)
+        store.append([native_span, english, lang, False])
+
+    def _addLocale(self, store, native, locale):
+        native_span = '<span lang="%s">%s</span>' % (re.sub(r'..*', '', locale), native)
+        store.append([native_span, locale])

def _matchesEntry(self, model, itr, *args):
         # Need to strip out the pango markup before attempting to match.
@@ -262,51 +277,64 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
         else:
             return False

-    def _selectLanguage(self, language):
-        treeview = self.builder.get_object("languageView")
-        selection = treeview.get_selection()
-        store = treeview.get_model()
-        itr = store.get_iter_first()
-        # store[itr][3] is True if this row is a separator in the view, so we
-        # want to skip those.
-        while itr and not store[itr][3] \
-                and language not in localization.expand_langs(store[itr][2]):
-            itr = store.iter_next(itr)
-
-        # If we were provided with an unsupported language, just use the default.
-        if not itr:
-            return
+    def _select_locale(self, locale):
+        """
+        Try to select the given locale in the language and locale
+        treeviews. This method tries to find the best match for the given
+        locale.
+
+        :return: a pair of selected iterators (language and locale)
+        :rtype: a pair of GtkTreeIter or None objects
+
+        """

-        selection.select_iter(itr)
-        path = store.get_path(itr)
-        # row_align=0.5 tells GTK to move the cell to the middle of the
-        # treeview viewport (0.0 should align it with the top, 1.0 with bottom)
-        # If the cell is the uppermost one, it should align it with the top
-        # of the viewport.
-        #
-        # Unfortunately, this does not work as expected due to a bug in GTK.
-        # So currently if the cell is the upper most one, it will sort of
-        # align it with the top of the viewport, leaving about 30% of
-        # it hidden. If the target cell is not the upper most one,
-        # it seems to just scroll the treeview a bit, leaving the
-        # target cell well out of the viewport.
-        #
-        # In short, once that GTK bug is fixed, row_align=0.5 should work
-        # correctly for showing both upper most cells and all other cells
-        # in the tree view.
-        treeview.scroll_to_cell(path, use_align=True, row_align=0.5)
-
-        return itr
+        # get lang and select it
+        parts = localization.parse_langcode(locale)
+        if "language" not in parts:
+            # invalid locale, cannot select
+            return (None, None)
+
+        lang_itr = set_treeview_selection(self._langView, parts["language"], col=2)
+
+        # find matches and use the one with the highest rank
+        locales = localization.get_language_locales(locale)
+        locale_itr = set_treeview_selection(self._localeView, locales[0], col=1)
+
+        return (lang_itr, locale_itr)
+
+    def _refresh_locale_store(self, lang):
+        """Refresh the localeStore with locales for the given language."""
+
+        self._localeStore.clear()
+        locales = localization.get_language_locales(lang)
+        for locale in locales:
+            self._addLocale(self._localeStore,
+                            localization.get_native_name(locale),
+                            locale)
+
+        # select the first locale (with the highest rank)
+        set_treeview_selection(self._localeView, locales[0], col=1)

# Signal handlers.
-    def on_selection_changed(self, selection):
+    def on_lang_selection_changed(self, selection):
+        (store, selected) = selection.get_selected_rows()
+
+        if selected:
+            lang = store[selected[0]][2]
+            self._refresh_locale_store(lang)
+        else:
+            if hasattr(self.window, "set_may_continue"):
+                self.window.set_may_continue(False)
+            self._localeStore.clear()
+
+    def on_locale_selection_changed(self, selection):
         (store, selected) = selection.get_selected_rows()
         if hasattr(self.window, "set_may_continue"):
             self.window.set_may_continue(len(selected) > 0)

if selected:
-            lang = store[selected[0]][2]
-            localization.setup_locale(lang, self.data.lang)
+            lang = store[selected[0]][1]
+            localization.setup_locale(lang)
             self.retranslate(lang)

def on_clear_icon_clicked(self, entry, icon_pos, event):
diff --git a/pyanaconda/ui/gui/utils.py b/pyanaconda/ui/gui/utils.py
index 4ea71be..4456ab1 100644
--- a/pyanaconda/ui/gui/utils.py
+++ b/pyanaconda/ui/gui/utils.py
@@ -153,3 +153,43 @@ def really_show(widget):
     """
     widget.set_no_show_all(False)
     widget.show()
+
+def set_treeview_selection(treeview, item, col=0):
+    """
+    Select the given item in the given treeview and scroll to it.
+
+    :param treeview: treeview to select and item in
+    :type treeview: GtkTreeView
+    :param item: item to be selected
+    :type item: str
+    :param col: column to search for the item in
+    :type col: int
+    :return: selected iterator or None if item was not found
+    :rtype: GtkTreeIter or None
+
+    """
+
+    model = treeview.get_model()
+    itr = model.get_iter_first()
+    while itr and not model[itr][col] == item:
+        itr = model.iter_next(itr)
+
+    if not itr:
+        # item not found, cannot be selected
+        return None
+
+    # otherwise select the item and scroll to it
+    selection = treeview.get_selection()
+    selection.select_iter(itr)
+    path = model.get_path(itr)
+
+    # row_align=0.5 tells GTK to move the cell to the middle of the
+    # treeview viewport (0.0 should align it with the top, 1.0 with bottom)
+    # If the cell is the uppermost one, it should align it with the top
+    # of the viewport.
+    #
+    # Unfortunately, this does not work as expected due to a bug in GTK.
+    # (see rhbz#970048)
+    treeview.scroll_to_cell(path, use_align=True, row_align=0.5)
+
+    return itr

[PATCH 08/14] Get rid of the non-deterministic expand_langs and its usage

[Vratislav Podzimek]

Using expand_langs was non-deterministic in a way that if e.g. "pt" came before
"pt_BR" in searching for a match for something from expand_langs("pt_BR"), it
was picked up. However, in such a case we want to pick up "pt_BR" as it matches
better.

This patch adds a function for finding matches (now the best one)
deterministicly and a function to determine if given langcode (e.g. "en")
matches some locale (e.g. "en_US"). The former one is useful e.g. for getting
the best rnotes to show based on the installation language, the latter one is
useful for adding langsupport groups to the installation transaction.

Related: rhbz#1006458
(cherry-picked c440106ab78ea3adf774dd917dd6a21a0fb0558d from master)

Signed-off-by: Vratislav Podzimek 
---
 pyanaconda/localization.py         | 125 +++++++++++++++++++++++++------------
 pyanaconda/packaging/yumpayload.py |   9 ++-
 pyanaconda/ui/gui/hubs/progress.py |  53 +++++++++-------
 3 files changed, 122 insertions(+), 65 deletions(-)

diff --git a/pyanaconda/localization.py b/pyanaconda/localization.py
index 349b47d..2b4d97c 100644
--- a/pyanaconda/localization.py
+++ b/pyanaconda/localization.py
@@ -88,64 +88,111 @@ def parse_langcode(langcode):
     else:
         return None

-def expand_langs(astring):
+def is_supported_locale(locale):
     """
-    Converts a single language into a "language search path". For example,
-    for "fr_FR.UTF-8@euro" would return set containing:
-    "fr", "fr_FR", "fr_FR.UTF-8@euro", "fr.UTF-8@euro", "fr_FR@euro",
-    "fr_FR.UTF-8", "fr@euro", "fr.UTF-8"
+    Function that tells if the given locale is supported by the Anaconda or
+    not. We consider locales supported by the langtable as supported by the
+    Anaconda.

-    :rtype: list of strings
+    :param locale: locale to test
+    :type locale: str
+    :return: whether the given locale is supported or not
+    :rtype: bool
+    :raise InvalidLocaleSpec: if an invalid locale is given (see LANGCODE_RE)

"""

-    langs = set([astring])
-
-    lang_dict = parse_langcode(astring)
-
-    if not lang_dict:
-        return list(langs)
+    en_name = get_english_name(locale)
+    return bool(en_name)

-    base, loc, enc, script = [lang_dict[key] for key in ("language",
-                                      "territory", "encoding", "script")]
+def langcode_matches_locale(langcode, locale):
+    """
+    Function that tells if the given langcode matches the given locale. I.e. if
+    all parts of appearing in the langcode (language, territory, script and
+    encoding) are the same as the matching parts of the locale.

-    if not base:
-        return list(langs)
+    :param langcode: a langcode (e.g. en, en_US, en_US@latin, etc.)
+    :type langcode: str
+    :param locale: a valid locale (e.g. en_US.UTF-8 or sr_RS.UTF-8@latin, etc.)
+    :type locale: str
+    :return: whether the given langcode matches the given locale or not
+    :rtype: bool

-    if not enc:
-        enc = "UTF-8"
+    """

-    langs.add(base)
-    langs.add("%s.%s" % (base, enc))
+    langcode_parts = parse_langcode(langcode)
+    locale_parts = parse_langcode(locale)

-    if loc:
-        langs.add("%s_%s" % (base, loc))
-        langs.add("%s_%s.%s" %(base, loc, enc))
-    if script:
-        langs.add("%s@%s" % (base, script))
-        langs.add("%s.%s@%s" % (base, enc, script))
+    if not langcode_parts or not locale_parts:
+        # to match, both need to be valid langcodes (need to have at least
+        # language specified)
+        return False

-    if loc and script:
-        langs.add("%s_%s@%s" % (base, loc, script))
+    # Check parts one after another. If some part appears in the langcode and
+    # doesn't match the one from the locale (or is missing in the locale),
+    # return False, otherwise they match
+    for part in ("language", "territory", "script", "encoding"):
+        if langcode_parts[part] and langcode_parts[part] != locale_parts.get(part):
+            return False

-    return list(langs)
+    return True

-def is_supported_locale(locale):
+def find_best_locale_match(locale, langcodes):
     """
-    Function that tells if the given locale is supported by the Anaconda or
-    not. We consider locales supported by the langtable as supported by the
-    Anaconda.
+    Find the best match for the locale in a list of langcodes. This is useful
+    when e.g. pt_BR is a locale and there are possibilities to choose an item
+    (e.g. rnote) for a list containing both pt and pt_BR or even also pt_PT.

-    :param locale: locale to test
+    :param locale: a valid locale (e.g. en_US.UTF-8 or sr_RS.UTF-8@latin, etc.)
     :type locale: str
-    :return: whether the given locale is supported or not
-    :rtype: bool
-    :raise InvalidLocaleSpec: if an invalid locale is given (see LANGCODE_RE)
+    :param langcodes: a list or generator of langcodes (e.g. en, en_US, en_US@latin, etc.)
+    :type langcodes: list(str) or generator(str)
+    :return: the best matching langcode from the list of None if none matches
+    :rtype: str or None

"""

-    en_name = get_english_name(locale)
-    return bool(en_name)
+    score_map = { "language" : 1000,
+                  "territory":  100,
+                  "script"   :   10,
+                  "encoding" :    1 }
+
+    def get_match_score(locale, langcode):
+        score = 0
+
+        locale_parts = parse_langcode(locale)
+        langcode_parts = parse_langcode(langcode)
+        if not locale_parts or not langcode_parts:
+            return score
+
+        for part, part_score in score_map.iteritems():
+            if locale_parts[part] and langcode_parts[part]:
+                if locale_parts[part] == langcode_parts[part]:
+                    # match
+                    score += part_score
+                else:
+                    # not match
+                    score -= part_score
+            elif langcode_parts[part] and not locale_parts[part]:
+                # langcode has something the locale doesn't have
+                score -= part_score
+
+        return score
+
+    scores = []
+
+    # get score for each langcode
+    for langcode in langcodes:
+        scores.append((langcode, get_match_score(locale, langcode)))
+
+    # find the best one
+    sorted_langcodes = sorted(scores, key=lambda item_score: item_score[1], reverse=True)
+
+    # matches matching only script or encoding or both are not useful
+    if sorted_langcodes and sorted_langcodes[0][1] > score_map["territory"]:
+        return sorted_langcodes[0][0]
+    else:
+        return None

def setup_locale(locale, lang=None):
     """
diff --git a/pyanaconda/packaging/yumpayload.py b/pyanaconda/packaging/yumpayload.py
index 5c9bbc1..fe12310 100644
--- a/pyanaconda/packaging/yumpayload.py
+++ b/pyanaconda/packaging/yumpayload.py
@@ -85,7 +85,7 @@ from pyanaconda.errors import *
 from pyanaconda.packaging import NoSuchGroup, NoSuchPackage
 from pyanaconda.progress import progressQ

-from pyanaconda.localization import expand_langs
+from pyanaconda.localization import langcode_matches_locale
 import itertools

from pykickstart.constants import KS_MISSING_IGNORE
@@ -1193,10 +1193,9 @@ reposdir=%s
         with _yum_lock:
             groups = yum_groups.get_groups()
             for lang_code in lang_codes:
-                for lang_code_guess in expand_langs(lang_code):
-                    for group in groups:
-                        if group.langonly == lang_code_guess:
-                            lang_groups.add(group.groupid)
+                for group in groups:
+                    if langcode_matches_locale(group.langonly, lang_code):
+                        lang_groups.add(group.groupid)

return list(lang_groups)

diff --git a/pyanaconda/ui/gui/hubs/progress.py b/pyanaconda/ui/gui/hubs/progress.py
index 36e2be1..b22d91a 100644
--- a/pyanaconda/ui/gui/hubs/progress.py
+++ b/pyanaconda/ui/gui/hubs/progress.py
@@ -26,9 +26,10 @@ from gi.repository import GLib, Gtk
 import itertools
 import os
 import sys
+import glob

from pyanaconda.i18n import _
-from pyanaconda.localization import expand_langs
+from pyanaconda.localization import langcode_matches_locale, find_best_locale_match
 from pyanaconda.product import productName
 from pyanaconda.flags import flags
 from pyanaconda.constants import THREAD_INSTALL, THREAD_CONFIGURATION, DEFAULT_LANG
@@ -140,28 +141,38 @@ class ProgressHub(Hub):
             self._progressNotebook.set_current_page(0)

def _get_rnotes(self):
-        import glob
-
         # We first look for rnotes in paths containing the language, then in
         # directories without the language component.  You know, just in case.
-        try:
-            langs = expand_langs(os.environ["LANG"]) + [""]
-        except KeyError:
-            # if for some reason LANG isn't set to anything, set it to
-            # the default value
-            os.environ["LANG"] = DEFAULT_LANG
-            langs = expand_langs(os.environ["LANG"]) + [""]
-
-        paths = ["/tmp/updates/pixmaps/rnotes/%s/",
-                 "/tmp/product/pixmaps/rnotes/%s/",
-                 "/usr/share/anaconda/pixmaps/rnotes/%s/"]
-
-        for (l, d) in itertools.product(langs, paths):
-            pixmaps = glob.glob((d % l) + "*.png") + glob.glob((d % l) + "*.jpg")
-            if len(pixmaps) > 0:
-                return pixmaps
-
-        return []
+
+        paths = ["/tmp/updates/pixmaps/rnotes/",
+                 "/tmp/product/pixmaps/rnotes/",
+                 "/usr/share/anaconda/pixmaps/rnotes/"]
+
+        all_lang_pixmaps = []
+        for path in paths:
+            all_lang_pixmaps += glob.glob(path + "*/*.png") + glob.glob(path + "*/*.jpg")
+
+        pixmap_langs = [pixmap.split(os.path.sep)[-2] for pixmap in all_lang_pixmaps]
+        best_lang = find_best_locale_match(os.environ["LANG"], pixmap_langs)
+
+        if not best_lang:
+            # nothing found, try the default language
+            best_lang = find_best_locale_match(DEFAULT_LANG, pixmap_langs)
+
+        if not best_lang:
+            # nothing found even for the default language, try non-localized rnotes
+            non_localized = []
+            for path in paths:
+                non_localized += glob.glob(path + "*.png") + glob.glob(path + "*.jpg")
+
+            return non_localized
+
+        best_lang_pixmaps = []
+        for path in paths:
+            best_lang_pixmaps += (glob.glob(path + best_lang + "/*.png") +
+                                  glob.glob(path + best_lang + "/*.jpg"))
+
+        return best_lang_pixmaps

def _cycle_rnotes(self):
         # Change the ransom notes image every minute by grabbing the next

[PATCH 09/14] Match langs with stripped accents when filtering languages

[Vratislav Podzimek]

This e.g. allows to search for "Čeština" by typing "cest" in the filtering
entry which was a common user behaviour appearing in the usability testing.

Related: rhbz#1006458
(cherry-picked 55096705e52348c60ada4a3e0336f122cffb9170 from master)

Signed-off-by: Vratislav Podzimek 
---
 pyanaconda/ui/gui/spokes/langsupport.py | 4 +++-
 pyanaconda/ui/gui/spokes/welcome.py     | 5 +++--
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/pyanaconda/ui/gui/spokes/langsupport.py b/pyanaconda/ui/gui/spokes/langsupport.py
index 66dbad2..1019457 100644
--- a/pyanaconda/ui/gui/spokes/langsupport.py
+++ b/pyanaconda/ui/gui/spokes/langsupport.py
@@ -23,6 +23,7 @@
 from gi.repository import Gtk, Pango
 from pyanaconda.flags import flags
 from pyanaconda.i18n import _, N_
+from pyanaconda.iutil import strip_accents
 from pyanaconda.ui.gui.spokes import NormalSpoke
 from pyanaconda.ui.gui.categories.localization import LocalizationCategory
 from pyanaconda import localization
@@ -172,7 +173,8 @@ class LangsupportSpoke(NormalSpoke):
         # Otherwise, filter the list showing only what is matched by the
         # text entry.  Either the English or native names can match.
         lowered = entry.lower()
-        if lowered in native.lower() or lowered in english.lower():
+        translit = strip_accents(unicode(native, "utf-8")).lower()
+        if lowered in native.lower() or lowered in english.lower() or lowered in translit:
             return True
         else:
             return False
diff --git a/pyanaconda/ui/gui/spokes/welcome.py b/pyanaconda/ui/gui/spokes/welcome.py
index f5b429e..14427df 100644
--- a/pyanaconda/ui/gui/spokes/welcome.py
+++ b/pyanaconda/ui/gui/spokes/welcome.py
@@ -35,7 +35,7 @@ from pyanaconda import keyboard
 from pyanaconda import flags
 from pyanaconda import geoloc
 from pyanaconda.i18n import _
-from pyanaconda.iutil import is_unsupported_hw
+from pyanaconda.iutil import is_unsupported_hw, strip_accents
 from pyanaconda.constants import DEFAULT_LANG

import logging
@@ -272,7 +272,8 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
         # Otherwise, filter the list showing only what is matched by the
         # text entry.  Either the English or native names can match.
         lowered = entry.lower()
-        if lowered in native.lower() or lowered in english.lower():
+        translit = strip_accents(unicode(native, "utf-8")).lower()
+        if lowered in native.lower() or lowered in english.lower() or lowered in translit:
             return True
         else:
             return False

[PATCH 10/14] Rework the Langsupport spoke to work with all locales

[Vratislav Podzimek]

This is a follow up for the Welcome spoke reworking.

Related: rhbz#1006458
(cherry-picked f41e79245666755a45b575900460f730aee9a52b from master)

Signed-off-by: Vratislav Podzimek 
---
 pyanaconda/ui/gui/spokes/langsupport.glade | 166 ++++++++++++++------
 pyanaconda/ui/gui/spokes/langsupport.py    | 241 +++++++++++++++++------------
 2 files changed, 263 insertions(+), 144 deletions(-)

diff --git a/pyanaconda/ui/gui/spokes/langsupport.glade b/pyanaconda/ui/gui/spokes/langsupport.glade
index afb5995..6727c9b 100644
--- a/pyanaconda/ui/gui/spokes/langsupport.glade
+++ b/pyanaconda/ui/gui/spokes/langsupport.glade
@@ -2,7 +2,7 @@
 <interface>
   <!-- interface-requires gtk+ 3.0 -->
   <!-- interface-requires AnacondaWidgets 1.0 -->
-  <object class="GtkListStore" id="langsupportStore">
+  <object class="GtkListStore" id="languageStore">
     <columns>
       <!-- column-name nativeName -->
       <column type="gchararray"/>
@@ -10,14 +10,10 @@
       <column type="gchararray"/>
       <!-- column-name langSetting -->
       <column type="gchararray"/>
-      <!-- column-name langSupportSelected -->
-      <column type="gboolean"/>
-      <!-- column-name isAdditional -->
-      <column type="gboolean"/>
     </columns>
   </object>
-  <object class="GtkTreeModelFilter" id="langsupportStoreFilter">
-    <property name="child_model">langsupportStore</property>
+  <object class="GtkTreeModelFilter" id="languageStoreFilter">
+    <property name="child_model">languageStore</property>
   </object>
   <object class="AnacondaSpokeWindow" id="langsupportWindow">
     <property name="startup_id">filler</property>
@@ -82,54 +78,47 @@
                   </packing>
                 </child>
                 <child>
-                  <object class="GtkBox" id="box1">
+                  <object class="GtkGrid" id="mainGrid">
                     <property name="visible">True</property>
                     <property name="can_focus">False</property>
+                    <property name="halign">center</property>
                     <property name="hexpand">True</property>
                     <property name="vexpand">True</property>
-                    <property name="orientation">vertical</property>
+                    <property name="row_spacing">6</property>
+                    <property name="column_spacing">6</property>
                     <child>
-                      <object class="GtkScrolledWindow" id="langsupportScrolledwindow1">
+                      <object class="GtkScrolledWindow" id="languageWindow">
                         <property name="visible">True</property>
                         <property name="can_focus">True</property>
                         <property name="hexpand">True</property>
                         <property name="vexpand">True</property>
+                        <property name="hscrollbar_policy">never</property>
                         <property name="shadow_type">in</property>
                         <child>
-                          <object class="GtkTreeView" id="langsupportView">
+                          <object class="GtkTreeView" id="languageView">
                             <property name="visible">True</property>
                             <property name="can_focus">True</property>
                             <property name="hexpand">True</property>
                             <property name="vexpand">True</property>
-                            <property name="model">langsupportStoreFilter</property>
+                            <property name="model">languageStoreFilter</property>
                             <property name="headers_visible">False</property>
                             <property name="headers_clickable">False</property>
+                            <property name="enable_search">False</property>
                             <property name="search_column">0</property>
                             <child internal-child="selection">
-                              <object class="GtkTreeSelection" id="langsupportViewSelection"/>
-                            </child>
-                            <child>
-                              <object class="GtkTreeViewColumn" id="langsupportSelectedCol">
-                                <property name="title" translatable="yes">column</property>
-                                <signal name="clicked" handler="on_langsupport_toggled" swapped="no"/>
-                                <child>
-                                  <object class="GtkCellRendererToggle" id="langsupportSelectedRenderer">
-                                    <signal name="toggled" handler="on_langsupport_toggled" swapped="no"/>
-                                  </object>
-                                  <attributes>
-                                    <attribute name="sensitive">4</attribute>
-                                    <attribute name="activatable">4</attribute>
-                                    <attribute name="active">3</attribute>
-                                  </attributes>
-                                </child>
+                              <object class="GtkTreeSelection" id="languageViewSelection">
+                                <signal name="changed" handler="on_lang_selection_changed" swapped="no"/>
                               </object>
                             </child>
                             <child>
-                              <object class="GtkTreeViewColumn" id="nativeNameCol">
-                                <property name="title" translatable="yes">column</property>
+                              <object class="GtkTreeViewColumn" id="nativeName">
+                                <property name="title" translatable="yes">nativeName</property>
                                 <property name="expand">True</property>
+                                <property name="clickable">True</property>
+                                <property name="sort_column_id">0</property>
                                 <child>
                                   <object class="GtkCellRendererText" id="nativeNameRenderer">
+                                    <property name="xalign">0.89999997615814209</property>
                                     <property name="font">Cantarell 12</property>
                                   </object>
                                   <attributes>
@@ -139,52 +128,127 @@
                               </object>
                             </child>
                             <child>
-                              <object class="GtkTreeViewColumn" id="englishNameCol">
-                                <property name="title" translatable="yes">column</property>
+                              <object class="GtkTreeViewColumn" id="englishName">
+                                <property name="title" translatable="yes">englishName</property>
                                 <property name="expand">True</property>
+                                <property name="clickable">True</property>
+                                <property name="sort_column_id">1</property>
                                 <child>
                                   <object class="GtkCellRendererText" id="englishNameRenderer">
+                                    <property name="xalign">0.10000000149011612</property>
                                     <property name="font">Cantarell Italic 14</property>
                                     <property name="foreground">gray</property>
                                   </object>
                                   <attributes>
-                                    <attribute name="markup">1</attribute>
+                                    <attribute name="text">1</attribute>
                                   </attributes>
                                 </child>
                               </object>
                             </child>
+                            <child>
+                              <object class="GtkTreeViewColumn" id="langSelectedColumn">
+                                <property name="title" translatable="yes">selected</property>
+                                <child>
+                                  <object class="GtkCellRendererPixbuf" id="langSelectedRenderer"/>
+                                </child>
+                              </object>
+                            </child>
                           </object>
                         </child>
                       </object>
                       <packing>
-                        <property name="expand">True</property>
-                        <property name="fill">True</property>
-                        <property name="position">1</property>
+                        <property name="left_attach">0</property>
+                        <property name="top_attach">0</property>
+                        <property name="width">1</property>
+                        <property name="height">1</property>
                       </packing>
                     </child>
                     <child>
-                      <object class="GtkEntry" id="langsupportEntry">
+                      <object class="GtkEntry" id="languageEntry">
                         <property name="visible">True</property>
                         <property name="can_focus">True</property>
                         <property name="valign">start</property>
-                        <property name="margin_top">6</property>
-                        <property name="hexpand">True</property>
                         <property name="invisible_char">●</property>
                         <property name="invisible_char_set">True</property>
                         <property name="secondary_icon_name">edit-clear-symbolic</property>
-                        <property name="placeholder_text" translatable="yes">Type here to search.</property>
+                        <property name="placeholder_text">Type here to search.</property>
                         <signal name="changed" handler="on_entry_changed" swapped="no"/>
-                        <signal name="icon-release" handler="on_clear_icon_clicked" swapped="no"/>
                       </object>
                       <packing>
-                        <property name="expand">False</property>
-                        <property name="fill">True</property>
-                        <property name="position">2</property>
+                        <property name="left_attach">0</property>
+                        <property name="top_attach">1</property>
+                        <property name="width">1</property>
+                        <property name="height">1</property>
+                      </packing>
+                    </child>
+                    <child>
+                      <object class="GtkScrolledWindow" id="localeWindow">
+                        <property name="height_request">150</property>
+                        <property name="visible">True</property>
+                        <property name="can_focus">True</property>
+                        <property name="hexpand">True</property>
+                        <property name="vexpand">True</property>
+                        <property name="hscrollbar_policy">never</property>
+                        <property name="shadow_type">in</property>
+                        <child>
+                          <object class="GtkTreeView" id="localeView">
+                            <property name="visible">True</property>
+                            <property name="can_focus">True</property>
+                            <property name="hexpand">True</property>
+                            <property name="vexpand">True</property>
+                            <property name="model">localeStore</property>
+                            <property name="headers_visible">False</property>
+                            <property name="enable_search">False</property>
+                            <property name="search_column">0</property>
+                            <child internal-child="selection">
+                              <object class="GtkTreeSelection" id="localeViewSelection"/>
+                            </child>
+                            <child>
+                              <object class="GtkTreeViewColumn" id="checked">
+                                <property name="title" translatable="yes">checked</property>
+                                <child>
+                                  <object class="GtkCellRendererToggle" id="checkedRenderer">
+                                    <signal name="toggled" handler="on_locale_toggled" swapped="no"/>
+                                  </object>
+                                  <attributes>
+                                    <attribute name="active">2</attribute>
+                                    <attribute name="sensitive">3</attribute>
+                                  </attributes>
+                                </child>
+                              </object>
+                            </child>
+                            <child>
+                              <object class="GtkTreeViewColumn" id="localeNativeName">
+                                <property name="title" translatable="yes">nativeName</property>
+                                <property name="expand">True</property>
+                                <property name="clickable">True</property>
+                                <property name="sort_column_id">0</property>
+                                <child>
+                                  <object class="GtkCellRendererText" id="localeNativeNameRenderer">
+                                    <property name="font">Cantarell 12</property>
+                                  </object>
+                                  <attributes>
+                                    <attribute name="markup">0</attribute>
+                                  </attributes>
+                                </child>
+                              </object>
+                            </child>
+                          </object>
+                        </child>
+                      </object>
+                      <packing>
+                        <property name="left_attach">1</property>
+                        <property name="top_attach">0</property>
+                        <property name="width">1</property>
+                        <property name="height">1</property>
                       </packing>
                     </child>
+                    <child>
+                      <placeholder/>
+                    </child>
                   </object>
                   <packing>
-                    <property name="expand">True</property>
+                    <property name="expand">False</property>
                     <property name="fill">True</property>
                     <property name="position">1</property>
                   </packing>
@@ -204,4 +268,16 @@
       </object>
     </child>
   </object>
+  <object class="GtkListStore" id="localeStore">
+    <columns>
+      <!-- column-name nativeName -->
+      <column type="gchararray"/>
+      <!-- column-name locale -->
+      <column type="gchararray"/>
+      <!-- column-name selected -->
+      <column type="gboolean"/>
+      <!-- column-name isAdditional -->
+      <column type="gboolean"/>
+    </columns>
+  </object>
 </interface>
diff --git a/pyanaconda/ui/gui/spokes/langsupport.py b/pyanaconda/ui/gui/spokes/langsupport.py
index 1019457..dadcab8 100644
--- a/pyanaconda/ui/gui/spokes/langsupport.py
+++ b/pyanaconda/ui/gui/spokes/langsupport.py
@@ -17,6 +17,7 @@
 # Red Hat, Inc.
 #
 # Red Hat Author(s): Radek Vykydal 
+#                    Vratislav Podzimek 
 #

# pylint: disable-msg=E0611
@@ -26,6 +27,7 @@ from pyanaconda.i18n import _, N_
 from pyanaconda.iutil import strip_accents
 from pyanaconda.ui.gui.spokes import NormalSpoke
 from pyanaconda.ui.gui.categories.localization import LocalizationCategory
+from pyanaconda.ui.gui.utils import set_treeview_selection
 from pyanaconda import localization

import re
@@ -35,14 +37,8 @@ log = logging.getLogger("anaconda")

__all__ = ["LangsupportSpoke"]

-COL_NATIVE_NAME   = 0
-COL_ENGLISH_NAME  = 1
-COL_LANG_SETTING  = 2
-COL_SELECTED      = 3
-COL_IS_ADDITIONAL = 4
-
 class LangsupportSpoke(NormalSpoke):
-    builderObjects = ["langsupportStore", "langsupportStoreFilter", "langsupportWindow"]
+    builderObjects = ["languageStore", "languageStoreFilter", "localeStore", "langsupportWindow"]
     mainWidgetName = "langsupportWindow"
     uiFile = "spokes/langsupport.glade"

@@ -53,60 +49,59 @@ class LangsupportSpoke(NormalSpoke):

def __init__(self, *args, **kwargs):
         NormalSpoke.__init__(self, *args, **kwargs)
+        self._selected_locales = set()

def initialize(self):
-        self._langsupportStore = self.builder.get_object("langsupportStore")
-        self._langsupportEntry = self.builder.get_object("langsupportEntry")
-        self._langsupportStoreFilter = self.builder.get_object("langsupportStoreFilter")
-        self._langsupportStoreFilter.set_visible_func(self._matches_entry, None)
-
-        # mark selected items in language list bold
-        for col, rend, idx in (("englishNameCol", "englishNameRenderer", COL_ENGLISH_NAME),
-                               ("nativeNameCol", "nativeNameRenderer", COL_NATIVE_NAME)):
+        self._languageStore = self.builder.get_object("languageStore")
+        self._languageEntry = self.builder.get_object("languageEntry")
+        self._languageStoreFilter = self.builder.get_object("languageStoreFilter")
+        self._langView = self.builder.get_object("languageView")
+        self._langSelectedRenderer = self.builder.get_object("langSelectedRenderer")
+        self._langSelectedColumn = self.builder.get_object("langSelectedColumn")
+        self._langSelection = self.builder.get_object("languageViewSelection")
+        self._localeStore = self.builder.get_object("localeStore")
+        self._localeView = self.builder.get_object("localeView")
+
+        # fill the list with available translations
+        for lang in localization.get_available_translations():
+            self._add_language(self._languageStore,
+                               localization.get_native_name(lang),
+                               localization.get_english_name(lang), lang)
+
+        # render a right arrow for the chosen language
+        self._right_arrow = Gtk.Image.new_from_file("/usr/share/anaconda/pixmaps/right-arrow-icon.png")
+        self._langSelectedColumn.set_cell_data_func(self._langSelectedRenderer,
+                                                    self._render_lang_selected)
+
+        # mark selected locales and languages with selected locales bold
+        localeNativeColumn = self.builder.get_object("localeNativeName")
+        localeNativeNameRenderer = self.builder.get_object("localeNativeNameRenderer")
+        localeNativeColumn.set_cell_data_func(localeNativeNameRenderer,
+                                              self._mark_selected_locale_bold)
+
+        for col, rend in [("nativeName", "nativeNameRenderer"),
+                          ("englishName", "englishNameRenderer")]:
             column = self.builder.get_object(col)
             renderer = self.builder.get_object(rend)
-            column.set_cell_data_func(renderer, self._mark_selected_bold, idx)
-
-        # source of lang code <-> UI name mapping
-        # (localization.get_available_translations() returns a generator)
-        self.locale_infos_for_data = list(localization.get_available_translations())
-        self.locale_infos_for_ui = self.locale_infos_for_data[:]
+            column.set_cell_data_func(renderer, self._mark_selected_language_bold)

-        for locale in sorted(self.locale_infos_for_ui):
-            self._add_language(self._langsupportStore,
-                               localization.get_native_name(locale),
-                               localization.get_english_name(locale),
-                               locale,
-                               False, True)
-
-        self._select_language(self._langsupportStore, self.data.lang.lang)
+        # make filtering work
+        self._languageStoreFilter.set_visible_func(self._matches_entry, None)

def apply(self):
-        self.data.lang.addsupport = [row[COL_LANG_SETTING]
-                                     for row in self._langsupportStore
-                                     if row[COL_SELECTED] and row[COL_IS_ADDITIONAL]]
+        # store only additional langsupport locales
+        self.data.lang.addsupport = sorted(self._selected_locales - set([self.data.lang.lang]))

def refresh(self):
-        self._langsupportEntry.set_text("")
-        lang_infos = self._find_localeinfos_for_code(self.data.lang.lang, self.locale_infos_for_ui)
-        if len(lang_infos) > 1:
-            log.warning("Found multiple locales for lang %s: %s, picking first" %
-                        (self.data.lang.lang, lang_infos))
-        # Just take the first found
-        # TODO - for corner cases choose the one that is common prefix
-        lang_infos = lang_infos[:1]
-
-        addsupp_infos = []
-        for code in self.data.lang.addsupport:
-            code_infos = self._find_localeinfos_for_code(code, self.locale_infos_for_ui)
-            addsupp_infos.extend(code_infos)
-
-        for row in self._langsupportStore:
-            if row[COL_LANG_SETTING] in addsupp_infos:
-                row[COL_SELECTED] = True
-            if row[COL_LANG_SETTING] in lang_infos:
-                row[COL_SELECTED] = True
-                row[COL_IS_ADDITIONAL] = False
+        self._languageEntry.set_text("")
+        self._selected_locales = set(self._installed_langsupports)
+
+        # select the first locale from the "to be installed" langsupports
+        self._select_locale(self._installed_langsupports[0])
+
+    @property
+    def _installed_langsupports(self):
+        return [self.data.lang.lang] + sorted(self.data.lang.addsupport)

@property
     def showable(self):
@@ -114,13 +109,8 @@ class LangsupportSpoke(NormalSpoke):

@property
     def status(self):
-        # TODO: translate
-        infos = self._find_localeinfos_for_code(self.data.lang.lang, self.locale_infos_for_data)[:1]
-        for code in self.data.lang.addsupport:
-            for info in self._find_localeinfos_for_code(code, self.locale_infos_for_data):
-                if info not in infos:
-                    infos.append(info)
-        return ", ".join(localization.get_english_name(info) for info in infos)
+        return ", ".join(localization.get_native_name(locale)
+                         for locale in self._installed_langsupports)

@property
     def mandatory(self):
@@ -130,41 +120,63 @@ class LangsupportSpoke(NormalSpoke):
     def completed(self):
         return True

-    def _find_localeinfos_for_code(self, code, infos):
-        if code in infos:
-            return [code]
-        else:
-            retval = [info for info in infos
-                      if code in localization.expand_langs(info)]
-            log.debug("locale infos found for %s: %s" % (code, retval))
-            return retval
-
-    def _add_language(self, store, native, english, setting, selected, additional):
-        store.append(['<span lang="%s">%s</span>' % (re.sub(r'..*', '', setting), native),
-                     english, setting, selected, additional])
-
-    def _select_language(self, store, language):
-        itr = store.get_iter_first()
-        while itr and language not in localization.expand_langs(store[itr][COL_LANG_SETTING]):
-            itr = store.iter_next(itr)
-
-        # If we were provided with an unsupported language, just use the default.
-        if not itr:
-            return
-
-        treeview = self.builder.get_object("langsupportView")
-        selection = treeview.get_selection()
-        selection.select_iter(itr)
-        path = store.get_path(itr)
-        treeview.scroll_to_cell(path)
+    def _add_language(self, store, native, english, lang):
+        native_span = '<span lang="%s">%s</span>' % (lang, native)
+        store.append([native_span, english, lang])
+
+    def _addLocale(self, store, native, locale):
+        native_span = '<span lang="%s">%s</span>' % (re.sub(r'..*', '', locale), native)
+
+        # native, locale, selected, additional
+        store.append([native_span, locale, locale in self._selected_locales,
+                      locale != self.data.lang.lang])
+
+    def _refresh_locale_store(self, lang):
+        """Refresh the localeStore with locales for the given language."""
+
+        self._localeStore.clear()
+        locales = localization.get_language_locales(lang)
+        for locale in locales:
+            self._addLocale(self._localeStore,
+                            localization.get_native_name(locale),
+                            locale)
+
+        # select the first locale (with the highest rank)
+        set_treeview_selection(self._localeView, locales[0], col=1)
+
+    def _select_locale(self, locale):
+        """
+        Try to select the given locale in the language and locale
+        treeviews. This method tries to find the best match for the given
+        locale.
+
+        :return: a pair of selected iterators (language and locale)
+        :rtype: a pair of GtkTreeIter or None objects
+
+        """
+
+        # get lang and select it
+        parts = localization.parse_langcode(locale)
+        if "language" not in parts:
+            # invalid locale, cannot select
+            return (None, None)
+
+        lang_itr = set_treeview_selection(self._langView, parts["language"], col=2)
+        self._refresh_locale_store(parts["language"]) #XXX: needed?
+
+        # find matches and use the one with the highest rank
+        locales = localization.get_language_locales(locale)
+        locale_itr = set_treeview_selection(self._localeView, locales[0], col=1)
+
+        return (lang_itr, locale_itr)

def _matches_entry(self, model, itr, *args):
         # Need to strip out the pango markup before attempting to match.
         # Otherwise, starting to type "span" for "spanish" will match everything
         # due to the enclosing span tag.
         (success, attrs, native, accel) = Pango.parse_markup(model[itr][COL_NATIVE_NAME], -1, "_")
-        english = model[itr][COL_ENGLISH_NAME]
-        entry = self._langsupportEntry.get_text().strip()
+        english = model[itr][1]
+        entry = self._languageEntry.get_text().strip()

# Nothing in the text entry?  Display everything.
         if not entry:
@@ -179,21 +191,52 @@ class LangsupportSpoke(NormalSpoke):
         else:
             return False

-    def _mark_selected_bold(self, column, renderer, model, itr, idx):
-        value = model[itr][idx]
-        if model[itr][COL_SELECTED]:
-            value = "<b>%s</b>" % value
-        renderer.set_property("markup", value)
+    def _render_lang_selected(self, column, renderer, model, itr, user_data=None):
+        (lang_store, sel_itr) = self._langSelection.get_selected()
+
+        lang_locales = set(localization.get_language_locales(model[itr][2]))
+        if sel_itr and lang_store[sel_itr][2] == model[itr][2]:
+            renderer.set_property("pixbuf", self._right_arrow.get_pixbuf())
+        else:
+            renderer.set_property("pixbuf", None)
+
+    def _mark_selected_locale_bold(self, column, renderer, model, itr, user_data=None):
+        if model[itr][2]:
+            renderer.set_property("weight", Pango.Weight.BOLD.real)
+        else:
+            renderer.set_property("weight", Pango.Weight.NORMAL.real)

-    def on_langsupport_toggled(self, renderer, path):
-        selected = not self._langsupportStoreFilter[path][COL_SELECTED]
-        itr = self._langsupportStoreFilter.get_iter(path)
-        itr = self._langsupportStoreFilter.convert_iter_to_child_iter(itr)
-        self._langsupportStore[itr][COL_SELECTED] = selected
+    def _mark_selected_language_bold(self, column, renderer, model, itr, user_data=None):
+        lang_locales = set(localization.get_language_locales(model[itr][2]))
+        if not lang_locales.isdisjoint(self._selected_locales):
+            renderer.set_property("weight", Pango.Weight.BOLD.real)
+        else:
+            renderer.set_property("weight", Pango.Weight.NORMAL.real)
+
+    # Signal handlers.
+    def on_lang_selection_changed(self, selection):
+        (store, selected) = selection.get_selected_rows()
+
+        if selected:
+            lang = store[selected[0]][2]
+            self._refresh_locale_store(lang)
+        else:
+            self._localeStore.clear()
+
+    def on_locale_toggled(self, renderer, path):
+        itr = self._localeStore.get_iter(path)
+        row = self._localeStore[itr]
+
+        row[2] = not row[2]
+
+        if row[2]:
+            self._selected_locales.add(row[1])
+        else:
+            self._selected_locales.remove(row[1])

def on_clear_icon_clicked(self, entry, icon_pos, event):
         if icon_pos == Gtk.EntryIconPosition.SECONDARY:
             entry.set_text("")

def on_entry_changed(self, *args):
-        self._langsupportStoreFilter.refilter()
+        self._languageStoreFilter.refilter()

[PATCH 11/14] Share code between the Welcome and Langsupport spokes

[Vratislav Podzimek]

These two spokes do the same in a lot of places so they can share the code. It
doesn't result in a lot of removed lines, but a lot of things will be fixed only
at one place instead of two in the future.

Related: rhbz#1006458
(cherry-picked 0aae0cfb7e5f27866ba660b2f8b0974ceddd88e1 from master)

Signed-off-by: Vratislav Podzimek 
---
 pyanaconda/ui/gui/spokes/langsupport.py            | 105 +-------------
 .../ui/gui/spokes/lib/lang_locale_handler.py       | 161 +++++++++++++++++++++
 pyanaconda/ui/gui/spokes/welcome.py                | 109 ++------------
 3 files changed, 177 insertions(+), 198 deletions(-)
 create mode 100644 pyanaconda/ui/gui/spokes/lib/lang_locale_handler.py

diff --git a/pyanaconda/ui/gui/spokes/langsupport.py b/pyanaconda/ui/gui/spokes/langsupport.py
index dadcab8..85b84f7 100644
--- a/pyanaconda/ui/gui/spokes/langsupport.py
+++ b/pyanaconda/ui/gui/spokes/langsupport.py
@@ -26,6 +26,7 @@ from pyanaconda.flags import flags
 from pyanaconda.i18n import _, N_
 from pyanaconda.iutil import strip_accents
 from pyanaconda.ui.gui.spokes import NormalSpoke
+from pyanaconda.ui.gui.spokes.lib.lang_locale_handler import LangLocaleHandler
 from pyanaconda.ui.gui.categories.localization import LocalizationCategory
 from pyanaconda.ui.gui.utils import set_treeview_selection
 from pyanaconda import localization
@@ -37,7 +38,7 @@ log = logging.getLogger("anaconda")

__all__ = ["LangsupportSpoke"]

-class LangsupportSpoke(NormalSpoke):
+class LangsupportSpoke(LangLocaleHandler, NormalSpoke):
     builderObjects = ["languageStore", "languageStoreFilter", "localeStore", "langsupportWindow"]
     mainWidgetName = "langsupportWindow"
     uiFile = "spokes/langsupport.glade"
@@ -49,6 +50,7 @@ class LangsupportSpoke(NormalSpoke):

def __init__(self, *args, **kwargs):
         NormalSpoke.__init__(self, *args, **kwargs)
+        LangLocaleHandler.__init__(self)
         self._selected_locales = set()

def initialize(self):
@@ -62,16 +64,7 @@ class LangsupportSpoke(NormalSpoke):
         self._localeStore = self.builder.get_object("localeStore")
         self._localeView = self.builder.get_object("localeView")

-        # fill the list with available translations
-        for lang in localization.get_available_translations():
-            self._add_language(self._languageStore,
-                               localization.get_native_name(lang),
-                               localization.get_english_name(lang), lang)
-
-        # render a right arrow for the chosen language
-        self._right_arrow = Gtk.Image.new_from_file("/usr/share/anaconda/pixmaps/right-arrow-icon.png")
-        self._langSelectedColumn.set_cell_data_func(self._langSelectedRenderer,
-                                                    self._render_lang_selected)
+        LangLocaleHandler.initialize(self)

# mark selected locales and languages with selected locales bold
         localeNativeColumn = self.builder.get_object("localeNativeName")
@@ -85,9 +78,6 @@ class LangsupportSpoke(NormalSpoke):
             renderer = self.builder.get_object(rend)
             column.set_cell_data_func(renderer, self._mark_selected_language_bold)

-        # make filtering work
-        self._languageStoreFilter.set_visible_func(self._matches_entry, None)
-
     def apply(self):
         # store only additional langsupport locales
         self.data.lang.addsupport = sorted(self._selected_locales - set([self.data.lang.lang]))
@@ -124,82 +114,13 @@ class LangsupportSpoke(NormalSpoke):
         native_span = '<span lang="%s">%s</span>' % (lang, native)
         store.append([native_span, english, lang])

-    def _addLocale(self, store, native, locale):
+    def _add_locale(self, store, native, locale):
         native_span = '<span lang="%s">%s</span>' % (re.sub(r'..*', '', locale), native)

# native, locale, selected, additional
         store.append([native_span, locale, locale in self._selected_locales,
                       locale != self.data.lang.lang])

-    def _refresh_locale_store(self, lang):
-        """Refresh the localeStore with locales for the given language."""
-
-        self._localeStore.clear()
-        locales = localization.get_language_locales(lang)
-        for locale in locales:
-            self._addLocale(self._localeStore,
-                            localization.get_native_name(locale),
-                            locale)
-
-        # select the first locale (with the highest rank)
-        set_treeview_selection(self._localeView, locales[0], col=1)
-
-    def _select_locale(self, locale):
-        """
-        Try to select the given locale in the language and locale
-        treeviews. This method tries to find the best match for the given
-        locale.
-
-        :return: a pair of selected iterators (language and locale)
-        :rtype: a pair of GtkTreeIter or None objects
-
-        """
-
-        # get lang and select it
-        parts = localization.parse_langcode(locale)
-        if "language" not in parts:
-            # invalid locale, cannot select
-            return (None, None)
-
-        lang_itr = set_treeview_selection(self._langView, parts["language"], col=2)
-        self._refresh_locale_store(parts["language"]) #XXX: needed?
-
-        # find matches and use the one with the highest rank
-        locales = localization.get_language_locales(locale)
-        locale_itr = set_treeview_selection(self._localeView, locales[0], col=1)
-
-        return (lang_itr, locale_itr)
-
-    def _matches_entry(self, model, itr, *args):
-        # Need to strip out the pango markup before attempting to match.
-        # Otherwise, starting to type "span" for "spanish" will match everything
-        # due to the enclosing span tag.
-        (success, attrs, native, accel) = Pango.parse_markup(model[itr][COL_NATIVE_NAME], -1, "_")
-        english = model[itr][1]
-        entry = self._languageEntry.get_text().strip()
-
-        # Nothing in the text entry?  Display everything.
-        if not entry:
-            return True
-
-        # Otherwise, filter the list showing only what is matched by the
-        # text entry.  Either the English or native names can match.
-        lowered = entry.lower()
-        translit = strip_accents(unicode(native, "utf-8")).lower()
-        if lowered in native.lower() or lowered in english.lower() or lowered in translit:
-            return True
-        else:
-            return False
-
-    def _render_lang_selected(self, column, renderer, model, itr, user_data=None):
-        (lang_store, sel_itr) = self._langSelection.get_selected()
-
-        lang_locales = set(localization.get_language_locales(model[itr][2]))
-        if sel_itr and lang_store[sel_itr][2] == model[itr][2]:
-            renderer.set_property("pixbuf", self._right_arrow.get_pixbuf())
-        else:
-            renderer.set_property("pixbuf", None)
-
     def _mark_selected_locale_bold(self, column, renderer, model, itr, user_data=None):
         if model[itr][2]:
             renderer.set_property("weight", Pango.Weight.BOLD.real)
@@ -214,15 +135,6 @@ class LangsupportSpoke(NormalSpoke):
             renderer.set_property("weight", Pango.Weight.NORMAL.real)

# Signal handlers.
-    def on_lang_selection_changed(self, selection):
-        (store, selected) = selection.get_selected_rows()
-
-        if selected:
-            lang = store[selected[0]][2]
-            self._refresh_locale_store(lang)
-        else:
-            self._localeStore.clear()
-
     def on_locale_toggled(self, renderer, path):
         itr = self._localeStore.get_iter(path)
         row = self._localeStore[itr]
@@ -233,10 +145,3 @@ class LangsupportSpoke(NormalSpoke):
             self._selected_locales.add(row[1])
         else:
             self._selected_locales.remove(row[1])
-
-    def on_clear_icon_clicked(self, entry, icon_pos, event):
-        if icon_pos == Gtk.EntryIconPosition.SECONDARY:
-            entry.set_text("")
-
-    def on_entry_changed(self, *args):
-        self._languageStoreFilter.refilter()
diff --git a/pyanaconda/ui/gui/spokes/lib/lang_locale_handler.py b/pyanaconda/ui/gui/spokes/lib/lang_locale_handler.py
new file mode 100644
index 0000000..dea6bef
--- /dev/null
+++ b/pyanaconda/ui/gui/spokes/lib/lang_locale_handler.py
@@ -0,0 +1,161 @@
+#
+# Copyright (C) 2011-2012  Red Hat, Inc.
+#
+# This copyrighted material is made available to anyone wishing to use,
+# modify, copy, or redistribute it subject to the terms and conditions of
+# the GNU General Public License v.2, or (at your option) any later version.
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY expressed or implied, including the implied warranties of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General
+# Public License for more details.  You should have received a copy of the
+# GNU General Public License along with this program; if not, write to the
+# Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+# 02110-1301, USA.  Any Red Hat trademarks that are incorporated in the
+# source code or documentation are not subject to the GNU General Public
+# License and may only be used or replicated with the express permission of
+# Red Hat, Inc.
+#
+# Red Hat Author(s): Vratislav Podzimek 
+#
+
+"""
+Module providing the LangLocaleHandler class that could be used as a mixin for
+screens handling languages or locales configuration.
+
+"""
+
+from gi.repository import Gtk, Pango
+from pyanaconda import localization
+from pyanaconda.iutil import strip_accents
+from pyanaconda.ui.gui.utils import set_treeview_selection
+
+class LangLocaleHandler(object):
+    """
+    Class that could be used as a mixin for screens handling languages or
+    locales configuration.
+
+    """
+
+    def __init__(self):
+        # the class inheriting from this class is responsible for populating
+        # these items with actual objects
+        self._languageStore = None
+        self._languageStoreFilter = None
+        self._languageEntry = None
+        self._langSelection = None
+        self._langSelectedRenderer = None
+        self._langSelectedColumn = None
+        self._langView = None
+        self._localeView = None
+        self._localeStore = None
+        self._localeSelection = None
+
+    def initialize(self):
+        # Render a right arrow for the chosen language
+        self._right_arrow = Gtk.Image.new_from_file("/usr/share/anaconda/pixmaps/right-arrow-icon.png")
+        self._langSelectedColumn.set_cell_data_func(self._langSelectedRenderer,
+                                                    self._render_lang_selected)
+
+        # fill the list with available translations
+        for lang in localization.get_available_translations():
+            self._add_language(self._languageStore,
+                               localization.get_native_name(lang),
+                               localization.get_english_name(lang), lang)
+
+        # make filtering work
+        self._languageStoreFilter.set_visible_func(self._matches_entry, None)
+
+    def _matches_entry(self, model, itr, *args):
+        # Nothing in the text entry?  Display everything.
+        entry = self._languageEntry.get_text().strip()
+        if not entry:
+            return True
+
+        # Need to strip out the pango markup before attempting to match.
+        # Otherwise, starting to type "span" for "spanish" will match everything
+        # due to the enclosing span tag.
+        # (success, attrs, native, accel)
+        native = Pango.parse_markup(model[itr][0], -1, "_")[2]
+        english = model[itr][1]
+
+        # Otherwise, filter the list showing only what is matched by the
+        # text entry.  Either the English or native names can match.
+        lowered = entry.lower()
+        translit = strip_accents(unicode(native, "utf-8")).lower()
+        if lowered in native.lower() or lowered in english.lower() or lowered in translit:
+            return True
+        else:
+            return False
+
+    def _render_lang_selected(self, column, renderer, model, itr, user_data=None):
+        (lang_store, sel_itr) = self._langSelection.get_selected()
+
+        if sel_itr and lang_store[sel_itr][2] == model[itr][2]:
+            renderer.set_property("pixbuf", self._right_arrow.get_pixbuf())
+        else:
+            renderer.set_property("pixbuf", None)
+
+    def _add_language(self, store, native, english, lang):
+        """Override this method with a valid implementation"""
+
+        raise NotImplementedError()
+
+    def _add_locale(self, store, native, locale):
+        """Override this method with a valid implementation"""
+
+        raise NotImplementedError()
+
+    def _select_locale(self, locale):
+        """
+        Try to select the given locale in the language and locale
+        treeviews. This method tries to find the best match for the given
+        locale.
+
+        :return: a pair of selected iterators (language and locale)
+        :rtype: a pair of GtkTreeIter or None objects
+
+        """
+
+        # get lang and select it
+        parts = localization.parse_langcode(locale)
+        if "language" not in parts:
+            # invalid locale, cannot select
+            return (None, None)
+
+        lang_itr = set_treeview_selection(self._langView, parts["language"], col=2)
+
+        # find matches and use the one with the highest rank
+        locales = localization.get_language_locales(locale)
+        locale_itr = set_treeview_selection(self._localeView, locales[0], col=1)
+
+        return (lang_itr, locale_itr)
+
+    def _refresh_locale_store(self, lang):
+        """Refresh the localeStore with locales for the given language."""
+
+        self._localeStore.clear()
+        locales = localization.get_language_locales(lang)
+        for locale in locales:
+            self._add_locale(self._localeStore,
+                             localization.get_native_name(locale),
+                             locale)
+
+        # select the first locale (with the highest rank)
+        set_treeview_selection(self._localeView, locales[0], col=1)
+
+    def on_lang_selection_changed(self, selection):
+        (store, selected) = selection.get_selected_rows()
+
+        if selected:
+            lang = store[selected[0]][2]
+            self._refresh_locale_store(lang)
+        else:
+            self._localeStore.clear()
+
+    def on_clear_icon_clicked(self, entry, icon_pos, event):
+        if icon_pos == Gtk.EntryIconPosition.SECONDARY:
+            entry.set_text("")
+
+    def on_entry_changed(self, *args):
+        self._languageStoreFilter.refilter()
+
diff --git a/pyanaconda/ui/gui/spokes/welcome.py b/pyanaconda/ui/gui/spokes/welcome.py
index 14427df..90d3e10 100644
--- a/pyanaconda/ui/gui/spokes/welcome.py
+++ b/pyanaconda/ui/gui/spokes/welcome.py
@@ -28,6 +28,7 @@ from gi.repository import Gtk, Pango
 from pyanaconda.ui.gui.hubs.summary import SummaryHub
 from pyanaconda.ui.gui.spokes import StandaloneSpoke
 from pyanaconda.ui.gui.utils import enlightbox, set_treeview_selection
+from pyanaconda.ui.gui.spokes.lib.lang_locale_handler import LangLocaleHandler

from pyanaconda import localization
 from pyanaconda.product import distributionText, isFinal, productName, productVersion
@@ -43,7 +44,7 @@ log = logging.getLogger("anaconda")

__all__ = ["WelcomeLanguageSpoke"]

-class WelcomeLanguageSpoke(StandaloneSpoke):
+class WelcomeLanguageSpoke(LangLocaleHandler, StandaloneSpoke):
     mainWidgetName = "welcomeWindow"
     uiFile = "spokes/welcome.glade"
     builderObjects = ["languageStore", "languageStoreFilter", "localeStore",
@@ -54,6 +55,7 @@ class WelcomeLanguageSpoke(StandaloneSpoke):

def __init__(self, *args, **kwargs):
         StandaloneSpoke.__init__(self, *args, **kwargs)
+        LangLocaleHandler.__init__(self)
         self._xklwrapper = keyboard.XklWrapper.get_instance()
         self._origStrings = {}

@@ -142,14 +144,6 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
     def _row_is_separator(self, model, itr, *args):
         return model[itr][3]

-    def _render_lang_selected(self, column, renderer, model, itr, user_data=None):
-        (lang_store, sel_itr) = self._langSelection.get_selected()
-
-        if sel_itr and lang_store[sel_itr][2] == model[itr][2]:
-            renderer.set_property("pixbuf", self._right_arrow.get_pixbuf())
-        else:
-            renderer.set_property("pixbuf", None)
-
     def initialize(self):
         self._languageStore = self.builder.get_object("languageStore")
         self._languageStoreFilter = self.builder.get_object("languageStoreFilter")
@@ -162,14 +156,11 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
         self._localeStore = self.builder.get_object("localeStore")
         self._localeSelection = self.builder.get_object("localeViewSelection")

+        LangLocaleHandler.initialize(self)
+
         # We need to tell the view whether something is a separator or not.
         self._langView.set_row_separator_func(self._row_is_separator, None)

-        # Render a right arrow for the chosen language
-        self._right_arrow = Gtk.Image.new_from_file("/usr/share/anaconda/pixmaps/right-arrow-icon.png")
-        self._langSelectedColumn.set_cell_data_func(self._langSelectedRenderer,
-                                                    self._render_lang_selected)
-
         # We can use the territory from geolocation here
         # to preselect the translation, when it's available.
         territory = geoloc.get_territory_code()
@@ -180,12 +171,6 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
             # kickstart/command line
             localization.setup_locale(locales[0], self.data.lang)

-        # fill the list with available translations
-        for lang in localization.get_available_translations():
-            self._addLanguage(self._languageStore,
-                              localization.get_native_name(lang),
-                              localization.get_english_name(lang), lang)
-
         # Move the default language (whatever was provided on the command line,
         # or by kickstart, or by geoip, or English if nothing else) to the top
         # of the list and select it by default.  People find it confusing to be
@@ -209,8 +194,6 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
         newItr = store.insert(1)
         store.set(newItr, 0, "", 1, "", 2, "", 3, True)

-        self._languageStoreFilter.set_visible_func(self._matchesEntry, None)
-
     def _retranslate_one(self, widgetName):
         widget = self.builder.get_object(widgetName)
         if not widget:
@@ -237,7 +220,7 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
             self._origStrings[welcomeLabel] = welcomeLabel.get_label()

before = self._origStrings[welcomeLabel]
-        xlated = _(before) % {"name" : productName.upper(), "version" : productVersion}
+        xlated = _(before) % (productName.upper(), productVersion)
         welcomeLabel.set_label(xlated)

# And of course, don't forget the underlying window.
@@ -249,84 +232,21 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
         self._languageEntry.set_text("")
         self._languageStoreFilter.refilter()

-    def _addLanguage(self, store, native, english, lang):
+    def _add_language(self, store, native, english, lang):
         native_span = '<span lang="%s">%s</span>' % (lang, native)
         store.append([native_span, english, lang, False])

-    def _addLocale(self, store, native, locale):
+    def _add_locale(self, store, native, locale):
         native_span = '<span lang="%s">%s</span>' % (re.sub(r'..*', '', locale), native)
         store.append([native_span, locale])

-    def _matchesEntry(self, model, itr, *args):
-        # Need to strip out the pango markup before attempting to match.
-        # Otherwise, starting to type "span" for "spanish" will match everything
-        # due to the enclosing span tag.
-        (success, attrs, native, accel) = Pango.parse_markup(model[itr][0], -1, "_")
-        english = model[itr][1]
-        entry = self._languageEntry.get_text().strip()
-
-        # Nothing in the text entry?  Display everything.
-        if not entry:
-            return True
-
-        # Otherwise, filter the list showing only what is matched by the
-        # text entry.  Either the English or native names can match.
-        lowered = entry.lower()
-        translit = strip_accents(unicode(native, "utf-8")).lower()
-        if lowered in native.lower() or lowered in english.lower() or lowered in translit:
-            return True
-        else:
-            return False
-
-    def _select_locale(self, locale):
-        """
-        Try to select the given locale in the language and locale
-        treeviews. This method tries to find the best match for the given
-        locale.
-
-        :return: a pair of selected iterators (language and locale)
-        :rtype: a pair of GtkTreeIter or None objects
-
-        """
-
-        # get lang and select it
-        parts = localization.parse_langcode(locale)
-        if "language" not in parts:
-            # invalid locale, cannot select
-            return (None, None)
-
-        lang_itr = set_treeview_selection(self._langView, parts["language"], col=2)
-
-        # find matches and use the one with the highest rank
-        locales = localization.get_language_locales(locale)
-        locale_itr = set_treeview_selection(self._localeView, locales[0], col=1)
-
-        return (lang_itr, locale_itr)
-
-    def _refresh_locale_store(self, lang):
-        """Refresh the localeStore with locales for the given language."""
-
-        self._localeStore.clear()
-        locales = localization.get_language_locales(lang)
-        for locale in locales:
-            self._addLocale(self._localeStore,
-                            localization.get_native_name(locale),
-                            locale)
-
-        # select the first locale (with the highest rank)
-        set_treeview_selection(self._localeView, locales[0], col=1)
-
     # Signal handlers.
     def on_lang_selection_changed(self, selection):
         (store, selected) = selection.get_selected_rows()
+        LangLocaleHandler.on_lang_selection_changed(self, selection)

-        if selected:
-            lang = store[selected[0]][2]
-            self._refresh_locale_store(lang)
-        else:
-            if hasattr(self.window, "set_may_continue"):
-                self.window.set_may_continue(False)
-            self._localeStore.clear()
+        if not selected and hasattr(self.window, "set_may_continue"):
+            self.window.set_may_continue(False)

def on_locale_selection_changed(self, selection):
         (store, selected) = selection.get_selected_rows()
@@ -338,13 +258,6 @@ class WelcomeLanguageSpoke(StandaloneSpoke):
             localization.setup_locale(lang)
             self.retranslate(lang)

-    def on_clear_icon_clicked(self, entry, icon_pos, event):
-        if icon_pos == Gtk.EntryIconPosition.SECONDARY:
-            entry.set_text("")
-
-    def on_entry_changed(self, *args):
-        self._languageStoreFilter.refilter()
-
     # Override the default in StandaloneSpoke so we can display the beta
     # warning dialog first.
     def _on_continue_clicked(self, cb):

[PATCH 12/14] More robust parsing of the layout and variant string specification

[Vratislav Podzimek]

We may get strings having more or less whitespace than we expect as keyboard
layout and variant specification strings from external libraries or from users
in kickstarts or as command-line options and we should handle those cases.

Signed-off-by: Vratislav Podzimek 
---
 pyanaconda/keyboard.py | 44 ++++++++++++++++++++++++++++++++------------
 1 file changed, 32 insertions(+), 12 deletions(-)

diff --git a/pyanaconda/keyboard.py b/pyanaconda/keyboard.py
index 3ea4826..138b4d0 100755
--- a/pyanaconda/keyboard.py
+++ b/pyanaconda/keyboard.py
@@ -34,6 +34,7 @@ keymaps.

import types
 import os
+import re
 import shutil

from pyanaconda import iutil
@@ -51,32 +52,48 @@ LOCALED_SERVICE = "org.freedesktop.locale1"
 LOCALED_OBJECT_PATH = "/org/freedesktop/locale1"
 LOCALED_IFACE = "org.freedesktop.locale1"

-DEFAULT_VC_FONT = "latarcyrheb-sun16"
+# should match and parse strings like 'cz' or 'cz (qwerty)' regardless of white
+# space
+LAYOUT_VARIANT_RE = re.compile(r'^\s*(\w+)\s*' # layout plus
+                               r'(?:(?:(\s*(\w+)\s*))' # variant in parentheses
+                               r'|(?:$))\s*') # or nothing

class KeyboardConfigError(Exception):
     """Exception class for keyboard configuration related problems"""

pass

-def _parse_layout_variant(layout):
+class InvalidLayoutVariantSpec(Exception):
+    """
+    Exception class for errors related to parsing layout and variant specification strings.
+
+    """
+
+    pass
+
+def _parse_layout_variant(layout_variant_str):
     """
     Parse layout and variant from the string that may look like 'layout' or
     'layout (variant)'.

+    :param layout_variant_str: keyboard layout and variant string specification
+    :type layout_variant_str: str
     :return: the (layout, variant) pair, where variant can be ""
     :rtype: tuple
+    :raise InvalidLayoutVariantSpec: if the given string isn't a valid layout
+                                     and variant specification string

"""

-    variant = ""
+    match = LAYOUT_VARIANT_RE.match(layout_variant_str)
+    if not match:
+        msg = "'%s' is not a valid keyboard layout and variant specification" % layout_variant_str
+        raise InvalidLayoutVariantSpec(msg)

-    lbracket_idx = layout.find("(")
-    rbracket_idx = layout.rfind(")")
-    if lbracket_idx != -1:
-        variant = layout[(lbracket_idx + 1) : rbracket_idx]
-        layout = layout[:lbracket_idx].strip()
+    layout, variant = match.groups()

-    return (layout, variant)
+    # groups may be (layout, None) if no variant was specified
+    return (layout, variant or "")

def _join_layout_variant(layout, variant=""):
     """
@@ -577,12 +594,15 @@ class XklWrapper(object):
         'cz (qwerty)').

:param layout: either 'layout' or 'layout (variant)'
-        :raise XklWrapperError: if the given layout cannot be added
+        :raise XklWrapperError: if the given layout is invalid or cannot be added

"""

-        #we can get 'layout' or 'layout (variant)'
-        (layout, variant) = _parse_layout_variant(layout)
+        try:
+            #we can get 'layout' or 'layout (variant)'
+            (layout, variant) = _parse_layout_variant(layout)
+        except InvalidLayoutVariantSpec as ilverr:
+            raise XklWrapperError("Failed to add layout: %s" % ilverr)

#do not add the same layout-variant combinanion multiple times
         if (layout, variant) in zip(self._rec.layouts, self._rec.variants):

[PATCH 13/14] Normalize keyboard layout and variant strings from langtable

[Vratislav Podzimek]

Langtable uses e.g. 'cz(qwerty)' as a layout ('cz') and variant ('qwerty')
specification. However, we use strings like 'cz (qwerty)' (with a space
separating the fields) as IDs in dictionaries etc., so we need to normalize all
strings to the same format.

Related: rhbz#1006458
(cherry-picked 8cfac2bf646d91b75af174f89424d882cfd1500b from master)

Signed-off-by: Vratislav Podzimek 
---
 pyanaconda/keyboard.py              | 15 +++++++++++++++
 pyanaconda/ui/gui/spokes/welcome.py |  5 +++--
 2 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/pyanaconda/keyboard.py b/pyanaconda/keyboard.py
index 138b4d0..70b67ce 100755
--- a/pyanaconda/keyboard.py
+++ b/pyanaconda/keyboard.py
@@ -112,6 +112,21 @@ def _join_layout_variant(layout, variant=""):
     else:
         return layout

+def normalize_layout_variant(layout_str):
+    """
+    Normalize keyboard layout and variant specification given as a single
+    string. E.g. for a 'layout(variant) string missing the space between the
+    left parenthesis return 'layout (variant)' which is a proper layout and
+    variant specification we use.
+
+    :param layout_str: a string specifying keyboard layout and its variant
+    :type layout_str: string
+
+    """
+
+    layout, variant = _parse_layout_variant(layout_str)
+    return _join_layout_variant(layout, variant)
+
 def populate_missing_items(keyboard):
     """
     Function that populates keyboard.vc_keymap and keyboard.x_layouts if
diff --git a/pyanaconda/ui/gui/spokes/welcome.py b/pyanaconda/ui/gui/spokes/welcome.py
index 90d3e10..45abe88 100644
--- a/pyanaconda/ui/gui/spokes/welcome.py
+++ b/pyanaconda/ui/gui/spokes/welcome.py
@@ -111,8 +111,9 @@ class WelcomeLanguageSpoke(LangLocaleHandler, StandaloneSpoke):

layouts = localization.get_locale_keyboards(locale)
         if layouts:
-            # take the first locale (with highest rank) from the list
-            new_layouts = [layouts[0]]
+            # take the first locale (with highest rank) from the list and
+            # store it normalized
+            new_layouts = [keyboard.normalize_layout_variant(layouts[0])]
             if not langtable.supports_ascii(layouts[0]):
                 # does not support typing ASCII chars, append the 'us' layout
                 new_layouts.append("us")

[PATCH 14/14] Center the Langsupport spoke's description

[Vratislav Podzimek]

Also add a colon at the end of the string and add some vertical space between it
and the language/locale scroll boxes.

Related: rhbz#1006458
(cherry-picked dda7ef7676b346a459bfd1c367b8e5cfd138d1ad from master)

Signed-off-by: Vratislav Podzimek 
---
 pyanaconda/ui/gui/spokes/langsupport.glade | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/pyanaconda/ui/gui/spokes/langsupport.glade b/pyanaconda/ui/gui/spokes/langsupport.glade
index 6727c9b..f0dabf6 100644
--- a/pyanaconda/ui/gui/spokes/langsupport.glade
+++ b/pyanaconda/ui/gui/spokes/langsupport.glade
@@ -58,13 +58,13 @@
               <object class="GtkBox" id="AnacondaSpokeWindow-action_area1">
                 <property name="can_focus">False</property>
                 <property name="orientation">vertical</property>
-                <property name="spacing">6</property>
+                <property name="spacing">12</property>
                 <child>
                   <object class="GtkLabel" id="selectLabel">
                     <property name="visible">True</property>
                     <property name="can_focus">False</property>
-                    <property name="halign">start</property>
-                    <property name="label" translatable="yes">Select additional language support to be installed</property>
+                    <property name="halign">center</property>
+                    <property name="label" translatable="yes">Select additional language support to be installed:</property>
                     <attributes>
                       <attribute name="font-desc" value="Cantarell 14"/>
                       <attribute name="weight" value="normal"/>

[Brian C. Lane]

On Wed, Sep 25, 2013 at 01:27:43PM -0400, Vratislav Podzimek wrote:
> There has been major changes of the language/locale handling code done in the
> F20 cycle which significantly improve user experience of the installer and also
> make our code cleaner, more deterministic and better documented.
>
> On behalf of the bug #1006458 [1] all these 14 patches should be ported from
> master to rhel7-branch. They contain a lot of changes, but they are just ports
> of the master patches, so they should be okay.
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1006458

Ack. I tested them against a nightly boot.iso and did not see any
problems.