anaconda-patches December 2015

anaconda-patches@lists.fedorahosted.org

9 participants
498 discussions

[master 8/9] Fix the tests for test_percentage.
by dashea 01 Dec '15

01 Dec '15

From: David Shea <dshea(a)redhat.com> Ignore DeprecationWarnings so that the deprecation warning from polib doesn't get in the way of the test, and actually call test_percentage in test_not_ok. --- tests/unittests/test_translated.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tests/unittests/test_translated.py b/tests/unittests/test_translated.py index 7011cb7..2f90634 100644 --- a/tests/unittests/test_translated.py +++ b/tests/unittests/test_translated.py @@ -77,7 +77,11 @@ def setUp(self): self.tmpdir = tempfile.mkdtemp() self.popath = os.path.join(self.tmpdir, "test.po") self.mopath = os.path.join(self.tmpdir, "test.mo") + + # Convert warnings into exceptions to make them easier to test for warnings.simplefilter("error") + # polib throws a DeprecationWarnings so ignore that + warnings.simplefilter("default", DeprecationWarning) def tearDown(self): shutil.rmtree(self.tmpdir) @@ -98,6 +102,8 @@ def test_not_ok(self): pofile.save(self.popath) pofile.save_as_mofile(self.mopath) + self.assertRaises(Warning, test_percentage, self.mopath) + class TestUsability(unittest.TestCase): def test_ok(self): # what lt's Plural-Forms is supposed to look like -- To view this commit on github, visit https://github.com/rhinstaller/anaconda/commit/45245ebebd1c2922bc3faaf0395c…

1 0

[master 7/9] Added a ci make target
by dashea 01 Dec '15

01 Dec '15

From: David Shea <dshea(a)redhat.com> --- Makefile | 3 +++ 1 file changed, 3 insertions(+) diff --git a/Makefile b/Makefile index d9b633e..c264b45 100644 --- a/Makefile +++ b/Makefile @@ -6,3 +6,6 @@ all: check: PYTHONPATH=. tests/pylint/runpylint.py python3 -m unittest discover tests/unittests + +ci: + $(MAKE) check -- To view this commit on github, visit https://github.com/rhinstaller/anaconda/commit/f82a6754069bc27c72e66b475715…

1 0

[master 6/9] Run the translation-canary/translated tests
by dashea 01 Dec '15

01 Dec '15

From: David Shea <dshea(a)redhat.com> These tests will be run on the source tarball during 'make release' and 'make ci'. --- Makefile.am | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/Makefile.am b/Makefile.am index 7920e6b..2583b48 100644 --- a/Makefile.am +++ b/Makefile.am @@ -96,7 +96,9 @@ scratch-bumpver: po-empty PYTHONPATH=$(srcdir)/translation-canary python3 -m translatable $(srcdir)/po/$(PACKAGE_NAME).pot release: - $(MAKE) dist && $(MAKE) tag && git checkout -- $(srcdir)/po/$(PACKAGE_NAME).pot + $(MAKE) dist + PYTHONPATH=$(srcdir)/translation-canary python3 -m translated $(PACKAGE_NAME)-$(PACKAGE_VERSION).tar.bz2 + git checkout -- $(srcdir)/po/$(PACKAGE_NAME).pot rc-release: scratch-bumpver scratch mock -r $(MOCKCHROOT) --scrub all || exit 1 @@ -177,6 +179,7 @@ runglade: glade ${GLADE_FILE} ci: rc-release + @PYTHONPATH=$(srcdir)/translation-canary python3 -m translated $(PACKAGE_NAME)-$(PACKAGE_VERSION).tar.bz2 @rm -f tests/test-suite.log.* @mkdir -p $(USER_SITE_PACKAGES) @cp $(abs_builddir)/tests/usercustomize.py $(USER_SITE_PACKAGES) -- To view this commit on github, visit https://github.com/rhinstaller/anaconda/commit/34cb84ee75cd75d7058ee1a499fc…

1 0

[master 5/9] Run translation-canary/translatable tests after pot-update
by dashea 01 Dec '15

01 Dec '15

From: David Shea <dshea(a)redhat.com> This ensures that unsuitable strings do not end up in the POT file. Using the xgettext_werror wrapper replaces the function of the gettext_warnings test. --- Makefile.am | 7 ++++++ po/Makevars | 3 +++ tests/Makefile.am | 2 -- tests/gettext_tests/gettext_warnings.sh | 44 --------------------------------- 4 files changed, 10 insertions(+), 46 deletions(-) delete mode 100755 tests/gettext_tests/gettext_warnings.sh diff --git a/Makefile.am b/Makefile.am index 3e0ae5a..7920e6b 100644 --- a/Makefile.am +++ b/Makefile.am @@ -23,6 +23,11 @@ SUBDIRS = data docs dracut po pyanaconda scripts tests widgets utils EXTRA_DIST = config.rpath COPYING .coveragerc +# Include the "translatable" files from translation-canary for POT updates +# during distcheck +EXTRA_DIST += $(srcdir)/translation-canary/translatable/*.py \ + $(srcdir)/translation-canary/translatable/*.sh + # Files generated for gettext need to be included in the distribution EXTRA_DIST += $(srcdir)/anaconda.po $(srcdir)/anaconda.po: $(srcdir)/anaconda @@ -88,6 +93,7 @@ scratch-bumpver: po-empty fi ; \ ( cd $(srcdir) && scripts/makebumpver --skip-zanata $${opts} ) || exit 1 ; \ $(MAKE) -C po $(PACKAGE_NAME).pot-update + PYTHONPATH=$(srcdir)/translation-canary python3 -m translatable $(srcdir)/po/$(PACKAGE_NAME).pot release: $(MAKE) dist && $(MAKE) tag && git checkout -- $(srcdir)/po/$(PACKAGE_NAME).pot @@ -113,6 +119,7 @@ bumpver: po-pull fi ; \ ( cd $(srcdir) && scripts/makebumpver $${opts} ) || exit 1 ; \ $(MAKE) -C po $(PACKAGE_NAME).pot-update && \ + PYTHONPATH=$(srcdir)/translation-canary python3 -m translatable $(srcdir)/po/$(PACKAGE_NAME).pot && \ zanata push $(ZANATA_PUSH_ARGS) # Install all packages specified as BuildRequires in the Anaconda specfile diff --git a/po/Makevars b/po/Makevars index f52adbe..f6a96d1 100644 --- a/po/Makevars +++ b/po/Makevars @@ -7,6 +7,9 @@ DOMAIN = $(PACKAGE) subdir = po top_builddir = .. +# Wrap xgettext in a script that fails on warnings +XGETTEXT = $(top_srcdir)/translation-canary/translatable/xgettext_werror.sh + # These options get passed to xgettext. XGETTEXT_OPTIONS = --keyword=_ --keyword=N_ --keyword=P_:1,2 --keyword=C_:1c,2 --keyword=CN_:1c,2 --keyword=CP_:1c,2,3 --from-code=UTF-8 diff --git a/tests/Makefile.am b/tests/Makefile.am index be2bdf0..6ea640d 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -46,7 +46,6 @@ dist_check_SCRIPTS = $(srcdir)/glade/*.py \ cppcheck/runcppcheck.sh \ testenv.sh \ $(srcdir)/gettext_tests/*.py \ - gettext_tests/gettext_warnings.sh \ storage/run_storage_tests.py \ install/run_install_test.sh \ $(srcdir)/gui/*.ks \ @@ -60,7 +59,6 @@ TESTS = nosetests.sh \ pylint/runpylint.py \ cppcheck/runcppcheck.sh \ gettext_tests/click.py \ - gettext_tests/gettext_warnings.sh \ gettext_tests/gettext_potfiles.py \ gettext_tests/style_guide.py \ gettext_tests/contexts.py \ diff --git a/tests/gettext_tests/gettext_warnings.sh b/tests/gettext_tests/gettext_warnings.sh deleted file mode 100755 index 2974e32..0000000 --- a/tests/gettext_tests/gettext_warnings.sh +++ /dev/null @@ -1,44 +0,0 @@ -#!/bin/sh -e - -if [ -z "$top_srcdir" ]; then - top_srcdir="$(dirname "$0")/../.." - . "${top_srcdir}/tests/testenv.sh" -fi -podir="${top_builddir}/po" -POTFILES="${podir}/POTFILES" - -# Extract XGETTEXT_OPTIONS from po/Makevars -# Makevars is one of our files, so it's in $srcdir -XGETTEXT_OPTIONS="$(sed -n 's/^[[:space:]]*XGETTEXT_OPTIONS[[:space:]]*=[[:space:]]*$.*$/\1/p' \ - "${top_srcdir}/po/Makevars")" - -# Fail if POTFILES doesn't exist, since set -e doesn't catch this for some -# dumb reason that I'm sure has a long and storied history -if [ ! -f "${POTFILES}" ] ; then - echo "POTFILES does not exist" - exit 1 -fi - -status=0 -# For each file in POTFILES, run xgettext and look for warnings -while read -r potfile ; do - # Strip the spaces and trailing backslash - potfile="$(echo "$potfile" | sed 's/^[[:space:]]*$[^[:space:]]*$[[:space:]]*\\\?$/\1/')" - - # $potfile is relative to the po/ directory, $test_potfile is - # relative to this script's working directory - test_potfile="${podir}/${potfile}" - - # If the file doesn't exist, try to make it - if [ ! -f "$test_potfile" ]; then - make -C "${podir}" "${potfile}" || exit 1 - fi - - xgettext_output="$(xgettext ${XGETTEXT_OPTIONS} -o /dev/null "$test_potfile" 2>&1)" || status=1 - if echo "$xgettext_output" | fgrep -q 'warning:' ; then - echo "$xgettext_output" - status=1 - fi -done < "$POTFILES" - -exit "$status" -- To view this commit on github, visit https://github.com/rhinstaller/anaconda/commit/ef40c961d78b3b2d51b5ea4bf116…

1 0

[PATCH 1/3] Initial framework and tests
by dashea 01 Dec '15

01 Dec '15

From: David Shea <dshea(a)redhat.com> The initial set of tests is: - translated strings come back with the same markup - translated .po files don't just completely bomb out - translations are > 10% translated (warning only, since no anonymous pull support makes this one awkward for running in jenkins) - translatable strings don't contain unnecessary markup - single-character translatable strings include a comment explaining what in the heck There is also a xgettext wrapper script in translatable that makes xgettext warnings louder. This one is not run with the rest of the tests since xgettext needs the original source file for these instead of just the POT file. --- .gitignore | 2 + README.rst | 36 +++++++++++++++ translatable/__init__.py | 85 +++++++++++++++++++++++++++++++++++ translatable/__main__.py | 33 ++++++++++++++ translatable/test_comment.py | 26 +++++++++++ translatable/test_markup.py | 38 ++++++++++++++++ translatable/xgettext_werror.sh | 46 +++++++++++++++++++ translated/__init__.py | 99 +++++++++++++++++++++++++++++++++++++++++ translated/__main__.py | 33 ++++++++++++++ translated/test_markup.py | 44 ++++++++++++++++++ translated/test_percentage.py | 42 +++++++++++++++++ translated/test_usability.py | 28 ++++++++++++ 12 files changed, 512 insertions(+) create mode 100644 .gitignore create mode 100644 README.rst create mode 100644 translatable/__init__.py create mode 100644 translatable/__main__.py create mode 100644 translatable/test_comment.py create mode 100644 translatable/test_markup.py create mode 100755 translatable/xgettext_werror.sh create mode 100644 translated/__init__.py create mode 100644 translated/__main__.py create mode 100644 translated/test_markup.py create mode 100644 translated/test_percentage.py create mode 100644 translated/test_usability.py diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..8d35cb3 --- /dev/null +++ b/.gitignore @@ -0,0 +1,2 @@ +__pycache__ +*.pyc diff --git a/README.rst b/README.rst new file mode 100644 index 0000000..71df68b --- /dev/null +++ b/README.rst @@ -0,0 +1,36 @@ +translation-canary +------------- + +Translations can crash your program. Creating software for a wide audience +means sending your strings away for translation, and giving up control of your +strings means that strings with extralinguistic content can come back broken. +No one is likely to even realize it until someone fires up your program in +Hungarian and it crashes because Gtk bombed out on some busted markup, and the +Hungarian speaker is sad, and you are sad, and everything is just the absolute +worst. + +This is the canary in the translation coalmine. + +There are two parts to this project: + +translatable: + This contains checks on the strings to be submitted for translation. This + ensures that the content of the original strings marked for translation are + suitable for translation. These tests are run on the POT file before + uploading the POT or the updated PO files to the translators. + +translated: + This contains checks on the strings returned from the translators. This + ensures that the content of the translated strings won't break anything. + These tests are run on the source distribution before creating a release. + +Both translatable and translated are run by running the module +(e.g., `python3 -m translatable`) with the input file(s) as the argument. + +In addition to the python modules, the translatable directory contains +xgettext_werror.sh, a wrapper for xgettext that treats warnings as errors. +xgettext will print warnings as it extracts translatable strings from source +files, and these warnings should be addressed instead of silently ignored as +they scroll by in the build output. To use the script in a package that uses +the gettext template files from autopoint or gettextize, set +XGETTEXT=/path/to/xgettext_werror.sh in Makevars. diff --git a/translatable/__init__.py b/translatable/__init__.py new file mode 100644 index 0000000..9531aa2 --- /dev/null +++ b/translatable/__init__.py @@ -0,0 +1,85 @@ +# Framework for testing translatable strings +# +# Copyright (C) 2015 Red Hat, Inc. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions of +# the GNU General Public License v.2, or (at your option) any later version. +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY expressed or implied, including the implied warranties of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. You should have received a copy of the +# GNU General Public License along with this program; if not, write to the +# Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301, USA. Any Red Hat trademarks that are incorporated in the +# source code or documentation are not subject to the GNU General Public +# License and may only be used or replicated with the express permission of +# Red Hat, Inc. +# +# Red Hat Author(s): David Shea <dshea(a)redhat.com> + +""" +Framework for running tests against translatable strings. + +Tests are loaded from modules in this directory. A test is any callable object +within the module with a name that starts with 'test_'. + +Each test is called with a POEntry object as an argument. A test passes if it +returns without raising an exception. +""" + +try: + import polib +except ImportError: + print("You need to install the python-polib package to read translations") + raise + +# Gather tests from this directory +import pkgutil +_tests = [] +for finder, mod_name, _ispkg in pkgutil.iter_modules(__path__): + # Skip __main__ + if mod_name == "__main__": + continue + + # Load the module + module = finder.find_module(mod_name).load_module() + + # Look for attributes that start with 'test_' and add them to the test list + for attrname, attr in module.__dict__.items(): + if attrname.startswith('test_') and callable(attr): + _tests.append(attr) + +def testString(poentry): + """Run all tests against the given translatable string. + + :param polib.POEntry poentry: The PO file entry to test + :returns: whether the tests succeeded or not + :rtype: bool + """ + success = True + for test in _tests: + try: + test(poentry) + except Exception as e: + success = False + print("%s failed on %s: %s" % (test.__name__, poentry.msgid, str(e))) + + return success + +def testPOT(potfile): + """Run all tests against all entries in a POT file. + + :param str potfile: The name of a .pot file to test + :return: whether the checks succeeded or not + :rtype: bool + """ + success = True + + parsed_pot = polib.pofile(potfile) + + for entry in parsed_pot: + if not testString(entry): + success = False + + return success diff --git a/translatable/__main__.py b/translatable/__main__.py new file mode 100644 index 0000000..905cc70 --- /dev/null +++ b/translatable/__main__.py @@ -0,0 +1,33 @@ +# Entry point for testing translatable strings +# +# Copyright (C) 2015 Red Hat, Inc. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions of +# the GNU General Public License v.2, or (at your option) any later version. +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY expressed or implied, including the implied warranties of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. You should have received a copy of the +# GNU General Public License along with this program; if not, write to the +# Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301, USA. Any Red Hat trademarks that are incorporated in the +# source code or documentation are not subject to the GNU General Public +# License and may only be used or replicated with the express permission of +# Red Hat, Inc. +# +# Red Hat Author(s): David Shea <dshea(a)redhat.com> + +import sys +from . import testPOT + +if len(sys.argv) < 2: + print("Usage: translatable <POTfile>") + sys.exit(1) + +status = 0 +for potfile in sys.argv[1:]: + if not testPOT(potfile): + status = 1 + +sys.exit(status) diff --git a/translatable/test_comment.py b/translatable/test_comment.py new file mode 100644 index 0000000..51410fb --- /dev/null +++ b/translatable/test_comment.py @@ -0,0 +1,26 @@ +# Check that a string that needs a comment has one +# # Copyright (C) 2015 Red Hat, Inc. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions of +# the GNU General Public License v.2, or (at your option) any later version. +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY expressed or implied, including the implied warranties of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. You should have received a copy of the +# GNU General Public License along with this program; if not, write to the +# Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301, USA. Any Red Hat trademarks that are incorporated in the +# source code or documentation are not subject to the GNU General Public +# License and may only be used or replicated with the express permission of +# Red Hat, Inc. +# +# Red Hat Author(s): David Shea <dshea(a)redhat.com> + +def test_comment(poentry): + # Single-character translatable strings (e.g., the 'c' of press c to + # continue) need some additional context in order to make sense. Make + # sure that they have it. + + if len(poentry.msgid) == 1 and not poentry.comment: + raise AssertionError("Single-character string missing a comment.") diff --git a/translatable/test_markup.py b/translatable/test_markup.py new file mode 100644 index 0000000..1faa6d6 --- /dev/null +++ b/translatable/test_markup.py @@ -0,0 +1,38 @@ +# Check that a string does not contain unnecessary Pango markup +# +# Copyright (C) 2015 Red Hat, Inc. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions of +# the GNU General Public License v.2, or (at your option) any later version. +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY expressed or implied, including the implied warranties of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. You should have received a copy of the +# GNU General Public License along with this program; if not, write to the +# Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301, USA. Any Red Hat trademarks that are incorporated in the +# source code or documentation are not subject to the GNU General Public +# License and may only be used or replicated with the express permission of +# Red Hat, Inc. +# +# Red Hat Author(s): David Shea <dshea(a)redhat.com> + +from pocketlint.pangocheck import is_markup, markup_necessary +import xml.etree.ElementTree as ET + +import gi +gi.require_version("GLib", "2.0") +from gi.repository import GLib + +def test_markup(poentry): + # Unnecessary markup is markup applied to an entire string, such as + # _("Bold Text"). This could be instead be translated as + # "%s" % _("Bold Text"), and then the translator doesn't have to see + # the markup at all. + + if is_markup(poentry.msgid): + # Wrap the string in <markup> nodes, parse it, test it + tree = ET.fromstring("<markup>%s</markup>" % GLib.markup_escape_text(poentry.msgid)) + if not markup_necessary(tree): + raise AssertionError("Unnecessary markup") diff --git a/translatable/xgettext_werror.sh b/translatable/xgettext_werror.sh new file mode 100755 index 0000000..36eefec --- /dev/null +++ b/translatable/xgettext_werror.sh @@ -0,0 +1,46 @@ +#!/bin/sh -e +# +# xgettext_werror.sh: Run xgettext and actually do something with the warnings +# +# xgettext prints out warnings for certain problems in translatable strings, +# such as format strings that cannot be translated due to position-based +# parameters. These warnings generally indicate something that needs to be +# addressed before the strings can be submitted for translation. This script +# exits with a status of 1 so that the warnings are not ignored as they scroll +# by in pages of build output. +# +# This script should be used in place of xgettext when rebuilding the .pot file, +# e.g. by setting XGETTEXT in po/Makevars. +# +# Copyright (C) 2015 Red Hat, Inc. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions of +# the GNU General Public License v.2, or (at your option) any later version. +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY expressed or implied, including the implied warranties of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. You should have received a copy of the +# GNU General Public License along with this program; if not, write to the +# Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301, USA. Any Red Hat trademarks that are incorporated in the +# source code or documentation are not subject to the GNU General Public +# License and may only be used or replicated with the express permission of +# Red Hat, Inc. +# +# Red Hat Author(s): David Shea <dshea(a)redhat.com> + +returncode=0 + +# Collect the output from xgettext. If xgettext fails, treat that as a failure +# Make sure that "warning:" doesn't get translated +xgettext_output="$(LC_MESSAGES=C xgettext "$@" 2>&1)" || returncode=$? + +# Look for warnings +if echo "$xgettext_output" | fgrep -q "warning: "; then + returncode=1 +fi + +# Print the output and return +echo "$xgettext_output" +exit $returncode diff --git a/translated/__init__.py b/translated/__init__.py new file mode 100644 index 0000000..af0af73 --- /dev/null +++ b/translated/__init__.py @@ -0,0 +1,99 @@ +# Framework for testing translations +# +# Copyright (C) 2015 Red Hat, Inc. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions of +# the GNU General Public License v.2, or (at your option) any later version. +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY expressed or implied, including the implied warranties of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. You should have received a copy of the +# GNU General Public License along with this program; if not, write to the +# Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301, USA. Any Red Hat trademarks that are incorporated in the +# source code or documentation are not subject to the GNU General Public +# License and may only be used or replicated with the express permission of +# Red Hat, Inc. +# +# Red Hat Author(s): David Shea <dshea(a)redhat.com> + +""" +Framework for running tests against translations. + +Tests are loaded from modules in this directory. A test is any callable object +within the module with a name that starts with 'test_'. + +Each test is called with the name of .mo file to test as an argument. A test +passes if it returns without raising an exception. +""" + +import sys, tempfile, shutil, os, warnings + +_tests = [] + +# Gather tests from this directory +import pkgutil +for finder, mod_name, _ispkg in pkgutil.iter_modules(__path__): + # Skip __main__ + if mod_name == "__main__": + continue + + # Load the module + module = finder.find_module(mod_name).load_module() + + # Look for attributes that start with 'test_' and add them to the test list + for attrname, attr in module.__dict__.items(): + if attrname.startswith('test_') and callable(attr): + _tests.append(attr) + +def testFile(mofile, prefix=None): + """Run all registered tests against the given .mo file. + + :param str mofile: The .mo file name to check + :param str prefix: An optional directory prefix to strip from error messages + :return: whether the checks succeeded or not + :rtype: bool + """ + success = True + for test in _tests: + # Don't print the tmpdir path in error messages + if prefix is not None and mofile.startswith(prefix): + moerror = mofile[len(prefix):] + else: + moerror = mofile + + try: + with warnings.catch_warnings(record=True) as w: + test(mofile) + + # Print any warnings collected + for warn in w: + print("%s warned on %s: %s" % (test.__name__, moerror, warn.message)) + except Exception as e: + success = False + print("%s failed on %s: %s" % (test.__name__, moerror, str(e))) + + return success + +def testArchive(archive): + """Runs all registered tests against all .mo files in the given archive. + + :param str archive: The path to an archive containing .mo files + :return: whether the checks succeeded or not + :rtype: bool + """ + success = True + + archive_dir = tempfile.mkdtemp(prefix='translation-tests.') + try: + shutil.unpack_archive(archive, archive_dir) + for dir, _dirnames, paths in os.walk(archive_dir): + for mofile in (os.path.join(dir, path) for path in paths + if path.endswith('.mo') or path.endswith('.gmo')): + if not testFile(mofile, prefix=archive_dir + "/"): + success = False + finally: + shutil.rmtree(archive_dir, ignore_errors=True) + + return success diff --git a/translated/__main__.py b/translated/__main__.py new file mode 100644 index 0000000..fc1ad13 --- /dev/null +++ b/translated/__main__.py @@ -0,0 +1,33 @@ +# Entry point for testing translations +# +# Copyright (C) 2015 Red Hat, Inc. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions of +# the GNU General Public License v.2, or (at your option) any later version. +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY expressed or implied, including the implied warranties of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. You should have received a copy of the +# GNU General Public License along with this program; if not, write to the +# Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301, USA. Any Red Hat trademarks that are incorporated in the +# source code or documentation are not subject to the GNU General Public +# License and may only be used or replicated with the express permission of +# Red Hat, Inc. +# +# Red Hat Author(s): David Shea <dshea(a)redhat.com> + +import sys +from . import testArchive + +if len(sys.argv) < 2: + print("Usage: translation_tests <archive> [<archive> ...]") + sys.exit(1) + +status = 0 +for archive in sys.argv[1:]: + if not testArchive(archive): + status = 1 + +sys.exit(status) diff --git a/translated/test_markup.py b/translated/test_markup.py new file mode 100644 index 0000000..a3ca417 --- /dev/null +++ b/translated/test_markup.py @@ -0,0 +1,44 @@ +# Check translations of pango markup +# +# This will look for translatable strings that appear to contain markup and +# check that the markup in the translation matches. +# +# Copyright (C) 2015 Red Hat, Inc. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions of +# the GNU General Public License v.2, or (at your option) any later version. +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY expressed or implied, including the implied warranties of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. You should have received a copy of the +# GNU General Public License along with this program; if not, write to the +# Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301, USA. Any Red Hat trademarks that are incorporated in the +# source code or documentation are not subject to the GNU General Public +# License and may only be used or replicated with the express permission of +# Red Hat, Inc. +# +# Red Hat Author(s): David Shea <dshea(a)redhat.com> + +try: + import polib +except ImportError: + print("You need to install the python-polib package to read translations") + raise + +from pocketlint.pangocheck import is_markup, markup_match + +def test_markup(mofile): + mo = polib.mofile(mofile) + + for entry in mo.translated_entries(): + if is_markup(entry.msgid): + # If this is a plural, check each of the plural translations + if entry.msgid_plural: + for plural_id, msgstr in entry.msgstr_plural.items(): + if not markup_match(entry.msgid, msgstr): + raise AssertionError("Markup does not match for %d translation of msgid %s" % + (plural_id, entry.msgid)) + elif not markup_match(entry.msgid, entry.msgstr): + raise AssertionError("Markup does not match for msgid %s" % entry.msgid) diff --git a/translated/test_percentage.py b/translated/test_percentage.py new file mode 100644 index 0000000..914fd09 --- /dev/null +++ b/translated/test_percentage.py @@ -0,0 +1,42 @@ +# Check what percentage of strings a .mo translates +# +# This will reject translations that fall below a certain threshold of +# translated strings. +# +# Copyright (C) 2015 Red Hat, Inc. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions of +# the GNU General Public License v.2, or (at your option) any later version. +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY expressed or implied, including the implied warranties of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. You should have received a copy of the +# GNU General Public License along with this program; if not, write to the +# Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301, USA. Any Red Hat trademarks that are incorporated in the +# source code or documentation are not subject to the GNU General Public +# License and may only be used or replicated with the express permission of +# Red Hat, Inc. +# +# Red Hat Author(s): David Shea <dshea(a)redhat.com> + +import os +import warnings + +try: + import polib +except ImportError: + print("You need to install the python-polib package to read translations") + raise + +threshold = 10 + +def test_percentage(mofile): + # Open the .po file instead, which should be in the same location as the + # .mo file in the source archive + pofile = polib.pofile(os.path.splitext(mofile)[0] + '.po') + if pofile.percent_translated() < threshold: + # Issue a warning instead of an exception, since these should probably + # be handled on a case-by-case basis + warnings.warn("amount translated of %d%% below threshold of %d%%" % (pofile.percent_translated(), threshold)) diff --git a/translated/test_usability.py b/translated/test_usability.py new file mode 100644 index 0000000..bc51bb9 --- /dev/null +++ b/translated/test_usability.py @@ -0,0 +1,28 @@ +# Check a .mo file for basic usability +# +# This will test that the file is well-formed and that the Plural-Forms value +# is parseable +# +# Copyright (C) 2015 Red Hat, Inc. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions of +# the GNU General Public License v.2, or (at your option) any later version. +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY expressed or implied, including the implied warranties of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. You should have received a copy of the +# GNU General Public License along with this program; if not, write to the +# Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301, USA. Any Red Hat trademarks that are incorporated in the +# source code or documentation are not subject to the GNU General Public +# License and may only be used or replicated with the express permission of +# Red Hat, Inc. +# +# Red Hat Author(s): David Shea <dshea(a)redhat.com> + +import gettext + +def test_usability(mofile): + with open(mofile, "rb") as fp: + _t = gettext.GNUTranslations(fp=fp) From 1c36849eebd6cc5d4d90ed988e4ff3664186a0bf Mon Sep 17 00:00:00 2001 From: David Shea <dshea(a)redhat.com> Date: Wed, 25 Nov 2015 11:19:26 -0500 Subject: [PATCH 2/3] Add pylint Also add a Makefile to make it easy to run pylint and maybe some other tests. --- .gitignore | 1 + Makefile | 7 +++++++ tests/pylint/runpylint.py | 17 +++++++++++++++++ translatable/__init__.py | 2 +- translatable/test_markup.py | 8 +++----- translated/__init__.py | 6 +++--- 6 files changed, 32 insertions(+), 9 deletions(-) create mode 100644 Makefile create mode 100755 tests/pylint/runpylint.py diff --git a/.gitignore b/.gitignore index 8d35cb3..acf2e36 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ __pycache__ *.pyc +tests/pylint/.pylint.d diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..ff31c18 --- /dev/null +++ b/Makefile @@ -0,0 +1,7 @@ +# This Makefile is just for running the tests + +all: + @echo "nothing to build" + +check: + tests/pylint/runpylint.py diff --git a/tests/pylint/runpylint.py b/tests/pylint/runpylint.py new file mode 100755 index 0000000..5807bfd --- /dev/null +++ b/tests/pylint/runpylint.py @@ -0,0 +1,17 @@ +#!/usr/bin/python3 + +import sys +from pocketlint import PocketLintConfig, PocketLinter + +class TranslationCanaryLintConfig(PocketLintConfig): + @property + def disabledOptions(self): + return [ "W9930", # Found interruptible system call %s + "I0011", # Locally disabling %s + ] + +if __name__ == "__main__": + conf = TranslationCanaryLintConfig() + linter = PocketLinter(conf) + rc = linter.run() + sys.exit(rc) diff --git a/translatable/__init__.py b/translatable/__init__.py index 9531aa2..3277ab3 100644 --- a/translatable/__init__.py +++ b/translatable/__init__.py @@ -61,7 +61,7 @@ def testString(poentry): for test in _tests: try: test(poentry) - except Exception as e: + except Exception as e: # pylint: disable=broad-except success = False print("%s failed on %s: %s" % (test.__name__, poentry.msgid, str(e))) diff --git a/translatable/test_markup.py b/translatable/test_markup.py index 1faa6d6..fed02ed 100644 --- a/translatable/test_markup.py +++ b/translatable/test_markup.py @@ -21,10 +21,6 @@ from pocketlint.pangocheck import is_markup, markup_necessary import xml.etree.ElementTree as ET -import gi -gi.require_version("GLib", "2.0") -from gi.repository import GLib - def test_markup(poentry): # Unnecessary markup is markup applied to an entire string, such as # _("Bold Text"). This could be instead be translated as @@ -33,6 +29,8 @@ def test_markup(poentry): if is_markup(poentry.msgid): # Wrap the string in <markup> nodes, parse it, test it - tree = ET.fromstring("<markup>%s</markup>" % GLib.markup_escape_text(poentry.msgid)) + # The markup is unescaped on purpose + # pylint: disable=unescaped-markup + tree = ET.fromstring("<markup>%s</markup>" % poentry.msgid) if not markup_necessary(tree): raise AssertionError("Unnecessary markup") diff --git a/translated/__init__.py b/translated/__init__.py index af0af73..3f167ee 100644 --- a/translated/__init__.py +++ b/translated/__init__.py @@ -70,7 +70,7 @@ def testFile(mofile, prefix=None): # Print any warnings collected for warn in w: print("%s warned on %s: %s" % (test.__name__, moerror, warn.message)) - except Exception as e: + except Exception as e: # pylint: disable=broad-except success = False print("%s failed on %s: %s" % (test.__name__, moerror, str(e))) @@ -88,8 +88,8 @@ def testArchive(archive): archive_dir = tempfile.mkdtemp(prefix='translation-tests.') try: shutil.unpack_archive(archive, archive_dir) - for dir, _dirnames, paths in os.walk(archive_dir): - for mofile in (os.path.join(dir, path) for path in paths + for dirpath, _dirnames, paths in os.walk(archive_dir): + for mofile in (os.path.join(dirpath, path) for path in paths if path.endswith('.mo') or path.endswith('.gmo')): if not testFile(mofile, prefix=archive_dir + "/"): success = False From f75afb87436d4c8e362db088c951be4d68da3b59 Mon Sep 17 00:00:00 2001 From: David Shea <dshea(a)redhat.com> Date: Mon, 30 Nov 2015 15:03:19 -0500 Subject: [PATCH 3/3] Add some unittests for the tests. Make sure the test functions actually blow up on the errors they're supposed to be looking for. --- Makefile | 3 +- tests/unittests/test_translatable.py | 45 ++++++++++++++ tests/unittests/test_translated.py | 116 +++++++++++++++++++++++++++++++++++ 3 files changed, 163 insertions(+), 1 deletion(-) create mode 100644 tests/unittests/test_translatable.py create mode 100644 tests/unittests/test_translated.py diff --git a/Makefile b/Makefile index ff31c18..d9b633e 100644 --- a/Makefile +++ b/Makefile @@ -4,4 +4,5 @@ all: @echo "nothing to build" check: - tests/pylint/runpylint.py + PYTHONPATH=. tests/pylint/runpylint.py + python3 -m unittest discover tests/unittests diff --git a/tests/unittests/test_translatable.py b/tests/unittests/test_translatable.py new file mode 100644 index 0000000..ac53e3d --- /dev/null +++ b/tests/unittests/test_translatable.py @@ -0,0 +1,45 @@ +# Copyright (C) 2015 Red Hat, Inc. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions of +# the GNU General Public License v.2, or (at your option) any later version. +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY expressed or implied, including the implied warranties of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. You should have received a copy of the +# GNU General Public License along with this program; if not, write to the +# Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301, USA. Any Red Hat trademarks that are incorporated in the +# source code or documentation are not subject to the GNU General Public +# License and may only be used or replicated with the express permission of +# Red Hat, Inc. +# +# Red Hat Author(s): David Shea <dshea(a)redhat.com> + +import unittest +from polib import POEntry + +from translatable.test_markup import test_markup +from translatable.test_comment import test_comment + +class TestMarkup(unittest.TestCase): + def test_ok(self): + # no markup + test_markup(POEntry(msgid="test string")) + + # internal markup + test_markup(POEntry(msgid="test string")) + + def test_unnecessary_markup(self): + self.assertRaises(AssertionError, test_markup, POEntry(msgid="test string")) + +class TestComment(unittest.TestCase): + def test_ok(self): + # Perfectly fine string + test_comment(POEntry(msgid="Hello, I am a test string")) + + # single-character string with a comment + test_comment(POEntry(msgid="c", comment="TRANSLATORS: 'c' to continue")) + + def test_no_comment(self): + self.assertRaises(AssertionError, test_comment, POEntry(msgid="c")) diff --git a/tests/unittests/test_translated.py b/tests/unittests/test_translated.py new file mode 100644 index 0000000..7011cb7 --- /dev/null +++ b/tests/unittests/test_translated.py @@ -0,0 +1,116 @@ +# Copyright (C) 2015 Red Hat, Inc. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions of +# the GNU General Public License v.2, or (at your option) any later version. +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY expressed or implied, including the implied warranties of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. You should have received a copy of the +# GNU General Public License along with this program; if not, write to the +# Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301, USA. Any Red Hat trademarks that are incorporated in the +# source code or documentation are not subject to the GNU General Public # License and may only be used or replicated with the express permission of # Red Hat, Inc. +# +# Red Hat Author(s): David Shea <dshea(a)redhat.com> + +import unittest +import tempfile +import warnings +import shutil +import os +import polib + +from translated.test_markup import test_markup +from translated.test_percentage import test_percentage +from translated.test_usability import test_usability + +# convert a polib.MOFile into a NamedTemporaryFile +def mofile(moobj): + f = tempfile.NamedTemporaryFile(suffix='.mo') + moobj.save(f.name) + return f + +# convenience function for creating a single-entry mofile +def mofile_from_entry(*args, **kwargs): + moobj = polib.MOFile() + moobj.append(polib.MOEntry(*args, **kwargs)) + return mofile(moobj) + +class TestMarkup(unittest.TestCase): + def test_ok(self): + # no markup + with mofile_from_entry(msgid="test string", msgstr="estay ingstray") as m: + test_markup(m.name) + + # matching markup + with mofile_from_entry(msgid="bold string", msgstr="oldbay ingstray") as m: + test_markup(m.name) + + # matching plural + with mofile_from_entry(msgid="%d bold string", msgid_plural="%d bold strings", + msgstr_plural={0: "%d oldbay ingstray", 1: "%d oldbay instrays"}) as m: + test_markup(m.name) + + def test_missing(self): + with mofile_from_entry(msgid="bold string", msgstr="oldbay ingstray") as m: + self.assertRaises(AssertionError, test_markup, m.name) + + def test_mismatch(self): + with mofile_from_entry(msgid="bold string", msgstr="oldbay ingstray") as m: + self.assertRaises(AssertionError, test_markup, m.name) + + def test_typo(self): + with mofile_from_entry(msgid="bold string", msgstr="<boldbay ingstray") as m: + self.assertRaises(AssertionError, test_markup, m.name) + + def test_mismatch_plural(self): + with mofile_from_entry(msgid="%d bold string", msgid_plural="%d bold strings", + msgstr_plural={0: "%d olbday ingstray", 1: "%d oldbay ingstrays"}) as m: + self.assertRaises(AssertionError, test_markup, m.name) + +class TestPercentage(unittest.TestCase): + # test_percentage actually looks at .po files, so the tests need to create + # both a .po and a .mo in self.tmpdir + + def setUp(self): + self.tmpdir = tempfile.mkdtemp() + self.popath = os.path.join(self.tmpdir, "test.po") + self.mopath = os.path.join(self.tmpdir, "test.mo") + warnings.simplefilter("error") + + def tearDown(self): + shutil.rmtree(self.tmpdir) + warnings.resetwarnings() + + def test_ok(self): + # 100% + pofile = polib.POFile() + pofile.append(polib.POEntry(msgid="test string", msgstr="estay ingstray")) + pofile.save(self.popath) + pofile.save_as_mofile(self.mopath) + test_percentage(self.mopath) + + def test_not_ok(self): + # 0% + pofile = polib.POFile() + pofile.append(polib.POEntry(msgid="test string", msgstr="")) + pofile.save(self.popath) + pofile.save_as_mofile(self.mopath) + +class TestUsability(unittest.TestCase): + def test_ok(self): + # what lt's Plural-Forms is supposed to look like + moobj = polib.MOFile() + moobj.metadata["Plural-Forms"] = "nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && (n%100<10 || n%100>=20) ? 1 : 2)\n" + + with mofile(moobj) as m: + test_usability(m.name) + + def test_busted_plural_forms(self): + # https://bugzilla.redhat.com/show_bug.cgi?id=1283599 + moobj = polib.MOFile() + moobj.metadata["Plural-Forms"] = "nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && (n%100<10 or n%100>=20) ? 1 : 2)\n" + + with mofile(moobj) as m: + self.assertRaises(Exception, test_usability, m.name) -- To view this commit on github, visit https://github.com/rhinstaller/anaconda/commit/73ae75a44a3bcf1475789c8f14e1…

1 0

[master 3/9] Add some unittests for the tests.
by dashea 01 Dec '15

01 Dec '15

From: David Shea <dshea(a)redhat.com> Make sure the test functions actually blow up on the errors they're supposed to be looking for. --- Makefile | 3 +- tests/unittests/test_translatable.py | 45 ++++++++++++++ tests/unittests/test_translated.py | 116 +++++++++++++++++++++++++++++++++++ 3 files changed, 163 insertions(+), 1 deletion(-) create mode 100644 tests/unittests/test_translatable.py create mode 100644 tests/unittests/test_translated.py diff --git a/Makefile b/Makefile index ff31c18..d9b633e 100644 --- a/Makefile +++ b/Makefile @@ -4,4 +4,5 @@ all: @echo "nothing to build" check: - tests/pylint/runpylint.py + PYTHONPATH=. tests/pylint/runpylint.py + python3 -m unittest discover tests/unittests diff --git a/tests/unittests/test_translatable.py b/tests/unittests/test_translatable.py new file mode 100644 index 0000000..ac53e3d --- /dev/null +++ b/tests/unittests/test_translatable.py @@ -0,0 +1,45 @@ +# Copyright (C) 2015 Red Hat, Inc. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions of +# the GNU General Public License v.2, or (at your option) any later version. +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY expressed or implied, including the implied warranties of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. You should have received a copy of the +# GNU General Public License along with this program; if not, write to the +# Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301, USA. Any Red Hat trademarks that are incorporated in the +# source code or documentation are not subject to the GNU General Public +# License and may only be used or replicated with the express permission of +# Red Hat, Inc. +# +# Red Hat Author(s): David Shea <dshea(a)redhat.com> + +import unittest +from polib import POEntry + +from translatable.test_markup import test_markup +from translatable.test_comment import test_comment + +class TestMarkup(unittest.TestCase): + def test_ok(self): + # no markup + test_markup(POEntry(msgid="test string")) + + # internal markup + test_markup(POEntry(msgid="test string")) + + def test_unnecessary_markup(self): + self.assertRaises(AssertionError, test_markup, POEntry(msgid="test string")) + +class TestComment(unittest.TestCase): + def test_ok(self): + # Perfectly fine string + test_comment(POEntry(msgid="Hello, I am a test string")) + + # single-character string with a comment + test_comment(POEntry(msgid="c", comment="TRANSLATORS: 'c' to continue")) + + def test_no_comment(self): + self.assertRaises(AssertionError, test_comment, POEntry(msgid="c")) diff --git a/tests/unittests/test_translated.py b/tests/unittests/test_translated.py new file mode 100644 index 0000000..7011cb7 --- /dev/null +++ b/tests/unittests/test_translated.py @@ -0,0 +1,116 @@ +# Copyright (C) 2015 Red Hat, Inc. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions of +# the GNU General Public License v.2, or (at your option) any later version. +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY expressed or implied, including the implied warranties of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. You should have received a copy of the +# GNU General Public License along with this program; if not, write to the +# Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301, USA. Any Red Hat trademarks that are incorporated in the +# source code or documentation are not subject to the GNU General Public # License and may only be used or replicated with the express permission of # Red Hat, Inc. +# +# Red Hat Author(s): David Shea <dshea(a)redhat.com> + +import unittest +import tempfile +import warnings +import shutil +import os +import polib + +from translated.test_markup import test_markup +from translated.test_percentage import test_percentage +from translated.test_usability import test_usability + +# convert a polib.MOFile into a NamedTemporaryFile +def mofile(moobj): + f = tempfile.NamedTemporaryFile(suffix='.mo') + moobj.save(f.name) + return f + +# convenience function for creating a single-entry mofile +def mofile_from_entry(*args, **kwargs): + moobj = polib.MOFile() + moobj.append(polib.MOEntry(*args, **kwargs)) + return mofile(moobj) + +class TestMarkup(unittest.TestCase): + def test_ok(self): + # no markup + with mofile_from_entry(msgid="test string", msgstr="estay ingstray") as m: + test_markup(m.name) + + # matching markup + with mofile_from_entry(msgid="bold string", msgstr="oldbay ingstray") as m: + test_markup(m.name) + + # matching plural + with mofile_from_entry(msgid="%d bold string", msgid_plural="%d bold strings", + msgstr_plural={0: "%d oldbay ingstray", 1: "%d oldbay instrays"}) as m: + test_markup(m.name) + + def test_missing(self): + with mofile_from_entry(msgid="bold string", msgstr="oldbay ingstray") as m: + self.assertRaises(AssertionError, test_markup, m.name) + + def test_mismatch(self): + with mofile_from_entry(msgid="bold string", msgstr="oldbay ingstray") as m: + self.assertRaises(AssertionError, test_markup, m.name) + + def test_typo(self): + with mofile_from_entry(msgid="bold string", msgstr="<boldbay ingstray") as m: + self.assertRaises(AssertionError, test_markup, m.name) + + def test_mismatch_plural(self): + with mofile_from_entry(msgid="%d bold string", msgid_plural="%d bold strings", + msgstr_plural={0: "%d olbday ingstray", 1: "%d oldbay ingstrays"}) as m: + self.assertRaises(AssertionError, test_markup, m.name) + +class TestPercentage(unittest.TestCase): + # test_percentage actually looks at .po files, so the tests need to create + # both a .po and a .mo in self.tmpdir + + def setUp(self): + self.tmpdir = tempfile.mkdtemp() + self.popath = os.path.join(self.tmpdir, "test.po") + self.mopath = os.path.join(self.tmpdir, "test.mo") + warnings.simplefilter("error") + + def tearDown(self): + shutil.rmtree(self.tmpdir) + warnings.resetwarnings() + + def test_ok(self): + # 100% + pofile = polib.POFile() + pofile.append(polib.POEntry(msgid="test string", msgstr="estay ingstray")) + pofile.save(self.popath) + pofile.save_as_mofile(self.mopath) + test_percentage(self.mopath) + + def test_not_ok(self): + # 0% + pofile = polib.POFile() + pofile.append(polib.POEntry(msgid="test string", msgstr="")) + pofile.save(self.popath) + pofile.save_as_mofile(self.mopath) + +class TestUsability(unittest.TestCase): + def test_ok(self): + # what lt's Plural-Forms is supposed to look like + moobj = polib.MOFile() + moobj.metadata["Plural-Forms"] = "nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && (n%100<10 || n%100>=20) ? 1 : 2)\n" + + with mofile(moobj) as m: + test_usability(m.name) + + def test_busted_plural_forms(self): + # https://bugzilla.redhat.com/show_bug.cgi?id=1283599 + moobj = polib.MOFile() + moobj.metadata["Plural-Forms"] = "nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && (n%100<10 or n%100>=20) ? 1 : 2)\n" + + with mofile(moobj) as m: + self.assertRaises(Exception, test_usability, m.name) -- To view this commit on github, visit https://github.com/rhinstaller/anaconda/commit/f75afb87436d4c8e362db088c951…

1 0

[master 2/9] Add pylint
by dashea 01 Dec '15

01 Dec '15

From: David Shea <dshea(a)redhat.com> Also add a Makefile to make it easy to run pylint and maybe some other tests. --- .gitignore | 1 + Makefile | 7 +++++++ tests/pylint/runpylint.py | 17 +++++++++++++++++ translatable/__init__.py | 2 +- translatable/test_markup.py | 8 +++----- translated/__init__.py | 6 +++--- 6 files changed, 32 insertions(+), 9 deletions(-) create mode 100644 Makefile create mode 100755 tests/pylint/runpylint.py diff --git a/.gitignore b/.gitignore index 8d35cb3..acf2e36 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ __pycache__ *.pyc +tests/pylint/.pylint.d diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..ff31c18 --- /dev/null +++ b/Makefile @@ -0,0 +1,7 @@ +# This Makefile is just for running the tests + +all: + @echo "nothing to build" + +check: + tests/pylint/runpylint.py diff --git a/tests/pylint/runpylint.py b/tests/pylint/runpylint.py new file mode 100755 index 0000000..5807bfd --- /dev/null +++ b/tests/pylint/runpylint.py @@ -0,0 +1,17 @@ +#!/usr/bin/python3 + +import sys +from pocketlint import PocketLintConfig, PocketLinter + +class TranslationCanaryLintConfig(PocketLintConfig): + @property + def disabledOptions(self): + return [ "W9930", # Found interruptible system call %s + "I0011", # Locally disabling %s + ] + +if __name__ == "__main__": + conf = TranslationCanaryLintConfig() + linter = PocketLinter(conf) + rc = linter.run() + sys.exit(rc) diff --git a/translatable/__init__.py b/translatable/__init__.py index 9531aa2..3277ab3 100644 --- a/translatable/__init__.py +++ b/translatable/__init__.py @@ -61,7 +61,7 @@ def testString(poentry): for test in _tests: try: test(poentry) - except Exception as e: + except Exception as e: # pylint: disable=broad-except success = False print("%s failed on %s: %s" % (test.__name__, poentry.msgid, str(e))) diff --git a/translatable/test_markup.py b/translatable/test_markup.py index 1faa6d6..fed02ed 100644 --- a/translatable/test_markup.py +++ b/translatable/test_markup.py @@ -21,10 +21,6 @@ from pocketlint.pangocheck import is_markup, markup_necessary import xml.etree.ElementTree as ET -import gi -gi.require_version("GLib", "2.0") -from gi.repository import GLib - def test_markup(poentry): # Unnecessary markup is markup applied to an entire string, such as # _("Bold Text"). This could be instead be translated as @@ -33,6 +29,8 @@ def test_markup(poentry): if is_markup(poentry.msgid): # Wrap the string in <markup> nodes, parse it, test it - tree = ET.fromstring("<markup>%s</markup>" % GLib.markup_escape_text(poentry.msgid)) + # The markup is unescaped on purpose + # pylint: disable=unescaped-markup + tree = ET.fromstring("<markup>%s</markup>" % poentry.msgid) if not markup_necessary(tree): raise AssertionError("Unnecessary markup") diff --git a/translated/__init__.py b/translated/__init__.py index af0af73..3f167ee 100644 --- a/translated/__init__.py +++ b/translated/__init__.py @@ -70,7 +70,7 @@ def testFile(mofile, prefix=None): # Print any warnings collected for warn in w: print("%s warned on %s: %s" % (test.__name__, moerror, warn.message)) - except Exception as e: + except Exception as e: # pylint: disable=broad-except success = False print("%s failed on %s: %s" % (test.__name__, moerror, str(e))) @@ -88,8 +88,8 @@ def testArchive(archive): archive_dir = tempfile.mkdtemp(prefix='translation-tests.') try: shutil.unpack_archive(archive, archive_dir) - for dir, _dirnames, paths in os.walk(archive_dir): - for mofile in (os.path.join(dir, path) for path in paths + for dirpath, _dirnames, paths in os.walk(archive_dir): + for mofile in (os.path.join(dirpath, path) for path in paths if path.endswith('.mo') or path.endswith('.gmo')): if not testFile(mofile, prefix=archive_dir + "/"): success = False -- To view this commit on github, visit https://github.com/rhinstaller/anaconda/commit/1c36849eebd6cc5d4d90ed988e4f…

1 0

Updated: [rhinstaller/anaconda/pulls/463 master] Translation tests
by dashea 01 Dec '15

01 Dec '15

The goal of this is to create and use a common framework for testing translatable and translated strings. I went ahead and added the code for that at https://github.com/rhinstaller/translation-canary. This is applicable to a lot of our projects, and in fact the catalyst for this was a problem in blivet's translated strings, but I figured that I would start here. The "translatable" tests are run after updating anaconda.pot. Any problems detected in the translatable strings raise an error before the strings are pushed to Zanata. Part of this was previously a test in gettext_warnings.sh. The "translated" tests are run after create the dist tarball. Here I can make a stronger case for running the tests at a weird time instead of just as a regular make check kind of test: since Zanata is out of our control, the translated strings can change right up to the last minute, so to ensure that translated strings aren't busted in a way that will crash anaconda we need to run checks on the actual source distribution. "translated" also includes a warning about languages with a translation level below a certain threshold, currently 10% of the strings. This one is a warning instead of an error since, I don't know, maybe it's ok to have a mostly untranslated language or two, and also the lack of anonymous Zanata pulls make it impossible for this one to do anything useful in Jenkins. Possible discussion topics: - Is this all just a terrible idea or what? - Is using a submodule appropriate or at all a thing we want to do? - Is causing 'make release' or pot-update to fail too big a hammer? Keeping in mind that the errors caught by these tests crash anaconda. Failing at pot-update might be too big a hammer. - better ideas for paths and names and such - ??? The other projects we have that could use this: - blivet - pykickstart - blivet-gui - libbytesize - meh - initial-setup - system-config-kickstart The tests already found errors in translations which would need to be fixed before creating another release: - es: `Your current %(product)s software selection requires...` has an extra `<a>` tag - fr: `Your current <a href="" title="%(tooltip)s">%(product)s software selection</a> requires...` has an unclosed `<a>` tag - hr: `Your current %(product)s software selection requires...` has a stray `</a>` - zh_TW: `Your current %(product)s software selection requires...` has a stray `</a>` Languages proposed for the currently very conservative chopping block: - Turkish (tr): 9% translated - Farsi (fa): 4% translated - Latvian (lv): 4% translated - Asturian (ast): 3% translated - Bengali (bn): 3% translated - British English (en_GB): 3% translated - Sinhala (si): 3% translated - Albanian (sq): 2% translated - Greek (el): 2% translated - Maithili (mai): 2% translated - Afrikaans (af): 1% translated - Amharic (am): 1% translated - Belarusian (be): 1% translated - Bosnian (bs): 1% translated - Icelandic (is): 2% translated - Iloko (ilo): 1% translated - Macedonian (mk): 1% translated - Malay (ms): 1% translated - Nepali (ne): 1% translated - Northern Sotho (nso): 1% translated - Slovenian (sl): 1% translated - Swiss German (de_CH): 1% translated - Tajik (tg): 1% translated - Urdu (ur): 1% translated - Vietnamese (vi): 1% translated - Welsh (cy): 1% translated - Zulu (zu): 1% translated -- To view this pull request on github, visit https://github.com/rhinstaller/anaconda/pull/463

1 0

[master 1/9] Initial framework and tests
by dashea 01 Dec '15

01 Dec '15

1 0

Re: [rhinstaller/anaconda/pulls/457 master] Rewrite tests/gui/
by dashea 01 Dec '15

01 Dec '15

Closed. -- To view this pull request on github, visit https://github.com/rhinstaller/anaconda/pull/457

1 0

← Newer
1
...
41
42
43
44
45
46
47
...
50
Older →

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

anaconda-patches December 2015