New: [rhinstaller/anaconda/pulls/238 rhel7-branch] fix the URL check for URLs with IPv6 literals

[rhinstaller/anaconda/pulls/238 rhel7-branch] fix the URL check for URLs with IPv6 literals

[dashea]

Added label: master.

[rhinstaller/anaconda/pulls/238 rhel7-branch] fix the URL check for URLs with IPv6 literals

[dashea]

Added label: rhel7-branch.

[rhel7-branch 1/2] Add tests for IPv6 literals in URLs

[dashea]

From: David Shea

Related: rhbz#1246133
---
 tests/regex_tests/url_test.py | 227 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 227 insertions(+)

diff --git a/tests/regex_tests/url_test.py b/tests/regex_tests/url_test.py
index 3c77f7d..8113cca 100644
--- a/tests/regex_tests/url_test.py
+++ b/tests/regex_tests/url_test.py
@@ -141,6 +141,228 @@ def url_regex_test(self):
                   ( "http://[e-mail]:3128/blah/blah?query=whatever#fragment",
                       ("http://", 'user', None, 'proxy.host', '3128', '/blah/blah', "query=whatever", "fragment") ),

+                  # Same, but with IPv4 literals
+                  ( "1.2.3.4",
+                      (None, None, None, '1.2.3.4', None, None, None, None) ),
+
+                  ( "1.2.3.4:3128",
+                      (None, None, None, '1.2.3.4', '3128', None, None, None) ),
+
+                  ( "user:password@1.2.3.4",
+                      (None, 'user', 'password', '1.2.3.4', None, None, None, None) ),
+
+                  ( "user@1.2.3.4",
+                      (None, 'user', None, '1.2.3.4', None, None, None, None) ),
+
+                  ( "user:password@1.2.3.4:3128",
+                      (None, 'user', 'password', '1.2.3.4', '3128', None, None, None) ),
+
+                  ( "user@1.2.3.4:3128",
+                      (None, 'user', None, '1.2.3.4', '3128', None, None, None) ),
+
+                  ( "1.2.3.4/blah/blah",
+                      (None, None, None, '1.2.3.4', None, '/blah/blah', None, None) ),
+
+                  ( "1.2.3.4:3128/blah/blah",
+                      (None, None, None, '1.2.3.4', '3128', '/blah/blah', None, None) ),
+
+                  ( "user:password@1.2.3.4/blah/blah",
+                      (None, 'user', 'password', '1.2.3.4', None, '/blah/blah', None, None) ),
+
+                  ( "user@1.2.3.4/blah/blah",
+                      (None, 'user', None, '1.2.3.4', None, '/blah/blah', None, None) ),
+
+                  ( "user:password@1.2.3.4:3128/blah/blah",
+                      (None, 'user', 'password', '1.2.3.4', '3128', "/blah/blah", None, None) ),
+
+                  ( "user@1.2.3.4:3128/blah/blah",
+                      (None, 'user', None, '1.2.3.4', '3128', "/blah/blah", None, None) ),
+
+
+
+                  ( "http://1.2.3.4",
+                      ('http://', None, None, '1.2.3.4', None, None, None, None) ),
+
+                  ( "http://1.2.3.4:3128",
+                      ('http://', None, None, '1.2.3.4', '3128', None, None, None) ),
+
+                  ( "http://user:password@1.2.3.4",
+                      ('http://', 'user', 'password', '1.2.3.4', None, None, None, None) ),
+
+                  ( "http://user@1.2.3.4",
+                      ('http://', 'user', None, '1.2.3.4', None, None, None, None) ),
+
+                  ( "http://user:password@1.2.3.4:3128",
+                      ('http://', 'user', 'password', '1.2.3.4', '3128', None, None, None) ),
+
+                  ( "http://user@1.2.3.4:3128",
+                      ('http://', 'user', None, '1.2.3.4', '3128', None, None, None) ),
+
+                  ( "http://1.2.3.4/blah/blah",
+                      ('http://', None, None, '1.2.3.4', None, '/blah/blah', None, None) ),
+
+                  ( "http://1.2.3.4:3128/blah/blah",
+                      ('http://', None, None, '1.2.3.4', '3128', '/blah/blah', None, None) ),
+
+                  ( "http://user:password@1.2.3.4/blah/blah",
+                      ("http://", 'user', 'password', '1.2.3.4', None, '/blah/blah', None, None) ),
+
+                  ( "http://%75ser:password@1.2.3.4/blah/blah",
+                      ("http://", '%75ser', 'password', '1.2.3.4', None, '/blah/blah', None, None) ),
+
+                  ( "http://user:%70assword@1.2.3.4/blah/blah",
+                      ("http://", 'user', '%70assword', '1.2.3.4', None, '/blah/blah', None, None) ),
+
+                  ( "http://user@1.2.3.4/blah/blah",
+                      ("http://", 'user', None, '1.2.3.4', None, '/blah/blah', None, None) ),
+
+                  ( "http://user@1.2.3.4/blah/bla%68",
+                      ("http://", 'user', None, '1.2.3.4', None, '/blah/bla%68', None, None) ),
+
+                  ( "http://user:password@1.2.3.4:3128/blah/blah",
+                      ("http://", 'user', 'password', '1.2.3.4', '3128', '/blah/blah', None, None) ),
+
+                  ( "http://user@1.2.3.4:3128/blah/blah",
+                      ("http://", 'user', None, '1.2.3.4', '3128', '/blah/blah', None, None) ),
+
+                  ( "http://user@1.2.3.4:3128/blah/blah?query",
+                      ("http://", 'user', None, '1.2.3.4', '3128', '/blah/blah', "query", None) ),
+
+                  ( "http://user@1.2.3.4:3128/blah/blah?query?",
+                      ("http://", 'user', None, '1.2.3.4', '3128', '/blah/blah', "query?", None) ),
+
+                  ( "http://user@1.2.3.4:3128/blah/blah?query=whatever",
+                      ("http://", 'user', None, '1.2.3.4', '3128', '/blah/blah', "query=whatever", None) ),
+
+                  ( "http://user@1.2.3.4:3128/blah/blah?query=whate%76er",
+                      ("http://", 'user', None, '1.2.3.4', '3128', '/blah/blah', "query=whate%76er", None) ),
+
+                  ( "http://user@1.2.3.4:3128/blah/blah?",
+                      ("http://", 'user', None, '1.2.3.4', '3128', '/blah/blah', "", None) ),
+
+                  ( "http://user@1.2.3.4:3128/blah/blah#fragment",
+                      ("http://", 'user', None, '1.2.3.4', '3128', '/blah/blah', None, "fragment") ),
+
+                  ( "http://user@1.2.3.4:3128/blah/blah#",
+                      ("http://", 'user', None, '1.2.3.4', '3128', '/blah/blah', None, "") ),
+
+                  ( "http://user@1.2.3.4:3128/blah/blah#fragm%65nt",
+                      ("http://", 'user', None, '1.2.3.4', '3128', '/blah/blah', None, "fragm%65nt") ),
+
+                  ( "http://user@1.2.3.4:3128/blah/blah?query=whatever#fragment",
+                      ("http://", 'user', None, '1.2.3.4', '3128', '/blah/blah', "query=whatever", "fragment") ),
+
+                  # An again, but with IPv6 literals
+                  ( "[dead::beef]",
+                      (None, None, None, '[dead::beef]', None, None, None, None) ),
+
+                  ( "[dead::beef]:3128",
+                      (None, None, None, '[dead::beef]', '3128', None, None, None) ),
+
+                  ( "user:password@[dead::beef]",
+                      (None, 'user', 'password', '[dead::beef]', None, None, None, None) ),
+
+                  ( "user@[dead::beef]",
+                      (None, 'user', None, '[dead::beef]', None, None, None, None) ),
+
+                  ( "user:password@[dead::beef]:3128",
+                      (None, 'user', 'password', '[dead::beef]', '3128', None, None, None) ),
+
+                  ( "user@[dead::beef]:3128",
+                      (None, 'user', None, '[dead::beef]', '3128', None, None, None) ),
+
+                  ( "[dead::beef]/blah/blah",
+                      (None, None, None, '[dead::beef]', None, '/blah/blah', None, None) ),
+
+                  ( "[dead::beef]:3128/blah/blah",
+                      (None, None, None, '[dead::beef]', '3128', '/blah/blah', None, None) ),
+
+                  ( "user:password@[dead::beef]/blah/blah",
+                      (None, 'user', 'password', '[dead::beef]', None, '/blah/blah', None, None) ),
+
+                  ( "user@[dead::beef]/blah/blah",
+                      (None, 'user', None, '[dead::beef]', None, '/blah/blah', None, None) ),
+
+                  ( "user:password@[dead::beef]:3128/blah/blah",
+                      (None, 'user', 'password', '[dead::beef]', '3128', "/blah/blah", None, None) ),
+
+                  ( "user@[dead::beef]:3128/blah/blah",
+                      (None, 'user', None, '[dead::beef]', '3128', "/blah/blah", None, None) ),
+
+
+
+                  ( "http://%5Bdead::beef]",
+                      ('http://', None, None, '[dead::beef]', None, None, None, None) ),
+
+                  ( "http://%5Bdead::beef%5D:3128",
+                      ('http://', None, None, '[dead::beef]', '3128', None, None, None) ),
+
+                  ( "http://user:password@%5Bdead::beef]",
+                      ('http://', 'user', 'password', '[dead::beef]', None, None, None, None) ),
+
+                  ( "http://user@%5Bdead::beef]",
+                      ('http://', 'user', None, '[dead::beef]', None, None, None, None) ),
+
+                  ( "http://user:password@%5Bdead::beef%5D:3128",
+                      ('http://', 'user', 'password', '[dead::beef]', '3128', None, None, None) ),
+
+                  ( "http://user@%5Bdead::beef%5D:3128",
+                      ('http://', 'user', None, '[dead::beef]', '3128', None, None, None) ),
+
+                  ( "http://%5Bdead::beef%5D/blah/blah",
+                      ('http://', None, None, '[dead::beef]', None, '/blah/blah', None, None) ),
+
+                  ( "http://%5Bdead::beef%5D:3128/blah/blah",
+                      ('http://', None, None, '[dead::beef]', '3128', '/blah/blah', None, None) ),
+
+                  ( "http://user:password@%5Bdead::beef%5D/blah/blah",
+                      ("http://", 'user', 'password', '[dead::beef]', None, '/blah/blah', None, None) ),
+
+                  ( "http://%75ser:password@%5Bdead::beef%5D/blah/blah",
+                      ("http://", '%75ser', 'password', '[dead::beef]', None, '/blah/blah', None, None) ),
+
+                  ( "http://user:%70assword@%5Bdead::beef%5D/blah/blah",
+                      ("http://", 'user', '%70assword', '[dead::beef]', None, '/blah/blah', None, None) ),
+
+                  ( "http://user@%5Bdead::beef%5D/blah/blah",
+                      ("http://", 'user', None, '[dead::beef]', None, '/blah/blah', None, None) ),
+
+                  ( "http://user@%5Bdead::beef%5D/blah/bla%68",
+                      ("http://", 'user', None, '[dead::beef]', None, '/blah/bla%68', None, None) ),
+
+                  ( "http://user:password@%5Bdead::beef%5D:3128/blah/blah",
+                      ("http://", 'user', 'password', '[dead::beef]', '3128', '/blah/blah', None, None) ),
+
+                  ( "http://user@%5Bdead::beef%5D:3128/blah/blah",
+                      ("http://", 'user', None, '[dead::beef]', '3128', '/blah/blah', None, None) ),
+
+                  ( "http://user@%5Bdead::beef%5D:3128/blah/blah?query",
+                      ("http://", 'user', None, '[dead::beef]', '3128', '/blah/blah', "query", None) ),
+
+                  ( "http://user@%5Bdead::beef%5D:3128/blah/blah?query?",
+                      ("http://", 'user', None, '[dead::beef]', '3128', '/blah/blah', "query?", None) ),
+
+                  ( "http://user@%5Bdead::beef%5D:3128/blah/blah?query=whatever",
+                      ("http://", 'user', None, '[dead::beef]', '3128', '/blah/blah', "query=whatever", None) ),
+
+                  ( "http://user@%5Bdead::beef%5D:3128/blah/blah?query=whate%76er",
+                      ("http://", 'user', None, '[dead::beef]', '3128', '/blah/blah', "query=whate%76er", None) ),
+
+                  ( "http://user@%5Bdead::beef%5D:3128/blah/blah?",
+                      ("http://", 'user', None, '[dead::beef]', '3128', '/blah/blah', "", None) ),
+
+                  ( "http://user@%5Bdead::beef%5D:3128/blah/blah#fragment",
+                      ("http://", 'user', None, '[dead::beef]', '3128', '/blah/blah', None, "fragment") ),
+
+                  ( "http://user@%5Bdead::beef%5D:3128/blah/blah#",
+                      ("http://", 'user', None, '[dead::beef]', '3128', '/blah/blah', None, "") ),
+
+                  ( "http://user@%5Bdead::beef%5D:3128/blah/blah#fragm%65nt",
+                      ("http://", 'user', None, '[dead::beef]', '3128', '/blah/blah', None, "fragm%65nt") ),
+
+                  ( "http://user@%5Bdead::beef%5D:3128/blah/blah?query=whatever#fragment",
+                      ("http://", 'user', None, '[dead::beef]', '3128', '/blah/blah', "query=whatever", "fragment") ),
+
                   # Invalid schemes
                   ( "0http://proxy.host/", None),
                   ( "h~ttp://proxy.host/", None),
@@ -162,6 +384,11 @@ def url_regex_test(self):
                   # Invalid fragments
                   ( "http://proxy.host/blah/blah#fragment#", None),
                   ( "http://proxy.host/blah/blah#%xxragment", None),
+
+                  # Unbracketed IPv6
+                  ( "fe80::1234:56:78", None),
+                  ( "fe80::1234:56:78/blah/blah", None),
+                  ( "http://fe80::1234:56:78/blah/blah", None)
                 ]

[rhel7-branch 2/2] Add parentheses around the IPV6 regex fragment.

[dashea]

From: David Shea

This way when it's combined with something else (like the hostname
regex), all the pipes don't mess up the surrounding regex.

Resolves: rhbz#1246133
---
 pyanaconda/regexes.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/pyanaconda/regexes.py b/pyanaconda/regexes.py
index b4e52a3..2235d89 100644
--- a/pyanaconda/regexes.py
+++ b/pyanaconda/regexes.py
@@ -77,7 +77,8 @@
 # Adapted from the IPv6address ABNF definition in RFC 3986, so it has all those
 # IPv4 compatibility bits too. All groups are non-capturing to make it easy to
 # use in an expression with groups and completely impossible to read
-IPV6_PATTERN_WITHOUT_ANCHORS = r'(?:(?:(?:[0-9a-fA-F]{1,4}:){6})(?:(?:(?:[0-9a-fA-F]{1,4}):(?:[0-9a-fA-F]{1,4}))|(?:' + IPV4_PATTERN_WITHOUT_ANCHORS + r')))|' + \
+IPV6_PATTERN_WITHOUT_ANCHORS = r'(?:' + \
+                               r'(?:(?:(?:[0-9a-fA-F]{1,4}:){6})(?:(?:(?:[0-9a-fA-F]{1,4}):(?:[0-9a-fA-F]{1,4}))|(?:' + IPV4_PATTERN_WITHOUT_ANCHORS + r')))|' + \
                                r'(?:::(?:(?:[0-9a-fA-F]{1,4}:){5})(?:(?:(?:[0-9a-fA-F]{1,4}):(?:[0-9a-fA-F]{1,4}))|(?:' + IPV4_PATTERN_WITHOUT_ANCHORS + r')))|' + \
                                r'(?:(?:[0-9a-fA-F]{1,4})?::(?:(?:[0-9a-fA-F]{1,4}:){4})(?:(?:(?:[0-9a-fA-F]{1,4}):(?:[0-9a-fA-F]{1,4}))|(?:' + IPV4_PATTERN_WITHOUT_ANCHORS + r')))|' + \
                                r'(?:(?:(?:[0-9a-fA-F]{1,4}:){,1}(?:[0-9a-fA-F]{1,4}))?::(?:(?:[0-9a-fA-F]{1,4}:){3})(?:(?:(?:[0-9a-fA-F]{1,4}):(?:[0-9a-fA-F]{1,4}))|(?:' + IPV4_PATTERN_WITHOUT_ANCHORS + r')))|' + \
@@ -85,7 +86,8 @@
                                r'(?:(?:(?:[0-9a-fA-F]{1,4}:){,3}(?:[0-9a-fA-F]{1,4}))?::(?:(?:[0-9a-fA-F]{1,4}:){1})(?:(?:(?:[0-9a-fA-F]{1,4}):(?:[0-9a-fA-F]{1,4}))|(?:' + IPV4_PATTERN_WITHOUT_ANCHORS + r')))|' + \
                                r'(?:(?:(?:[0-9a-fA-F]{1,4}:){,4}(?:[0-9a-fA-F]{1,4}))?::(?:(?:(?:[0-9a-fA-F]{1,4}):(?:[0-9a-fA-F]{1,4}))|(?:' + IPV4_PATTERN_WITHOUT_ANCHORS + r')))|' + \
                                r'(?:(?:(?:[0-9a-fA-F]{1,4}:){,5}(?:[0-9a-fA-F]{1,4}))?::(?:[0-9a-fA-F]{1,4}))|' + \
-                               r'(?:(?:(?:[0-9a-fA-F]{1,4}:){,6}(?:[0-9a-fA-F]{1,4}))?::)'
+                               r'(?:(?:(?:[0-9a-fA-F]{1,4}:){,6}(?:[0-9a-fA-F]{1,4}))?::)' + \
+                               r')'

# Hostname validation
 # A hostname consists of sections separated by periods. Each of these sections

[rhinstaller/anaconda/pulls/238 rhel7-branch] fix the URL check for URLs with IPv6 literals

[dashea]

Added label: ACK.

[rhinstaller/anaconda/pulls/238 rhel7-branch] fix the URL check for URLs with IPv6 literals

[dashea]

Darn, too bad this version doesn't have the ci target, would be nice to see jenkins test the new tests :)

[rhinstaller/anaconda/pulls/238 rhel7-branch] fix the URL check for URLs with IPv6 literals

[dashea]

Because of this I remember that I want to remove this regex madness of IP addresses (which doesn't cache all) and add IPy package instead of it. But this commit is alright so push it and I will change it on top of the master.
Thank you

[rhinstaller/anaconda/pulls/238 rhel7-branch] fix the URL check for URLs with IPv6 literals

[dashea]

Closed.

[rhinstaller/anaconda/pulls/238 rhel7-branch] fix the URL check for URLs with IPv6 literals

[dashea]

Pushed.