Multiple test failures w/ pypy{,3}-6.0.0

[mgorny]

$ tox -e pypy
GLOB sdist-make: /tmp/pyflakes/setup.py
pypy create: /tmp/pyflakes/.tox/pypy
pypy inst: /tmp/pyflakes/.tox/dist/pyflakes-2.0.0.zip
pypy installed: cffi==1.11.5,greenlet==0.4.13,pyflakes==2.0.0,readline==6.2.4.1
pypy runtests: PYTHONHASHSEED='444501725'
pypy runtests: commands[0] | pip install flake8==2.1.0 pep8==1.5.6
Collecting flake8==2.1.0
  Downloading https://files.pythonhosted.org/packages/8c/a7/bce19129cb750a4f0720440801de328eafb18dd6bb94638d7bd350a8eead/flake8-2.1.0.tar.gz
Collecting pep8==1.5.6
  Downloading https://files.pythonhosted.org/packages/c5/de/5bfd80ea3b7e3fdf76ba12c832be60b886a2651cb9badd11848ebd9d0d11/pep8-1.5.6-py2.py3-none-any.whl
Requirement already satisfied: pyflakes>=0.7.3 in ./.tox/pypy/site-packages (from flake8==2.1.0) (2.0.0)
Collecting mccabe>=0.2.1 (from flake8==2.1.0)
  Using cached https://files.pythonhosted.org/packages/87/89/479dc97e18549e21354893e4ee4ef36db1d237534982482c3681ee6e7b57/mccabe-0.6.1-py2.py3-none-any.whl
Building wheels for collected packages: flake8
  Running setup.py bdist_wheel for flake8 ... done
  Stored in directory: /home/mgorny/.cache/pip/wheels/1a/4c/c1/2126020602d3f40be5eed59aaf25c5decc4593de41a985312d
Successfully built flake8
Installing collected packages: pep8, mccabe, flake8
Successfully installed flake8-2.1.0 mccabe-0.6.1 pep8-1.5.6
pypy runtests: commands[1] | flake8 --version
2.1.0 (pep8: 1.5.6, mccabe: 0.6.1, pyflakes: 2.0.0) PyPy 2.7.13 on Linux
pypy runtests: commands[2] | python setup.py test -q
running test
running egg_info
writing pyflakes.egg-info/PKG-INFO
writing dependency_links to pyflakes.egg-info/dependency_links.txt
writing entry points to pyflakes.egg-info/entry_points.txt
writing top-level names to pyflakes.egg-info/top_level.txt
reading manifest file 'pyflakes.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'pyflakes.egg-info/SOURCES.txt'
running build_ext
.s....s.........................ss.........s....sss..ss.....s...sss.............s.......s........s.......s..............ss...sssssssssss.s.........s.......s..s......s....................s....................ss................................................................................................................s...F......F..F....................ss........................................................................................s.......s........s.......s..............ss...s....s.........................ss.........s....sss..ss.....s...sss.....s.......s......................Fs....F.....F...........F........
======================================================================
FAIL: test_indentationErrorInDoctest (pyflakes.test.test_doctests.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pyflakes/pyflakes/test/test_doctests.py", line 358, in test_indentationErrorInDoctest
    self.assertEqual(exc.col, 13)
AssertionError: 14 != 13

======================================================================
FAIL: test_offsetWithMultiLineArgs (pyflakes.test.test_doctests.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pyflakes/pyflakes/test/test_doctests.py", line 376, in test_offsetWithMultiLineArgs
    self.assertEqual(exc1.col, 19)
AssertionError: 20 != 19

======================================================================
FAIL: test_syntaxErrorInDoctest (pyflakes.test.test_doctests.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pyflakes/pyflakes/test/test_doctests.py", line 330, in test_syntaxErrorInDoctest
    self.assertEqual(exc.col, 26)
AssertionError: 27 != 26

======================================================================
FAIL: test_eofSyntaxErrorWithTab (pyflakes.test.test_api.CheckTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pyflakes/pyflakes/test/test_api.py", line 471, in test_eofSyntaxErrorWithTab
    """ % (sourcePath, column, last_line)])
  File "/tmp/pyflakes/pyflakes/test/test_api.py", line 349, in assertHasErrors
    (count, err.getvalue()), (len(errorList), ''.join(errorList)))
AssertionError: Tuples differ: (1, '/tmp/tmpzLc7m6:2:6: inval... != (1, '/tmp/tmpzLc7m6:2:5: inval...

First differing element 1:
'/tmp/tmpzLc7m6:2:6: invalid syntax\n\tfoo =\n\t    ^\n'
'/tmp/tmpzLc7m6:2:5: invalid syntax\n\tfoo =\n\t   ^\n'

- (1, '/tmp/tmpzLc7m6:2:6: invalid syntax\n\tfoo =\n\t    ^\n')
?                       ^                                -

+ (1, '/tmp/tmpzLc7m6:2:5: invalid syntax\n\tfoo =\n\t   ^\n')
?                       ^


======================================================================
FAIL: test_multilineSyntaxError (pyflakes.test.test_api.CheckTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pyflakes/pyflakes/test/test_api.py", line 431, in test_multilineSyntaxError
    """ % (sourcePath, message)])
  File "/tmp/pyflakes/pyflakes/test/test_api.py", line 349, in assertHasErrors
    (count, err.getvalue()), (len(errorList), ''.join(errorList)))
AssertionError: Tuples differ: (1, "/tmp/tmpKVtrN2:8:11: end ... != (1, "/tmp/tmpKVtrN2:8:11: EOF ...

First differing element 1:
"/tmp/tmpKVtrN2:8:11: end of file (EOF) while scanning triple-quoted string literal\n    '''quux'''\n          ^\n"
"/tmp/tmpKVtrN2:8:11: EOF while scanning triple-quoted string literal\n    '''quux'''\n          ^\n"

  (1,
-  "/tmp/tmpKVtrN2:8:11: end of file (EOF) while scanning triple-quoted string literal\n    '''quux'''\n          ^\n")
?                        -------------   -

+  "/tmp/tmpKVtrN2:8:11: EOF while scanning triple-quoted string literal\n    '''quux'''\n          ^\n")

======================================================================
FAIL: test_errors_syntax (pyflakes.test.test_api.IntegrationTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pyflakes/pyflakes/test/test_api.py", line 749, in test_errors_syntax
    self.assertEqual(d, ('', error_msg, True))
AssertionError: Tuples differ: ('', '/tmp/tmpJSmDhI/temp:1:6:... != ('', '/tmp/tmpJSmDhI/temp:1:5:...

First differing element 1:
'/tmp/tmpJSmDhI/temp:1:6: invalid syntax\nimport\n     ^\n'
'/tmp/tmpJSmDhI/temp:1:5: invalid syntax\nimport\n    ^\n'

- ('', '/tmp/tmpJSmDhI/temp:1:6: invalid syntax\nimport\n     ^\n', 1)
?                             ^                              -      ^

+ ('', '/tmp/tmpJSmDhI/temp:1:5: invalid syntax\nimport\n    ^\n', True)
?                             ^                                    ^^^^


======================================================================
FAIL: test_errors_syntax (pyflakes.test.test_api.TestMain)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pyflakes/pyflakes/test/test_api.py", line 749, in test_errors_syntax
    self.assertEqual(d, ('', error_msg, True))
AssertionError: Tuples differ: ('', '/tmp/tmp6iJma3/temp:1:6:... != ('', '/tmp/tmp6iJma3/temp:1:5:...

First differing element 1:
'/tmp/tmp6iJma3/temp:1:6: invalid syntax\nimport\n     ^\n'
'/tmp/tmp6iJma3/temp:1:5: invalid syntax\nimport\n    ^\n'

- ('', '/tmp/tmp6iJma3/temp:1:6: invalid syntax\nimport\n     ^\n', True)
?                             ^                              -

+ ('', '/tmp/tmp6iJma3/temp:1:5: invalid syntax\nimport\n    ^\n', True)
?                             ^


----------------------------------------------------------------------
Ran 626 tests in 4.000s

FAILED (failures=7, skipped=65)
Test failed: <unittest.runner.TextTestResult run=626 errors=0 failures=7>
error: Test failed: <unittest.runner.TextTestResult run=626 errors=0 failures=7>
ERROR: InvocationError: '/tmp/pyflakes/.tox/pypy/bin/python setup.py test -q'
_____________________________________________________________________ summary _____________________________________________________________________
ERROR:   pypy: commands failed

$ tox -e pypy3
GLOB sdist-make: /tmp/pyflakes/setup.py
pypy3 create: /tmp/pyflakes/.tox/pypy3
pypy3 inst: /tmp/pyflakes/.tox/dist/pyflakes-2.0.0.zip
pypy3 installed: cffi==1.11.5,greenlet==0.4.13,pyflakes==2.0.0,readline==6.2.4.1
pypy3 runtests: PYTHONHASHSEED='3614297916'
pypy3 runtests: commands[0] | pip install flake8==2.1.0 pep8==1.5.6
Collecting flake8==2.1.0
  Cache entry deserialization failed, entry ignored
  Using cached https://files.pythonhosted.org/packages/8c/a7/bce19129cb750a4f0720440801de328eafb18dd6bb94638d7bd350a8eead/flake8-2.1.0.tar.gz
Collecting pep8==1.5.6
  Cache entry deserialization failed, entry ignored
  Using cached https://files.pythonhosted.org/packages/c5/de/5bfd80ea3b7e3fdf76ba12c832be60b886a2651cb9badd11848ebd9d0d11/pep8-1.5.6-py2.py3-none-any.whl
Requirement already satisfied: pyflakes>=0.7.3 in ./.tox/pypy3/site-packages (from flake8==2.1.0) (2.0.0)
Collecting mccabe>=0.2.1 (from flake8==2.1.0)
  Cache entry deserialization failed, entry ignored
  Using cached https://files.pythonhosted.org/packages/87/89/479dc97e18549e21354893e4ee4ef36db1d237534982482c3681ee6e7b57/mccabe-0.6.1-py2.py3-none-any.whl
Building wheels for collected packages: flake8
  Running setup.py bdist_wheel for flake8 ... done
  Stored in directory: /home/mgorny/.cache/pip/wheels/1a/4c/c1/2126020602d3f40be5eed59aaf25c5decc4593de41a985312d
Successfully built flake8
Installing collected packages: pep8, mccabe, flake8
Successfully installed flake8-2.1.0 mccabe-0.6.1 pep8-1.5.6
pypy3 runtests: commands[1] | flake8 --version
2.1.0 (pep8: 1.5.6, mccabe: 0.6.1, pyflakes: 2.0.0) PyPy 3.5.3 on Linux
pypy3 runtests: commands[2] | python setup.py test -q
running test
running egg_info
writing pyflakes.egg-info/PKG-INFO
writing dependency_links to pyflakes.egg-info/dependency_links.txt
writing entry points to pyflakes.egg-info/entry_points.txt
writing top-level names to pyflakes.egg-info/top_level.txt
reading manifest file 'pyflakes.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'pyflakes.egg-info/SOURCES.txt'
running build_ext
.....................................s...........ss..s.........s.s.sss..................s..................s......s.................s....s....................s......s.....................................s...s.............s...............................s...................................................................s...F......F..F................s...s.............s...............................s...................................................s..................s......s..............................................s...........ss..s.........s.s.....s.......s.......................Fs....FFF...F...........F........
======================================================================
FAIL: test_indentationErrorInDoctest (pyflakes.test.test_doctests.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pyflakes/pyflakes/test/test_doctests.py", line 358, in test_indentationErrorInDoctest
    self.assertEqual(exc.col, 13)
AssertionError: 14 != 13

======================================================================
FAIL: test_offsetWithMultiLineArgs (pyflakes.test.test_doctests.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pyflakes/pyflakes/test/test_doctests.py", line 376, in test_offsetWithMultiLineArgs
    self.assertEqual(exc1.col, 19)
AssertionError: 20 != 19

======================================================================
FAIL: test_syntaxErrorInDoctest (pyflakes.test.test_doctests.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pyflakes/pyflakes/test/test_doctests.py", line 330, in test_syntaxErrorInDoctest
    self.assertEqual(exc.col, 26)
AssertionError: 27 != 26

======================================================================
FAIL: test_eofSyntaxErrorWithTab (pyflakes.test.test_api.CheckTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pyflakes/pyflakes/test/test_api.py", line 471, in test_eofSyntaxErrorWithTab
    """ % (sourcePath, column, last_line)])
  File "/tmp/pyflakes/pyflakes/test/test_api.py", line 349, in assertHasErrors
    (count, err.getvalue()), (len(errorList), ''.join(errorList)))
AssertionError: Tuples differ: (1, '/tmp/tmpucswcorh:2:6: invalid syntax\n\tfoo =\n\t    ^\n') != (1, '/tmp/tmpucswcorh:2:5: invalid syntax\n\tfoo =\n\t   ^\n')

First differing element 1:
'/tmp/tmpucswcorh:2:6: invalid syntax\n\tfoo =\n\t    ^\n'
'/tmp/tmpucswcorh:2:5: invalid syntax\n\tfoo =\n\t   ^\n'

- (1, '/tmp/tmpucswcorh:2:6: invalid syntax\n\tfoo =\n\t    ^\n')
?                         ^                                -

+ (1, '/tmp/tmpucswcorh:2:5: invalid syntax\n\tfoo =\n\t   ^\n')
?                         ^


======================================================================
FAIL: test_multilineSyntaxError (pyflakes.test.test_api.CheckTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pyflakes/pyflakes/test/test_api.py", line 431, in test_multilineSyntaxError
    """ % (sourcePath, message)])
  File "/tmp/pyflakes/pyflakes/test/test_api.py", line 349, in assertHasErrors
    (count, err.getvalue()), (len(errorList), ''.join(errorList)))
AssertionError: Tuples differ: (1, "[18 chars]:11: end of file (EOF) while scanning triple-q[48 chars]^\n") != (1, "[18 chars]:11: EOF while scanning triple-quoted string l[34 chars]^\n")

First differing element 1:
"/tmp[14 chars]:11: end of file (EOF) while scanning triple-q[47 chars] ^\n"
"/tmp[14 chars]:11: EOF while scanning triple-quoted string l[33 chars] ^\n"

  (1,
-  '/tmp/tmpnla8400t:8:11: end of file (EOF) while scanning triple-quoted string '
?                          -------------   -

+  '/tmp/tmpnla8400t:8:11: EOF while scanning triple-quoted string literal\n'
?                                                                  +++++++++

-  'literal\n'
   "    '''quux'''\n"
   '          ^\n')

======================================================================
FAIL: test_nonDefaultFollowsDefaultSyntaxError (pyflakes.test.test_api.CheckTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pyflakes/pyflakes/test/test_api.py", line 491, in test_nonDefaultFollowsDefaultSyntaxError
    %s""" % (sourcePath, column, last_line)])
  File "/tmp/pyflakes/pyflakes/test/test_api.py", line 349, in assertHasErrors
    (count, err.getvalue()), (len(errorList), ''.join(errorList)))
AssertionError: Tuples differ: (1, '[14 chars]5r:1:7: non-default argument follows default a[39 chars]^\n') != (1, '[14 chars]5r:1:8: non-default argument follows default a[40 chars]^\n')

First differing element 1:
'/tmp/tmphse0k15r:1:7: non-default argument follows default a[38 chars] ^\n'
'/tmp/tmphse0k15r:1:8: non-default argument follows default a[39 chars] ^\n'

  (1,
-  '/tmp/tmphse0k15r:1:7: non-default argument follows default argument\n'
?                      ^

+  '/tmp/tmphse0k15r:1:8: non-default argument follows default argument\n'
?                      ^

   'def foo(bar=baz, bax):\n'
-  '      ^\n')
+  '       ^\n')
?   +


======================================================================
FAIL: test_nonKeywordAfterKeywordSyntaxError (pyflakes.test.test_api.CheckTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pyflakes/pyflakes/test/test_api.py", line 516, in test_nonKeywordAfterKeywordSyntaxError
    %s""" % (sourcePath, column, message, last_line)])
  File "/tmp/pyflakes/pyflakes/test/test_api.py", line 349, in assertHasErrors
    (count, err.getvalue()), (len(errorList), ''.join(errorList)))
AssertionError: Tuples differ: (1, '[15 chars]i:1:12: positional argument follows keyword ar[38 chars]^\n') != (1, '[15 chars]i:1:13: positional argument follows keyword ar[39 chars]^\n')

First differing element 1:
'/tmp/tmps7dat83i:1:12: positional argument follows keyword ar[37 chars] ^\n'
'/tmp/tmps7dat83i:1:13: positional argument follows keyword ar[38 chars] ^\n'

  (1,
-  '/tmp/tmps7dat83i:1:12: positional argument follows keyword argument\n'
?                       ^

+  '/tmp/tmps7dat83i:1:13: positional argument follows keyword argument\n'
?                       ^

   'foo(bar=baz, bax)\n'
-  '           ^\n')
+  '            ^\n')
?   +


======================================================================
FAIL: test_errors_syntax (pyflakes.test.test_api.IntegrationTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pyflakes/pyflakes/test/test_api.py", line 749, in test_errors_syntax
    self.assertEqual(d, ('', error_msg, True))
AssertionError: Tuples differ: ('', '/tmp/tmpeb455z9q/temp:1:6: invalid syntax\nimport\n     ^\n', 1) != ('', '/tmp/tmpeb455z9q/temp:1:5: invalid syntax\nimport\n    ^\n', True)

First differing element 1:
'/tmp/tmpeb455z9q/temp:1:6: invalid syntax\nimport\n     ^\n'
'/tmp/tmpeb455z9q/temp:1:5: invalid syntax\nimport\n    ^\n'

- ('', '/tmp/tmpeb455z9q/temp:1:6: invalid syntax\nimport\n     ^\n', 1)
?                               ^                              -      ^

+ ('', '/tmp/tmpeb455z9q/temp:1:5: invalid syntax\nimport\n    ^\n', True)
?                               ^                                    ^^^^


======================================================================
FAIL: test_errors_syntax (pyflakes.test.test_api.TestMain)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pyflakes/pyflakes/test/test_api.py", line 749, in test_errors_syntax
    self.assertEqual(d, ('', error_msg, True))
AssertionError: Tuples differ: ('', '/tmp/tmpt5zbfcfr/temp:1:6: invalid syntax\nimport\n     ^\n', True) != ('', '/tmp/tmpt5zbfcfr/temp:1:5: invalid syntax\nimport\n    ^\n', True)

First differing element 1:
'/tmp/tmpt5zbfcfr/temp:1:6: invalid syntax\nimport\n     ^\n'
'/tmp/tmpt5zbfcfr/temp:1:5: invalid syntax\nimport\n    ^\n'

- ('', '/tmp/tmpt5zbfcfr/temp:1:6: invalid syntax\nimport\n     ^\n', True)
?                               ^                              -

+ ('', '/tmp/tmpt5zbfcfr/temp:1:5: invalid syntax\nimport\n    ^\n', True)
?                               ^


----------------------------------------------------------------------
Ran 626 tests in 6.249s

FAILED (failures=9, skipped=37)
Test failed: <unittest.runner.TextTestResult run=626 errors=0 failures=9>
error: Test failed: <unittest.runner.TextTestResult run=626 errors=0 failures=9>
ERROR: InvocationError: '/tmp/pyflakes/.tox/pypy3/bin/python setup.py test -q'
_____________________________________________________________________ summary _____________________________________________________________________
ERROR:   pypy3: commands failed

[jayvdb]

Noting travis doesnt have pypy 3.6 yet. travis-ci/travis-ci#9542 (or nightlies).

I expect jayvdb@acf98b2 is the fix for pypy3.6 but I know this bug in pypy was fixed prior to 3.6, but havent found which version it was fixed in yet.

[jayvdb]

I'm pretty sure the bug was fixed in PyPy 5.10 or 5.10.1 (Python 3.5.3).

We cant use python version for this, as PyPy 5.9, 5.10 and PyPy 6.0 all report CPython version 3.5.3.
https://bitbucket.org/pypy/pypy/src/cad71ec84cef8b29b6c97e3f83cadb4065e6a28a/pypy/module/sys/version.py?at=py3.5&fileviewer=file-view-default
https://github.com/mozillazg/pypy/blob/py3.5/pypy/module/sys/version.py

PyPy 6.1 looks like it will be py36
https://bitbucket.org/pypy/pypy/src/289d8154d34d7eca4bc4fb3dd9fdce765742eae6/pypy/module/sys/version.py?at=py3.6&fileviewer=file-view-default

[jayvdb]

Ugh, the problem was known in #322 (a9b242f) and hidden then instead of creating an issue.

[asottile]

I've fixed the tests for 7.1.1 in #468