3.5.4rc1 - Cpython — ContextQMD

.. bpo: 30730 .. date: 084 .. nonce: rJsyTH .. original section: Library .. release date: 2017-07-23 .. section: Security

Prevent environment variables injection in subprocess on Windows. Prevent passing other environment variables and command arguments.

.. bpo: 30694 .. date: 083 .. nonce: WkMWM_ .. original section: Library .. section: Security

Upgrade expat copy from 2.2.0 to 2.2.1 to get fixes of multiple security vulnerabilities including: :cve:2017-9233 (External entity infinite loop DoS), :cve:2016-9063 (Integer overflow, re-fix), :cve:2016-0718 (Fix regression bugs from 2.2.0's fix to :cve:2016-0718) and :cve:2012-0876 (Counter hash flooding with SipHash). Note: the :cve:2016-5300 (Use os-specific entropy sources like getrandom) doesn't impact Python, since Python already gets entropy from the OS to set the expat secret using XML_SetHashSalt().

.. bpo: 30500 .. date: 081 .. nonce: 1VG7R- .. original section: Library .. section: Security

Fix urllib.parse.splithost() to correctly parse fragments. For example, splithost('//127.0.0.1#@evil.com/') now correctly returns the 127.0.0.1 host, instead of treating @evil.com as the host in an authentication (login@host).

.. bpo: 29591 .. date: 076 .. nonce: ExKblw .. original section: Library .. section: Security

Update expat copy from 2.1.1 to 2.2.0 to get fixes of :cve:2016-0718 and :cve:2016-4472. See https://sourceforge.net/p/expat/bugs/537/ for more information.

.. bpo: 30876 .. date: 2017-07-11-06-31-32 .. nonce: x35jZX .. section: Core and Builtins

Relative import from unloaded package now reimports the package instead of failing with SystemError. Relative import from non-package now fails with ImportError rather than SystemError.

.. bpo: 30765 .. date: 2017-06-26-14-29-50 .. nonce: Q5iBmf .. section: Core and Builtins

Avoid blocking in pthread_mutex_lock() when PyThread_acquire_lock() is asked not to block.

.. bpo: 27945 .. date: 100 .. nonce: p29r3O .. section: Core and Builtins

Fixed various segfaults with dict when input collections are mutated during searching, inserting or comparing. Based on patches by Duane Griffin and Tim Mitchell.

.. bpo: 25794 .. date: 099 .. nonce: xfPwqm .. section: Core and Builtins

Fixed type.setattr() and type.delattr() for non-interned attribute names. Based on patch by Eryk Sun.

.. bpo: 29935 .. date: 098 .. nonce: vgjdJo .. section: Core and Builtins

Fixed error messages in the index() method of tuple, list and deque when pass indices of wrong type.

.. bpo: 28876 .. date: 097 .. nonce: cU-sGT .. section: Core and Builtins

bool(range) works even if len(range) raises :exc:OverflowError.

.. bpo: 29600 .. date: 096 .. nonce: 77wQ6C .. section: Core and Builtins

Fix wrapping coroutine return values in StopIteration.

.. bpo: 29537 .. date: 095 .. nonce: lu1ysY .. section: Core and Builtins

Restore runtime compatibility with bytecode files generated by CPython 3.5.0 to 3.5.2, and adjust the eval loop to avoid the problems that could be caused by the malformed variant of the BUILD_MAP_UNPACK_WITH_CALL opcode that they may contain. Patch by Petr Viktorin, Serhiy Storchaka, and Nick Coghlan.

.. bpo: 28598 .. date: 094 .. nonce: QxbzQn .. section: Core and Builtins

Support rmod for subclasses of str being called before str.mod. Patch by Martijn Pieters.

.. bpo: 29602 .. date: 093 .. nonce: qyyskC .. section: Core and Builtins

Fix incorrect handling of signed zeros in complex constructor for complex subclasses and for inputs having a complex method. Patch by Serhiy Storchaka.

.. bpo: 29347 .. date: 092 .. nonce: 1RPPGN .. section: Core and Builtins

Fixed possibly dereferencing undefined pointers when creating weakref objects.

.. bpo: 29438 .. date: 091 .. nonce: IKxD6I .. section: Core and Builtins

Fixed use-after-free problem in key sharing dict.

.. bpo: 29319 .. date: 090 .. nonce: KLDUZf .. section: Core and Builtins

Prevent RunMainFromImporter overwriting sys.path[0].

.. bpo: 29337 .. date: 089 .. nonce: bjX8AE .. section: Core and Builtins

Fixed possible BytesWarning when compare the code objects. Warnings could be emitted at compile time.

.. bpo: 29478 .. date: 088 .. nonce: rTQ-qy .. section: Core and Builtins

If max_line_length=None is specified while using the Compat32 policy, it is no longer ignored. Patch by Mircea Cosbuc.

.. bpo: 29403 .. date: 2017-07-20-02-29-49 .. nonce: 3RinCV .. section: Library

Fix unittest.mock's autospec to not fail on method-bound builtin functions. Patch by Aaron Gallagher.

.. bpo: 30961 .. date: 2017-07-18-23-47-51 .. nonce: 064jz0 .. section: Library

Fix decrementing a borrowed reference in tracemalloc.

.. bpo: 30886 .. date: 2017-07-10-12-14-22 .. nonce: nqQj34 .. section: Library

Fix multiprocessing.Queue.join_thread(): it now waits until the thread completes, even if the thread was started by the same process which created the queue.

.. bpo: 29854 .. date: 2017-07-07-02-18-57 .. nonce: J8wKb_ .. section: Library

Fix segfault in readline when using readline's history-size option. Patch by Nir Soffer.

.. bpo: 30807 .. date: 2017-06-29-22-04-44 .. nonce: sLtjY- .. section: Library

signal.setitimer() may disable the timer when passed a tiny value. Tiny values (such as 1e-6) are valid non-zero values for setitimer(), which is specified as taking microsecond-resolution intervals. However, on some platform, our conversion routine could convert 1e-6 into a zero interval, therefore disabling the timer instead of (re-)scheduling it.

.. bpo: 30441 .. date: 2017-06-29-14-25-14 .. nonce: 3Wh9kc .. section: Library

Fix bug when modifying os.environ while iterating over it

.. bpo: 30532 .. date: 2017-06-26-11-01-59 .. nonce: qTeL1o .. section: Library

Fix email header value parser dropping folding white space in certain cases.

.. bpo: 29169 .. date: 087 .. nonce: 8ypApm .. section: Library

Update zlib to 1.2.11.

.. bpo: 30879 .. date: 086 .. nonce: N3KI-o .. section: Library

os.listdir() and os.scandir() now emit bytes names when called with bytes-like argument.

.. bpo: 30746 .. date: 085 .. nonce: 7drQI0 .. section: Library

Prohibited the '=' character in environment variable names in os.putenv() and os.spawn*().

.. bpo: 29755 .. date: 082 .. nonce: diQcY_ .. section: Library

Fixed the lgettext() family of functions in the gettext module. They now always return bytes.

.. bpo: 30645 .. date: 080 .. nonce: oYzbbW .. section: Library

Fix path calculation in imp.load_package(), fixing it for cases when a package is only shipped with bytecodes. Patch by Alexandru Ardelean.

.. bpo: 23890 .. date: 079 .. nonce: GCFAAZ .. section: Library

unittest.TestCase.assertRaises() now manually breaks a reference cycle to not keep objects alive longer than expected.

.. bpo: 30149 .. date: 078 .. nonce: hE649r .. section: Library

inspect.signature() now supports callables with variable-argument parameters wrapped with partialmethod. Patch by Donghee Na.

.. bpo: 29931 .. date: 077 .. nonce: tfcTwK .. section: Library

Fixed comparison check for ipaddress.ip_interface objects. Patch by Sanjay Sundaresan.

.. bpo: 24484 .. date: 075 .. nonce: vFem8K .. section: Library

Avoid race condition in multiprocessing cleanup.

.. bpo: 28994 .. date: 074 .. nonce: 9vzun1 .. section: Library

The traceback no longer displayed for SystemExit raised in a callback registered by atexit.

.. bpo: 30508 .. date: 073 .. nonce: wNWRS2 .. section: Library

Don't log exceptions if Task/Future "cancel()" method was called.

.. bpo: 28556 .. date: 072 .. nonce: mESP7G .. section: Library

Updates to typing module: Add generic AsyncContextManager, add support for ContextManager on all versions. Original PRs by Jelle Zijlstra and Ivan Levkivskyi

.. bpo: 29870 .. date: 071 .. nonce: p960Ih .. section: Library

Fix ssl sockets leaks when connection is aborted in asyncio/ssl implementation. Patch by Michaël Sghaïer.

.. bpo: 29743 .. date: 070 .. nonce: en2P4s .. section: Library

Closing transport during handshake process leaks open socket. Patch by Nikolay Kim

.. bpo: 27585 .. date: 069 .. nonce: 0Ugqqu .. section: Library

Fix waiter cancellation in asyncio.Lock. Patch by Mathieu Sornay.

.. bpo: 30418 .. date: 068 .. nonce: EwISQm .. section: Library

On Windows, subprocess.Popen.communicate() now also ignore EINVAL on stdin.write() if the child process is still running but closed the pipe.

.. bpo: 30378 .. date: 067 .. nonce: R_19_5 .. section: Library

Fix the problem that logging.handlers.SysLogHandler cannot handle IPv6 addresses.

.. bpo: 29960 .. date: 066 .. nonce: g0wr3r .. section: Library

Preserve generator state when _random.Random.setstate() raises an exception. Patch by Bryan Olson.

.. bpo: 30414 .. date: 065 .. nonce: jGl1Lb .. section: Library

multiprocessing.Queue._feed background running thread do not break from main loop on exception.

.. bpo: 30003 .. date: 064 .. nonce: BOl9HE .. section: Library

Fix handling escape characters in HZ codec. Based on patch by Ma Lin.

.. bpo: 30301 .. date: 063 .. nonce: ywOkjN .. section: Library

Fix AttributeError when using SimpleQueue.empty() under spawn and forkserver start methods.

.. bpo: 30329 .. date: 062 .. nonce: EuT36N .. section: Library

imaplib and poplib now catch the Windows socket WSAEINVAL error (code 10022) on shutdown(SHUT_RDWR): An invalid operation was attempted. This error occurs sometimes on SSL connections.

.. bpo: 30375 .. date: 061 .. nonce: 9c8qM7 .. section: Library

Warnings emitted when compile a regular expression now always point to the line in the user code. Previously they could point into inners of the re module if emitted from inside of groups or conditionals.

.. bpo: 30048 .. date: 060 .. nonce: ELRx8R .. section: Library

Fixed Task.cancel() can be ignored when the task is running coroutine and the coroutine returned without any more await.

.. bpo: 29990 .. date: 059 .. nonce: HWV6KE .. section: Library

Fix range checking in GB18030 decoder. Original patch by Ma Lin.

.. bpo: 26293 .. date: 058 .. nonce: wig0YG .. section: Library

Change resulted because of zipfile breakage. (See also: bpo-29094)

.. bpo: 30243 .. date: 057 .. nonce: RHQt0v .. section: Library

Removed the init methods of _json's scanner and encoder. Misusing them could cause memory leaks or crashes. Now scanner and encoder objects are completely initialized in the new methods.

.. bpo: 30185 .. date: 056 .. nonce: Tiu1n8 .. section: Library

Avoid KeyboardInterrupt tracebacks in forkserver helper process when Ctrl-C is received.

.. bpo: 28556 .. date: 055 .. nonce: 51gjbP .. section: Library

Various updates to typing module: add typing.NoReturn type, use WrapperDescriptorType, minor bug-fixes. Original PRs by Jim Fasarakis-Hilliard and Ivan Levkivskyi.

.. bpo: 30205 .. date: 054 .. nonce: BsxO34 .. section: Library

Fix getsockname() for unbound AF_UNIX sockets on Linux.

.. bpo: 30070 .. date: 053 .. nonce: XM_B41 .. section: Library

Fixed leaks and crashes in errors handling in the parser module.

.. bpo: 30061 .. date: 052 .. nonce: 2w_dX9 .. section: Library

Fixed crashes in IOBase methods next() and readlines() when readline() or next() respectively return non-sizeable object. Fixed possible other errors caused by not checking results of PyObject_Size(), PySequence_Size(), or PyMapping_Size().

.. bpo: 30068 .. date: 051 .. nonce: n4q47r .. section: Library

_io._IOBase.readlines will check if it's closed first when hint is present.

.. bpo: 29694 .. date: 050 .. nonce: LWKxb1 .. section: Library

Fixed race condition in pathlib mkdir with flags parents=True. Patch by Armin Rigo.

.. bpo: 29692 .. date: 049 .. nonce: oyWrAE .. section: Library

Fixed arbitrary unchaining of RuntimeError exceptions in contextlib.contextmanager. Patch by Siddharth Velankar.

.. bpo: 29998 .. date: 048 .. nonce: poeIKD .. section: Library

Pickling and copying ImportError now preserves name and path attributes.

.. bpo: 29942 .. date: 047 .. nonce: CsGNuT .. section: Library

Fix a crash in itertools.chain.from_iterable when encountering long runs of empty iterables.

.. bpo: 27863 .. date: 046 .. nonce: pPYHHI .. section: Library

Fixed multiple crashes in ElementTree caused by race conditions and wrong types.

.. bpo: 28699 .. date: 045 .. nonce: wZztZP .. section: Library

Fixed a bug in pools in multiprocessing.pool that raising an exception at the very first of an iterable may swallow the exception or make the program hang. Patch by Davin Potts and Xiang Zhang.

.. bpo: 25803 .. date: 044 .. nonce: CPDR0W .. section: Library

Avoid incorrect errors raised by Path.mkdir(exist_ok=True) when the OS gives priority to errors such as EACCES over EEXIST.

.. bpo: 29861 .. date: 043 .. nonce: t2ZoRK .. section: Library

Release references to tasks, their arguments and their results as soon as they are finished in multiprocessing.Pool.

.. bpo: 29884 .. date: 042 .. nonce: kWXR8W .. section: Library

faulthandler: Restore the old sigaltstack during teardown. Patch by Christophe Zeitouny.

.. bpo: 25455 .. date: 041 .. nonce: ZsahHN .. section: Library

Fixed crashes in repr of recursive buffered file-like objects.

.. bpo: 29800 .. date: 040 .. nonce: d2xASa .. section: Library

Fix crashes in partial.repr if the keys of partial.keywords are not strings. Patch by Michael Seifert.

.. bpo: 29742 .. date: 039 .. nonce: 8hqfEO .. section: Library

get_extra_info() raises exception if get called on closed ssl transport. Patch by Nikolay Kim.

.. bpo: 8256 .. date: 038 .. nonce: jAwGQH .. section: Library

Fixed possible failing or crashing input() if attributes "encoding" or "errors" of sys.stdin or sys.stdout are not set or are not strings.

.. bpo: 28298 .. date: 037 .. nonce: xfm84U .. section: Library

Fix a bug that prevented array 'Q', 'L' and 'I' from accepting big intables (objects that have int) as elements. Patch by Oren Milman.

.. bpo: 29615 .. date: 036 .. nonce: OpFKzg .. section: Library

SimpleXMLRPCDispatcher no longer chains KeyError (or any other exception) to exception(s) raised in the dispatched methods. Patch by Petr Motejlek.

.. bpo: 29704 .. date: 035 .. nonce: WHbx27 .. section: Library

asyncio.subprocess.SubprocessStreamProtocol no longer closes before all pipes are closed.

.. bpo: 29703 .. date: 034 .. nonce: ZdsPCR .. section: Library

Fix asyncio to support instantiation of new event loops in child processes.

.. bpo: 29376 .. date: 033 .. nonce: rrJhJy .. section: Library

Fix assertion error in threading._DummyThread.is_alive().

.. bpo: 29110 .. date: 032 .. nonce: wmE-_T .. section: Library

Fix file object leak in aifc.open() when file is given as a filesystem path and is not in valid AIFF format. Patch by Anthony Zhang.

.. bpo: 28961 .. date: 031 .. nonce: Rt93vg .. section: Library

Fix unittest.mock._Call helper: don't ignore the name parameter anymore. Patch written by Jiajun Huang.

.. bpo: 29532 .. date: 030 .. nonce: YCwVQn .. section: Library

Altering a kwarg dictionary passed to functools.partial() no longer affects a partial object after creation.

.. bpo: 28556 .. date: 029 .. nonce: p6967e .. section: Library

Various updates to typing module: typing.Counter, typing.ChainMap, improved ABC caching, etc. Original PRs by Jelle Zijlstra, Ivan Levkivskyi, Manuel Krebber, and Łukasz Langa.

.. bpo: 29100 .. date: 028 .. nonce: LAAERS .. section: Library

Fix datetime.fromtimestamp() regression introduced in Python 3.6.0: check minimum and maximum years.

.. bpo: 29519 .. date: 027 .. nonce: oGGgZ4 .. section: Library

Fix weakref spewing exceptions during interpreter shutdown when used with a rare combination of multiprocessing and custom codecs.

.. bpo: 29416 .. date: 026 .. nonce: KJGyI_ .. section: Library

Prevent infinite loop in pathlib.Path.mkdir

.. bpo: 29444 .. date: 025 .. nonce: cEwgmk .. section: Library

Fixed out-of-bounds buffer access in the group() method of the match object. Based on patch by WGH.

.. bpo: 29335 .. date: 024 .. nonce: _KC7IK .. section: Library

Fix subprocess.Popen.wait() when the child process has exited to a stopped instead of terminated state (ex: when under ptrace).

.. bpo: 29290 .. date: 023 .. nonce: XBqptF .. section: Library

Fix a regression in argparse that help messages would wrap at non-breaking spaces.

.. bpo: 28735 .. date: 022 .. nonce: admHLO .. section: Library

Fixed the comparison of mock.MagickMock with mock.ANY.

.. bpo: 29011 .. date: 021 .. nonce: MI5f2R .. section: Library

Fix an important omission by adding Deque to the typing module.

.. bpo: 29219 .. date: 020 .. nonce: kxui7t .. section: Library

Fixed infinite recursion in the repr of uninitialized ctypes.CDLL instances.

.. bpo: 28969 .. date: 019 .. nonce: j3HJYO .. section: Library

Fixed race condition in C implementation of functools.lru_cache. KeyError could be raised when cached function with full cache was simultaneously called from different threads with the same uncached arguments.

.. bpo: 29142 .. date: 018 .. nonce: xo6kAv .. section: Library

In urllib.request, suffixes in no_proxy environment variable with leading dots could match related hostnames again (e.g. .b.c matches a.b.c). Patch by Milan Oberkirch.

.. bpo: 30176 .. date: 013 .. nonce: VivmCg .. section: Documentation

Add missing attribute related constants in curses documentation.

.. bpo: 26985 .. date: 012 .. nonce: NB5_9S .. section: Documentation

Add missing info of code object in inspect documentation.

.. bpo: 28929 .. date: 011 .. nonce: Md7kb0 .. section: Documentation

Link the documentation to its source file on GitHub.

.. bpo: 25008 .. date: 010 .. nonce: CeIzyU .. section: Documentation

Document smtpd.py as effectively deprecated and add a pointer to aiosmtpd, a third-party asyncio-based replacement.

.. bpo: 26355 .. date: 009 .. nonce: SDq_8Y .. section: Documentation

Add canonical header link on each page to corresponding major version of the documentation. Patch by Matthias Bussonnier.

.. bpo: 29349 .. date: 008 .. nonce: PjSo-t .. section: Documentation

Fix Python 2 syntax in code for building the documentation.

.. bpo: 30822 .. date: 2017-07-20-14-29-54 .. nonce: X0wREo .. section: Tests

Fix regrtest command line parser to allow passing -u extralargefile to run test_zipfile64.

.. bpo: 30383 .. date: 2017-06-27-13-52-43 .. nonce: rCmrv7 .. section: Tests

regrtest: Enhance regrtest and backport features from the master branch. Add options: --coverage, --testdir, --list-tests (list test files, don't run them), --list-cases (list test identifiers, don't run them, :issue:30523), --matchfile (load a list of test filters from a text file, :issue:30540), --slowest (alias to --slow). Enhance output: add timestamp, test result, currently running tests, "Tests result: xxx" summary with total duration, etc. Fix reference leak hunting in regrtest, --huntrleaks: regrtest now warms up caches, create explicitly all internal singletons which are created on demand to prevent false positives when checking for reference leaks. (:issue:30675).

.. bpo: 30357 .. date: 004 .. nonce: n4CPEa .. section: Tests

test_thread: setUp() now uses support.threading_setup() and support.threading_cleanup() to wait until threads complete to avoid random side effects on following tests. Initial patch written by Grzegorz Grzywacz.

.. bpo: 28087 .. date: 003 .. nonce: m8dc4R .. section: Tests

Skip test_asyncore and test_eintr poll failures on macOS. Skip some tests of select.poll when running on macOS due to unresolved issues with the underlying system poll function on some macOS versions.

.. bpo: 30197 .. date: 002 .. nonce: c5wRfu .. section: Tests

Enhanced functions swap_attr() and swap_item() in the test.support module. They now work when delete replaced attribute or item inside the with statement. The old value of the attribute or item (or None if it doesn't exist) now will be assigned to the target of the "as" clause, if there is one.

.. bpo: 29571 .. date: 001 .. nonce: r6Dixr .. section: Tests

to match the behaviour of the re.LOCALE flag, test_re.test_locale_flag now uses locale.getpreferredencoding(False) to determine the candidate encoding for the test regex (allowing it to correctly skip the test when the default locale encoding is a multi-byte encoding)

.. bpo: 29243 .. date: 007 .. nonce: WDK4hT .. section: Build

Prevent unnecessary rebuilding of Python during make test, make install and some other make targets when configured with --enable-optimizations.

.. bpo: 23404 .. date: 006 .. nonce: PdYVWg .. section: Build

Don't regenerate generated files based on file modification time anymore: the action is now explicit. Replace make touch with make regen-all.

.. bpo: 29643 .. date: 005 .. nonce: 4WLIJQ .. section: Build

Fix --enable-optimization didn't work.

.. bpo: 30687 .. date: 017 .. nonce: 8mqHnu .. section: Windows

Locate msbuild.exe on Windows when building rather than vcvarsall.bat

.. bpo: 29392 .. date: 016 .. nonce: OtqS5t .. section: Windows

Prevent crash when passing invalid arguments into msvcrt module.

.. bpo: 27867 .. date: 015 .. nonce: VMCoJU .. section: C API

Function PySlice_GetIndicesEx() is replaced with a macro if Py_LIMITED_API is set to the value between 0x03050400 and 0x03060000 (not including) or 0x03060100 or higher.

.. bpo: 29083 .. date: 014 .. nonce: tGTjr_ .. section: C API

Fixed the declaration of some public API functions. PyArg_VaParse() and PyArg_VaParseTupleAndKeywords() were not available in limited API. PyArg_ValidateKeywordArguments(), PyArg_UnpackTuple() and Py_BuildValue() were not available in limited API of version < 3.3 when PY_SSIZE_T_CLEAN is defined.