========================== NumPy 1.11.0 Release Notes

This release supports Python 2.6 - 2.7 and 3.2 - 3.5 and contains a number of enhancements and improvements. Note also the build system changes listed below as they may have subtle effects.

No Windows (TM) binaries are provided for this release due to a broken toolchain. One of the providers of Python packages for Windows (TM) is your best bet.

Highlights

Details of these improvements can be found below.

• The datetime64 type is now timezone naive.
• A dtype parameter has been added to randint.
• Improved detection of two arrays possibly sharing memory.
• Automatic bin size estimation for np.histogram.
• Speed optimization of A @ A.T and dot(A, A.T).
• New function np.moveaxis for reordering array axes.

Build System Changes

• NumPy now uses setuptools for its builds instead of plain distutils. This fixes usage of install_requires='numpy' in the setup.py files of projects that depend on NumPy (see gh-6551). It potentially affects the way that build/install methods for NumPy itself behave though. Please report any unexpected behavior on the NumPy issue tracker.
• Bento build support and related files have been removed.
• Single file build support and related files have been removed.

Future Changes

The following changes are scheduled for NumPy 1.12.0.

• Support for Python 2.6, 3.2, and 3.3 will be dropped.
• Relaxed stride checking will become the default. See the 1.8.0 release notes for a more extended discussion of what this change implies.
• The behavior of the datetime64 "not a time" (NaT) value will be changed to match that of floating point "not a number" (NaN) values: all comparisons involving NaT will return False, except for NaT != NaT which will return True.
• Indexing with floats will raise IndexError, e.g., a[0, 0.0].
• Indexing with non-integer array_like will raise IndexError, e.g., a['1', '2']
• Indexing with multiple ellipsis will raise IndexError, e.g., a[..., ...].
• Non-integers used as index values will raise TypeError, e.g., in reshape, take, and specifying reduce axis.

In a future release the following changes will be made.

• The rand function exposed in numpy.testing will be removed. That function is left over from early NumPy and was implemented using the Python random module. The random number generators from numpy.random should be used instead.
• The ndarray.view method will only allow c_contiguous arrays to be viewed using a dtype of different size causing the last dimension to change. That differs from the current behavior where arrays that are f_contiguous but not c_contiguous can be viewed as a dtype type of different size causing the first dimension to change.
• Slicing a MaskedArray will return views of both data and mask. Currently the mask is copy-on-write and changes to the mask in the slice do not propagate to the original mask. See the FutureWarnings section below for details.

Compatibility notes

datetime64 changes

In prior versions of NumPy the experimental datetime64 type always stored times in UTC. By default, creating a datetime64 object from a string or printing it would convert from or to local time::

# old behavior
>>> np.datetime64('2000-01-01T00:00:00')
numpy.datetime64('2000-01-01T00:00:00-0800')  # note the timezone offset -08:00

A consensus of datetime64 users agreed that this behavior is undesirable and at odds with how datetime64 is usually used (e.g., by pandas <http://pandas.pydata.org>__). For most use cases, a timezone naive datetime type is preferred, similar to the datetime.datetime type in the Python standard library. Accordingly, datetime64 no longer assumes that input is in local time, nor does it print local times::

>>> np.datetime64('2000-01-01T00:00:00')
numpy.datetime64('2000-01-01T00:00:00')

For backwards compatibility, datetime64 still parses timezone offsets, which it handles by converting to UTC. However, the resulting datetime is timezone naive::

>>> np.datetime64('2000-01-01T00:00:00-08')
DeprecationWarning: parsing timezone aware datetimes is deprecated;
this will raise an error in the future
numpy.datetime64('2000-01-01T08:00:00')

As a corollary to this change, we no longer prohibit casting between datetimes with date units and datetimes with time units. With timezone naive datetimes, the rule for casting from dates to times is no longer ambiguous.

linalg.norm return type changes

The return type of the linalg.norm function is now floating point without exception. Some of the norm types previously returned integers.

polynomial fit changes

The various fit functions in the numpy polynomial package no longer accept non-integers for degree specification.

np.dot now raises TypeError instead of ValueError

This behavior mimics that of other functions such as np.inner. If the two arguments cannot be cast to a common type, it could have raised a TypeError or ValueError depending on their order. Now, np.dot will now always raise a TypeError.

FutureWarning to changed behavior

• In np.lib.split an empty array in the result always had dimension (0,) no matter the dimensions of the array being split. This has been changed so that the dimensions will be preserved. A FutureWarning for this change has been in place since Numpy 1.9 but, due to a bug, sometimes no warning was raised and the dimensions were already preserved.

% and // operators

These operators are implemented with the remainder and floor_divide functions respectively. Those functions are now based around fmod and are computed together so as to be compatible with each other and with the Python versions for float types. The results should be marginally more accurate or outright bug fixes compared to the previous results, but they may differ significantly in cases where roundoff makes a difference in the integer returned by floor_divide. Some corner cases also change, for instance, NaN is always returned for both functions when the divisor is zero, divmod(1.0, inf) returns (0.0, 1.0) except on MSVC 2008, and divmod(-1.0, inf) returns (-1.0, inf).

C API

Removed the check_return and inner_loop_selector members of the PyUFuncObject struct (replacing them with reserved slots to preserve struct layout). These were never used for anything, so it's unlikely that any third-party code is using them either, but we mention it here for completeness.

object dtype detection for old-style classes

In python 2, objects which are instances of old-style user-defined classes no longer automatically count as 'object' type in the dtype-detection handler. Instead, as in python 3, they may potentially count as sequences, but only if they define both a __len__ and a __getitem__ method. This fixes a segfault and inconsistency between python 2 and 3.

New Features

• np.histogram now provides plugin estimators for automatically estimating the optimal number of bins. Passing one of ['auto', 'fd', 'scott', 'rice', 'sturges'] as the argument to 'bins' results in the corresponding estimator being used.

• A benchmark suite using Airspeed Velocity <https://asv.readthedocs.io/>__ has been added, converting the previous vbench-based one. You can run the suite locally via python runtests.py --bench. For more details, see benchmarks/README.rst.

• A new function np.shares_memory that can check exactly whether two arrays have memory overlap is added. np.may_share_memory also now has an option to spend more effort to reduce false positives.

• SkipTest and KnownFailureException exception classes are exposed in the numpy.testing namespace. Raise them in a test function to mark the test to be skipped or mark it as a known failure, respectively.

• f2py.compile has a new extension keyword parameter that allows the fortran extension to be specified for generated temp files. For instance, the files can be specified to be *.f90. The verbose argument is also activated, it was previously ignored.

• A dtype parameter has been added to np.random.randint Random ndarrays of the following types can now be generated:

  • np.bool_,
  • np.int8, np.uint8,
  • np.int16, np.uint16,
  • np.int32, np.uint32,
  • np.int64, np.uint64,
  • np.int_, np.intp

  The specification is by precision rather than by C type. Hence, on some platforms np.int64 may be a long instead of long long even if the specified dtype is long long because the two may have the same precision. The resulting type depends on which C type numpy uses for the given precision. The byteorder specification is also ignored, the generated arrays are always in native byte order.

• A new np.moveaxis function allows for moving one or more array axes to a new position by explicitly providing source and destination axes. This function should be easier to use than the current rollaxis function as well as providing more functionality.

• The deg parameter of the various numpy.polynomial fits has been extended to accept a list of the degrees of the terms to be included in the fit, the coefficients of all other terms being constrained to zero. The change is backward compatible, passing a scalar deg will behave as before.

• A divmod function for float types modeled after the Python version has been added to the npy_math library.

Improvements

np.gradient now supports an axis argument

The axis parameter was added to np.gradient for consistency. It allows to specify over which axes the gradient is calculated.

np.lexsort now supports arrays with object data-type

The function now internally calls the generic npy_amergesort when the type does not implement a merge-sort kind of argsort method.

np.ma.core.MaskedArray now supports an order argument

When constructing a new MaskedArray instance, it can be configured with an order argument analogous to the one when calling np.ndarray. The addition of this argument allows for the proper processing of an order argument in several MaskedArray-related utility functions such as np.ma.core.array and np.ma.core.asarray.

Memory and speed improvements for masked arrays

Creating a masked array with mask=True (resp. mask=False) now uses np.ones (resp. np.zeros) to create the mask, which is faster and avoids a big memory peak. Another optimization was done to avoid a memory peak and useless computations when printing a masked array.

ndarray.tofile now uses fallocate on linux

The function now uses the fallocate system call to reserve sufficient disk space on file systems that support it.

Optimizations for operations of the form A.T @ A and A @ A.T

Previously, gemm BLAS operations were used for all matrix products. Now, if the matrix product is between a matrix and its transpose, it will use syrk BLAS operations for a performance boost. This optimization has been extended to @, numpy.dot, numpy.inner, and numpy.matmul.

Note: Requires the transposed and non-transposed matrices to share data.

np.testing.assert_warns can now be used as a context manager

This matches the behavior of assert_raises.

Speed improvement for np.random.shuffle

np.random.shuffle is now much faster for 1d ndarrays.

Changes

Pyrex support was removed from numpy.distutils

The method build_src.generate_a_pyrex_source will remain available; it has been monkeypatched by users to support Cython instead of Pyrex. It's recommended to switch to a better supported method of build Cython extensions though.

np.broadcast can now be called with a single argument

The resulting object in that case will simply mimic iteration over a single array. This change obsoletes distinctions like

if len(x) == 1:
    shape = x[0].shape
else:
    shape = np.broadcast(\*x).shape

Instead, np.broadcast can be used in all cases.

np.trace now respects array subclasses

This behavior mimics that of other functions such as np.diagonal and ensures, e.g., that for masked arrays np.trace(ma) and ma.trace() give the same result.

np.dot now raises TypeError instead of ValueError

This behavior is now consistent with other functions such as np.inner. If the two arguments cannot be cast to a common type, it could have raised a TypeError or ValueError depending on their order. Now, np.dot will now always raise a TypeError.

linalg.norm return type changes

The linalg.norm function now does all its computations in floating point and returns floating results. This change fixes bugs due to integer overflow and the failure of abs with signed integers of minimum value, e.g., int8(-128). For consistency, floats are used even where an integer might work.

Deprecations

Views of arrays in Fortran order

The F_CONTIGUOUS flag was used to signal that views using a dtype that changed the element size would change the first index. This was always problematical for arrays that were both F_CONTIGUOUS and C_CONTIGUOUS because C_CONTIGUOUS took precedence. Relaxed stride checking results in more such dual contiguous arrays and breaks some existing code as a result. Note that this also affects changing the dtype by assigning to the dtype attribute of an array. The aim of this deprecation is to restrict views to C_CONTIGUOUS arrays at some future time. A work around that is backward compatible is to use a.T.view(...).T instead. A parameter may also be added to the view method to explicitly ask for Fortran order views, but that will not be backward compatible.

Invalid arguments for array ordering

It is currently possible to pass in arguments for the order parameter in methods like array.flatten or array.ravel that were not one of the following: 'C', 'F', 'A', 'K' (note that all of these possible values are both unicode and case insensitive). Such behavior will not be allowed in future releases.

Random number generator in the testing namespace

The Python standard library random number generator was previously exposed in the testing namespace as testing.rand. Using this generator is not recommended and it will be removed in a future release. Use generators from numpy.random namespace instead.

Random integer generation on a closed interval

In accordance with the Python C API, which gives preference to the half-open interval over the closed one, np.random.random_integers is being deprecated in favor of calling np.random.randint, which has been enhanced with the dtype parameter as described under "New Features". However, np.random.random_integers will not be removed anytime soon.

FutureWarnings

Assigning to slices/views of MaskedArray

Currently a slice of a masked array contains a view of the original data and a copy-on-write view of the mask. Consequently, any changes to the slice's mask will result in a copy of the original mask being made and that new mask being changed rather than the original. For example, if we make a slice of the original like so, view = original[:], then modifications to the data in one array will affect the data of the other but, because the mask will be copied during assignment operations, changes to the mask will remain local. A similar situation occurs when explicitly constructing a masked array using MaskedArray(data, mask), the returned array will contain a view of data but the mask will be a copy-on-write view of mask.

In the future, these cases will be normalized so that the data and mask arrays are treated the same way and modifications to either will propagate between views. In 1.11, numpy will issue a MaskedArrayFutureWarning warning whenever user code modifies the mask of a view that in the future may cause values to propagate back to the original. To silence these warnings and make your code robust against the upcoming changes, you have two options: if you want to keep the current behavior, call masked_view.unshare_mask() before modifying the mask. If you want to get the future behavior early, use masked_view._sharedmask = False. However, note that setting the _sharedmask attribute will break following explicit calls to masked_view.unshare_mask().