.. currentmodule:: trio.abc

.. http://docutils.sourceforge.net/docs/ref/rst/directives.html#list-table

.. list-table:: Overview: abstract base classes for I/O
   :widths: auto
   :header-rows: 1

   * - Abstract base class
     - Inherits from...
     - Adds these abstract methods...
     - And these concrete methods.
     - Example implementations
   * - :class:`AsyncResource`
     -
     - :meth:`~AsyncResource.aclose`
     - ``__aenter__``, ``__aexit__``
     - :ref:`async-file-objects`
   * - :class:`SendStream`
     - :class:`AsyncResource`
     - :meth:`~SendStream.send_all`,
       :meth:`~SendStream.wait_send_all_might_not_block`
     -
     - :class:`~trio.testing.MemorySendStream`
   * - :class:`ReceiveStream`
     - :class:`AsyncResource`
     - :meth:`~ReceiveStream.receive_some`
     - ``__aiter__``, ``__anext__``
     - :class:`~trio.testing.MemoryReceiveStream`
   * - :class:`Stream`
     - :class:`SendStream`, :class:`ReceiveStream`
     -
     -
     - :class:`~trio.SSLStream`
   * - :class:`HalfCloseableStream`
     - :class:`Stream`
     - :meth:`~HalfCloseableStream.send_eof`
     -
     - :class:`~trio.SocketStream`, :class:`~trio.StapledStream`
   * - :class:`Listener`
     - :class:`AsyncResource`
     - :meth:`~Listener.accept`
     -
     - :class:`~trio.SocketListener`, :class:`~trio.SSLListener`
   * - :class:`SendChannel`
     - :class:`AsyncResource`
     - :meth:`~SendChannel.send`
     -
     - `~trio.MemorySendChannel`
   * - :class:`ReceiveChannel`
     - :class:`AsyncResource`
     - :meth:`~ReceiveChannel.receive`
     - ``__aiter__``, ``__anext__``
     - `~trio.MemoryReceiveChannel`
   * - `Channel`
     - `SendChannel`, `ReceiveChannel`
     -
     -
     -

.. autoclass:: trio.abc.AsyncResource
   :members:

.. currentmodule:: trio

.. autofunction:: aclose_forcefully

.. currentmodule:: trio.abc

.. autoclass:: trio.abc.SendStream
   :members:
   :show-inheritance:

.. autoclass:: trio.abc.ReceiveStream
   :members:
   :show-inheritance:

.. autoclass:: trio.abc.Stream
   :members:
   :show-inheritance:

.. autoclass:: trio.abc.HalfCloseableStream
   :members:
   :show-inheritance:

.. currentmodule:: trio.abc

.. autoclass:: trio.abc.Listener
   :members:
   :show-inheritance:

.. autoclass:: trio.abc.SendChannel
   :members:
   :show-inheritance:

.. autoclass:: trio.abc.ReceiveChannel
   :members:
   :show-inheritance:

.. autoclass:: trio.abc.Channel
   :members:
   :show-inheritance:

.. currentmodule:: trio


Generic stream tools
~~~~~~~~~~~~~~~~~~~~

Trio currently provides a generic helper for writing servers that
listen for connections using one or more
:class:`~trio.abc.Listener`\s, and a generic utility class for working
with streams. And if you want to test code that's written against the
streams interface, you should also check out :ref:`testing-streams` in
:mod:`trio.testing`.

.. autofunction:: serve_listeners

.. autoclass:: StapledStream
   :members:
   :show-inheritance:


.. _high-level-networking:

Sockets and networking

Trio provides SSL/TLS support based on the standard library :mod:`ssl`
module. Trio's :class:`SSLStream` and :class:`SSLListener` take their
configuration from a :class:`ssl.SSLContext`, which you can create
using :func:`ssl.create_default_context` and customize using the
other constants and functions in the :mod:`ssl` module.

.. warning:: Avoid instantiating :class:`ssl.SSLContext` directly.
   A newly constructed :class:`~ssl.SSLContext` has less secure
   defaults than one returned by :func:`ssl.create_default_context`.

Instead of using :meth:`ssl.SSLContext.wrap_socket`, you
create a :class:`SSLStream`:

.. autoclass:: SSLStream
   :show-inheritance:
   :members:

And if you're implementing a server, you can use :class:`SSLListener`:

.. autoclass:: SSLListener
   :show-inheritance:
   :members:

Some methods on :class:`SSLStream` raise :exc:`NeedHandshakeError` if
you call them before the handshake completes:

.. autoexception:: NeedHandshakeError


Datagram TLS support

Generally, the API exposed by :mod:`trio.socket` mirrors that of the
standard library :mod:`socket` module. Most constants (like
``SOL_SOCKET``) and simple utilities (like :func:`~socket.inet_aton`)
are simply re-exported unchanged. But there are also some differences,
which are described here.

First, Trio provides analogues to all the standard library functions
that return socket objects; their interface is identical, except that
they're modified to return Trio socket objects instead:

.. autofunction:: socket

.. autofunction:: socketpair

.. autofunction:: fromfd

.. function:: fromshare(data)

   Like :func:`socket.fromshare`, but returns a Trio socket object.

In addition, there is a new function to directly convert a standard
library socket into a Trio socket:

.. autofunction:: from_stdlib_socket

Unlike :class:`socket.socket`, :func:`trio.socket.socket` is a
function, not a class; if you want to check whether an object is a
Trio socket, use ``isinstance(obj, trio.socket.SocketType)``.

For name lookup, Trio provides the standard functions, but with some
changes:

.. autofunction:: getaddrinfo

.. autofunction:: getnameinfo

.. autofunction:: getprotobyname

Trio intentionally DOES NOT include some obsolete, redundant, or
broken features:

* :func:`~socket.gethostbyname`, :func:`~socket.gethostbyname_ex`,
  :func:`~socket.gethostbyaddr`: obsolete; use
  :func:`~socket.getaddrinfo` and :func:`~socket.getnameinfo` instead.

* :func:`~socket.getservbyport`: obsolete and `buggy
  <https://bugs.python.org/issue30482>`__; instead, do:

  .. code-block:: python

     _, service_name = await getnameinfo(('127.0.0.1', port), NI_NUMERICHOST)

* :func:`~socket.getservbyname`: obsolete and `buggy
  <https://bugs.python.org/issue30482>`__; instead, do:

  .. code-block:: python

     await getaddrinfo(None, service_name)

* :func:`~socket.getfqdn`: obsolete; use :func:`getaddrinfo` with the
  ``AI_CANONNAME`` flag.

* :func:`~socket.getdefaulttimeout`,
  :func:`~socket.setdefaulttimeout`: instead, use Trio's standard
  support for :ref:`cancellation`.

* On Windows, ``SO_REUSEADDR`` is not exported, because it's a trap:
  the name is the same as Unix ``SO_REUSEADDR``, but the semantics are
  `different and extremely broken
  <https://msdn.microsoft.com/en-us/library/windows/desktop/ms740621(v=vs.85).aspx>`__. In
  the very rare cases where you actually want ``SO_REUSEADDR`` on
  Windows, then it can still be accessed from the standard library's
  :mod:`socket` module.


Socket objects
~~~~~~~~~~~~~~

.. class:: SocketType

   .. note:: :class:`trio.socket.SocketType` is an abstract class and
      cannot be instantiated directly; you get concrete socket objects
      by calling constructors like :func:`trio.socket.socket`.
      However, you can use it to check if an object is a Trio socket
      via ``isinstance(obj, trio.socket.SocketType)``.

   Trio socket objects are overall very similar to the :ref:`standard
   library socket objects <python:socket-objects>`, with a few
   important differences:

   First, and most obviously, everything is made "Trio-style":
   blocking methods become async methods, and the following attributes
   are *not* supported:

   * :meth:`~socket.socket.setblocking`: Trio sockets always act like
     blocking sockets; if you need to read/write from multiple sockets
     at once, then create multiple tasks.
   * :meth:`~socket.socket.settimeout`: see :ref:`cancellation` instead.
   * :meth:`~socket.socket.makefile`: Python's file-like API is
     synchronous, so it can't be implemented on top of an async
     socket.
   * :meth:`~socket.socket.sendall`: Could be supported, but you're
     better off using the higher-level
     :class:`~trio.SocketStream`, and specifically its
     :meth:`~trio.SocketStream.send_all` method, which also does
     additional error checking.

   In addition, the following methods are similar to the equivalents
   in :class:`socket.socket`, but have some Trio-specific quirks:

   .. method:: connect
      :async:

      Connect the socket to a remote address.

      Similar to :meth:`socket.socket.connect`, except async.

      .. warning::

         Due to limitations of the underlying operating system APIs, it is
         not always possible to properly cancel a connection attempt once it
         has begun. If :meth:`connect` is cancelled, and is unable to
         abort the connection attempt, then it will:

         1. forcibly close the socket to prevent accidental reuse
         2. raise :exc:`~trio.Cancelled`.

         tl;dr: if :meth:`connect` is cancelled then the socket is
         left in an unknown state – possibly open, and possibly
         closed. The only reasonable thing to do is to close it.

   .. method:: is_readable

      Check whether the socket is readable or not.

   .. method:: sendfile

      `Not implemented yet! <https://github.com/python-trio/trio/issues/45>`__

   We also keep track of an extra bit of state, because it turns out
   to be useful for :class:`trio.SocketStream`:

   .. attribute:: did_shutdown_SHUT_WR

      This :class:`bool` attribute is True if you've called
      ``sock.shutdown(SHUT_WR)`` or ``sock.shutdown(SHUT_RDWR)``, and
      False otherwise.

   The following methods are identical to their equivalents in
   :class:`socket.socket`, except async, and the ones that take address
   arguments require pre-resolved addresses:

   * :meth:`~socket.socket.accept`
   * :meth:`~socket.socket.bind`
   * :meth:`~socket.socket.recv`
   * :meth:`~socket.socket.recv_into`
   * :meth:`~socket.socket.recvfrom`
   * :meth:`~socket.socket.recvfrom_into`
   * :meth:`~socket.socket.recvmsg` (if available)
   * :meth:`~socket.socket.recvmsg_into` (if available)
   * :meth:`~socket.socket.send`
   * :meth:`~socket.socket.sendto`
   * :meth:`~socket.socket.sendmsg` (if available)

   All methods and attributes *not* mentioned above are identical to
   their equivalents in :class:`socket.socket`:

   * :attr:`~socket.socket.family`
   * :attr:`~socket.socket.type`
   * :attr:`~socket.socket.proto`
   * :meth:`~socket.socket.fileno`
   * :meth:`~socket.socket.listen`
   * :meth:`~socket.socket.getpeername`
   * :meth:`~socket.socket.getsockname`
   * :meth:`~socket.socket.close`
   * :meth:`~socket.socket.shutdown`
   * :meth:`~socket.socket.setsockopt`
   * :meth:`~socket.socket.getsockopt`
   * :meth:`~socket.socket.dup`
   * :meth:`~socket.socket.detach`
   * :meth:`~socket.socket.share`
   * :meth:`~socket.socket.set_inheritable`
   * :meth:`~socket.socket.get_inheritable`


.. currentmodule:: trio


.. _async-file-io:

Asynchronous filesystem I/O
---------------------------

Trio provides built-in facilities for performing asynchronous
filesystem operations like reading or renaming a file. Generally, we
recommend that you use these instead of Python's normal synchronous
file APIs. But the tradeoffs here are somewhat subtle: sometimes
people switch to async I/O, and then they're surprised and confused
when they find it doesn't speed up their program. The next section
explains the theory behind async file I/O, to help you better
understand your code's behavior. Or, if you just want to get started,
you can :ref:`jump down to the API overview <async-file-io-overview>`.


Background: Why is async file I/O useful? The answer may surprise you

If you want to perform general filesystem operations like creating and
listing directories, renaming files, or checking file metadata – or if
you just want a friendly way to work with filesystem paths – then you
want :class:`trio.Path`. It's an asyncified replacement for the
standard library's :class:`pathlib.Path`, and provides the same
comprehensive set of operations.

For reading and writing to files and file-like objects, Trio also
provides a mechanism for wrapping any synchronous file-like object
into an asynchronous interface. If you have a :class:`trio.Path`
object you can get one of these by calling its :meth:`~trio.Path.open`
method; or if you know the file's name you can open it directly with
:func:`trio.open_file`. Alternatively, if you already have an open
file-like object, you can wrap it with :func:`trio.wrap_file` – one
case where this is especially useful is to wrap :class:`io.BytesIO` or
:class:`io.StringIO` when writing tests.


Asynchronous path objects

.. Suppress type annotations here, they refer to lots of internal types.
   The normal Python docs go into better detail.
.. autofunction:: open_file(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=None, opener=None)

.. autofunction:: wrap_file(file)

.. interface:: Asynchronous file interface

   Trio's asynchronous file objects have an interface that
   automatically adapts to the object being wrapped. Intuitively, you
   can mostly treat them like a regular :term:`file object`, except
   adding an ``await`` in front of any of methods that do I/O. The
   definition of :term:`file object` is a little vague in Python
   though, so here are the details:

   * Synchronous attributes/methods: if any of the following
     attributes or methods are present, then they're re-exported
     unchanged: ``closed``, ``encoding``, ``errors``, ``fileno``,
     ``isatty``, ``newlines``, ``readable``, ``seekable``,
     ``writable``, ``buffer``, ``raw``, ``line_buffering``,
     ``closefd``, ``name``, ``mode``, ``getvalue``, ``getbuffer``.

   * Async methods: if any of the following methods are present, then
     they're re-exported as an async method: ``flush``, ``read``,
     ``read1``, ``readall``, ``readinto``, ``readline``,
     ``readlines``, ``seek``, ``tell``, ``truncate``, ``write``,
     ``writelines``, ``readinto1``, ``peek``, ``detach``.

   Special notes:

   * Async file objects implement Trio's
     :class:`~trio.abc.AsyncResource` interface: you close them by
     calling :meth:`~trio.abc.AsyncResource.aclose` instead of
     ``close`` (!!), and they can be used as async context
     managers. Like all :meth:`~trio.abc.AsyncResource.aclose`
     methods, the ``aclose`` method on async file objects is
     guaranteed to close the file before returning, even if it is
     cancelled or otherwise raises an error.

   * Using the same async file object from multiple tasks
     simultaneously: because the async methods on async file objects
     are implemented using threads, it's only safe to call two of them
     at the same time from different tasks IF the underlying
     synchronous file object is thread-safe. You should consult the
     documentation for the object you're wrapping. For objects
     returned from :func:`trio.open_file` or :meth:`trio.Path.open`,
     it depends on whether you open the file in binary mode or text
     mode: `binary mode files are task-safe/thread-safe, text mode
     files are not
     <https://docs.python.org/3/library/io.html#multi-threading>`__.

   * Async file objects can be used as async iterators to iterate over
     the lines of the file:

     .. code-block:: python

        async with await trio.open_file(...) as f:
            async for line in f:
                print(line)

   * The ``detach`` method, if present, returns an async file object.

   This should include all the attributes exposed by classes in
   :mod:`io`. But if you're wrapping an object that has other
   attributes that aren't on the list above, then you can access them
   via the ``.wrapped`` attribute:

   .. attribute:: wrapped

      The underlying synchronous file object.


.. _subprocess:

Spawning subprocesses
---------------------

Trio provides support for spawning other programs as subprocesses,
communicating with them via pipes, sending them signals, and waiting
for them to exit.

Most of the time, this is done through our high-level interface,
`trio.run_process`. It lets you either run a process to completion
while optionally capturing the output, or else run it in a background
task and interact with it while it's running:

.. autofunction:: trio.run_process

.. autoclass:: trio._subprocess.HasFileno(Protocol)

   .. automethod:: fileno

.. autoclass:: trio._subprocess.StrOrBytesPath

.. autoclass:: trio.Process()

   .. autoattribute:: returncode

   .. automethod:: wait

   .. automethod:: poll

   .. automethod:: kill

   .. automethod:: terminate

   .. automethod:: send_signal

   .. note:: :meth:`~subprocess.Popen.communicate` is not provided as a
      method on :class:`~trio.Process` objects; call :func:`~trio.run_process`
      normally for simple capturing, or write the loop yourself if you
      have unusual needs. :meth:`~subprocess.Popen.communicate` has
      quite unusual cancellation behavior in the standard library (on
      some platforms it spawns a background thread which continues to
      read from the child process even after the timeout has expired)
      and we wanted to provide an interface with fewer surprises.

If `trio.run_process` is too limiting, we also offer a low-level API,
`trio.lowlevel.open_process`. For example, if you want to spawn a
child process that will outlive the parent process and be
orphaned, then `~trio.run_process` can't do that, but
`~trio.lowlevel.open_process` can.


.. _subprocess-options:

Options for starting subprocesses

The command to run and its arguments usually must be passed to Trio's
subprocess APIs as a sequence of strings, where the first element in
the sequence specifies the command to run and the remaining elements
specify its arguments, one argument per element. This form is used
because it avoids potential quoting pitfalls; for example, you can run
``["cp", "-f", source_file, dest_file]`` without worrying about
whether ``source_file`` or ``dest_file`` contains spaces.

If you only run subprocesses without ``shell=True`` and on UNIX,
that's all you need to know about specifying the command. If you use
``shell=True`` or run on Windows, you probably should read the
rest of this section to be aware of potential pitfalls.

With ``shell=True`` on UNIX, you must specify the command as a single
string, which will be passed to the shell as if you'd entered it at an
interactive prompt. The advantage of this option is that it lets you
use shell features like pipes and redirection without writing code to
handle them. For example, you can write ``Process("ls | grep
some_string", shell=True)``.  The disadvantage is that you must
account for the shell's quoting rules, generally by wrapping in
:func:`shlex.quote` any argument that might contain spaces, quotes, or
other shell metacharacters.  If you don't do that, your safe-looking
``f"ls | grep {some_string}"`` might end in disaster when invoked with
``some_string = "foo; rm -rf /"``.

On Windows, the fundamental API for process spawning (the
``CreateProcess()`` system call) takes a string, not a list, and it's
actually up to the child process to decide how it wants to split that
string into individual arguments. Since the C language specifies that
``main()`` should take a list of arguments, *most* programs you
encounter will follow the rules used by the Microsoft C/C++ runtime.
:class:`subprocess.Popen`, and thus also Trio, uses these rules
when it converts an argument sequence to a string, and they
are `documented
<https://docs.python.org/3/library/subprocess.html#converting-argument-sequence>`__
alongside the :mod:`subprocess` module. There is no documented
Python standard library function that can directly perform that
conversion, so even on Windows, you almost always want to pass an
argument sequence rather than a string. But if the program you're
spawning doesn't split its command line back into individual arguments
in the standard way, you might need to pass a string to work around this.
(Or you might just be out of luck: as far as I can tell, there's simply
no way to pass an argument containing a double-quote to a Windows
batch file.)

On Windows with ``shell=True``, things get even more chaotic. Now
there are two separate sets of quoting rules applied, one by the
Windows command shell ``CMD.EXE`` and one by the process being
spawned, and they're *different*. (And there's no :func:`shlex.quote`
to save you: it uses UNIX-style quoting rules, even on Windows.)  Most
special characters interpreted by the shell ``&<>()^|`` are not
treated as special if the shell thinks they're inside double quotes,
but ``%FOO%`` environment variable substitutions still are, and the
shell doesn't provide any way to write a double quote inside a
double-quoted string. Outside double quotes, any character (including
a double quote) can be escaped using a leading ``^``.  But since a
pipeline is processed by running each command in the pipeline in a
subshell, multiple layers of escaping can be needed:

.. code-block:: sh

    echo ^^^&x | find "x" | find "x"          # prints: &x

And if you combine pipelines with () grouping, you can need even more
levels of escaping:

.. code-block:: sh

    (echo ^^^^^^^&x | find "x") | find "x"    # prints: &x

Since process creation takes a single arguments string, ``CMD.EXE``\'s
quoting does not influence word splitting, and double quotes are not
removed during CMD.EXE's expansion pass. Double quotes are troublesome
because CMD.EXE handles them differently from the MSVC runtime rules; in:

.. code-block:: sh

    prog.exe "foo \"bar\" baz"

the program will see one argument ``foo "bar" baz`` but CMD.EXE thinks
``bar\`` is not quoted while ``foo \`` and ``baz`` are. All of this
makes it a formidable task to reliably interpolate anything into a
``shell=True`` command line on Windows, and Trio falls back on the
:mod:`subprocess` behavior: If you pass a sequence with
``shell=True``, it's quoted in the same way as a sequence with
``shell=False``, and had better not contain any shell metacharacters
you weren't planning on.

Further reading:

* https://stackoverflow.com/questions/30620876/how-to-properly-escape-filenames-in-windows-cmd-exe

* https://stackoverflow.com/questions/4094699/how-does-the-windows-command-interpreter-cmd-exe-parse-scripts


Signals
-------

.. currentmodule:: trio

.. autofunction:: open_signal_receiver
   :with: signal_aiter