.. _whatsnew_changes_7.7:

%%%%%%%%%%%%%%%%%%%%%%%%%%%% Changes in Varnish-Cache 7.7 %%%%%%%%%%%%%%%%%%%%%%%%%%%%

For information about updating your current Varnish deployment to the new version, see :ref:whatsnew_upgrading_7.7.

NOTE: In this Varnish-Cache release, we changed how timestamps are taken for the http2 protocol, which could look like a performance regression, but is not. See :ref:whatsnew_changes_7.7_h2_timestamps.

A more detailed and technical account of changes in Varnish-Cache, with links to issues that have been fixed and pull requests that have been merged, may be found in the change log_.

.. _change log: https://github.com/varnishcache/varnish-cache/blob/master/doc/changes.rst

Security

VSV 15:


The client connection is now always closed when a malformed request
is received.

varnishd
========

Parameters

The bitfield parameters debug, experimental, feature, vcc_feature and vsl_mask now consistently support the special values all and none. The output format of varnishadm param.show has been adjusted to always output the parameter value relative to either all or none in a format which would also be accepted by varnishadm param.set and the varnishd -p option.

The new http_req_overflow_status parameter now allows to optionally send a response with a status between 400 and 499 (inclusive) if a request exceeds http_req_size. The default of 0 keeps the existing behavior to just close the connection in this case.

The new ban_any_variant parameter allows to configure the maximum number of possibly non matching variants evaluated against the ban list during lookup. The default value of 10000 avoids excessive time spent for ban checks during lookups, which could cause noticeable delays for cases with a very high number of bans and/or variants (in the 1000s).

Setting ban_any_variant to 0 changes the behavior of the lookup-time ban check to only consider matching objects for tests against the ban list, which can be considered a bugfix, depending on the exact interpretation of the semantics of ban expressions with regards to variants. 0 will become the new default in a future release of Varnish-Cache.

The varnishd parameters max_restarts and max_retries have been made more consistent, which prevents a potential panic that could be triggered when reducing the value of max_restarts under certain conditions.

Jails


The ``linux`` jail gained control of transparent huge pages (THP) settings: The
``transparent_hugepage`` suboption can be set to ``ignore`` to do nothing,
``enable`` to enable THP (actually, disable the disable), ``disable`` to disable
THP or ``try-disable`` to try do disable, but not emit an error if disabling
fails. ``try-disable`` is the default.

Error handling from the jail subsystem has been streamlined to avoid some
confusing and/or contradictory error messages as well as turn assertion failures
into error messages.

Other changes in varnishd

An issue has been fixed which could cause a crash when varnishd receives an invalid Content-Range header from a backend.

The management process now logs the PID of the process from which it received a signal.

Changes to VCL

VCL variables


Two new VCL variables ``req.filters`` and ``bereq.filters`` can now be used to
pass request bodies through a list of VFPs and backend request bodies through a
list of VDPs respectively. This can be useful for processing or transforming
request bodies as they go through varnish. See :ref:`vcl-var(7)` for more
details.

VCL now supports ``unset req.grace`` and ``unset req.ttl`` to reset the
respective variables to the "no effect" value, which is also the default.

The scope of VCL variables ``req.is_hitmiss`` and ``req.is_hitpass`` is now
restricted to ``vcl_miss, vcl_deliver, vcl_pass, vcl_synth`` and ``vcl_pass,
vcl_deliver, vcl_synth``, respectively.

The ``Content-Length`` header is now consistently removed after ``unset
bereq.body`` on the backend side.

``req.hash`` is now also readable from ``vcl_synth`` and ``vcl_pipe``.

Other changes to VCL

Behavior of the VCL include statement with the +glob option has been clarified to not search directories in vcl_path. Using +glob includes with a relative path that does not start with "./" will now result in a VCL compile failure.

Validation of the PROXY2 PP2_TYPE_AUTHORITY TLV sent with .via backends has been corrected: IP addresses are no longer accepted as an authority and port numbers are automatically removed.

return (fail(...)) can now take strings returned from a vmod.

varnishadm

The CLI command backend.list -j now outputs IPs/port information.

Generic Logging (VSL)

affecting varnishlog, varnishncsa and varnishtop:

.. _whatsnew_changes_7.7_h2_timestamps:

http2 related timestamps


Timestamps for http/2 requests have been corrected and made similar to how they
are taken for http/1.

For http/1, the start time, internally called "t_first", is taken as soon as any
part of the request (headers) is received. Previously, http/2 took it later,
possibly much later if long header lines were involved. http/2 now takes it the
same way as http/1 when the first bit of the first HEADERS frame of the request
arrives.

Timing behavior for http/1 and http/2 is different and can not be directly
compared. But with this change, the ``Timestamp`` VSL records for http/2 now at
least reflect reality better.

**NOTE** that after upgrading Varnish-Cache, processing and response times for
http/2 will now be reported as worse than before the upgrade, potentially *much*
worse. This is **NOT** a performance regression, but rather due to the corrected
timestamps, which arguably were wrong for http/2.

http2 logging
~~~~~~~~~~~~~

For http/2, normal client behavior like timeouts or closed connection was logged
with a ``SessError`` tag and ``ENHANCE_YOUR_CALM`` in additional ``Debug`` log
records. This behavior was misleading and has been corrected.

http/2 error detail reporting in ``Debug`` log records has been clarified:
Connection errors are now prefixed with ``H2CE_``, and stream errors with
``H2SE_``, respectively.

http/2 ``BogoHeader`` log records now contain the first offending byte value in
hex.

Interactive mode in varnishstat, varnishtop and varnishhist
===========================================================

Handling of curses errors in the interactive mode of ``varnishstat``,
``varnishtop`` and ``varnishhist`` has been streamlined and one wrong assertion
has been fixed, which could cause a crash with certain terminal types as set
through the ``TERM`` environment variable.

varnishncsa
===========

The ``hitmiss`` and ``hitpass`` handling indicators have been added to the
``Varnish:handling`` format of ``varnishncsa``.

``varnishncsa`` now handles headers unset and changed from VCL more
consistently: request headers are logged as they were received from the client
and as they were sent to the backend, while response headers are logged as they
were sent to the client and as they were received from the backend.

varnishstat
===========

Pressing the ``0`` key in ``varnishstat`` interactive (curses) mode now resets
averages.

The backend ``happy`` VSC bitfield is now set to all ones for backends with no
configured probe.

varnishtest
===========

``varnishtest`` can now send arbitrary http/2 settings frames and arbitrary
PROXY2 tlvs.

``varnishtest`` has been changed to always set a ``VARNISH_DEFAULT_N``
environment variable to ensure that ``varnish`` invoked from ``varnishtest``
always has a valid workdir.

Changes for developers and VMOD authors
=======================================

``miniobj.h``: Helper macros ``SIZEOF_FLEX_OBJ()`` and ``ALLOC_FLEX_OBJ()`` have
been added to facilitate use of structs with flexible array members.

The acceptor code has been refactored for basic support of pluggable acceptors.

Two fields have been added to the VMOD data registered with varnish-cache:

- ``vcs`` for Version Control System is intended as an identifier from the
  source code management system, e.g. the git revision, to identify the exact
  source code which was used to build a VMOD binary.

- ``version`` is intended as a more user friendly identifier as to which
  version of a vmod a binary represents.

The panic output and the ``debug.vmod`` CLI command output now contain these
identifiers.

Where supported by the compiler and linker, the ``vcs`` identifier is also
reachable via the ``.vmod_vcs`` section of the vmod shared object ELF file and
can be extracted, for example, using ``readelf -p.vmod_vcs <file>``

To set the version, ``vmodtool.py`` now accepts a ``$Version`` stanza in vmod
vcc files. If ``$Version`` is not present, an attempt is made to extract
``PACKAGE_STRING`` from an automake ``Makefile``, otherwise ``NOVERSION`` is
used as the version identifier.

A new facility has been added allowing transport delivery functions to disembark
the worker thread which had been handling a request's VCL code during delivery
by returning ``VTR_D_DISEMBARK`` from the ``vtr_deliver_f`` function.

This will enable future optimizations to make transport protocol code more
efficient.

To enable this facility, a new request processing step ``finish`` has been added
once delivery is complete.

*eof*