release notes/v0.31.0.md
k6 v0.31.0 is here! :tada: It's a smaller release with some significant performance improvements, a new http_req_failed metric and changes to the output subsystem that enable output extensions with xk6!
The state of k6's output packages has been a development pain point for a long time, which made it difficult to add new outputs in a consistent way. In the refactor done in v0.31.0, this has been mostly addressed and outputs now implement a simpler and cleaner Output interface.
In addition to this, it is now possible to implement custom k6 output extensions in Go with xk6! This is very useful if you use a system that's currently not supported by the built-in outputs, or need some custom handling of the metrics k6 produces.
Writing output extensions is done very similarly to how JS module extensions are currently written, though instead of calling js/modules.Register(), you should implement the new Output interface and call output.RegisterExtension() with your constructor.
We are working on the proper documentation for this, as well as the overdue xk6 documentation about JS extensions, so keep a lookout for those on k6.io/docs.
It's now possible to declare expected HTTP response statuses for either the entire test or for individual HTTP requests, and k6 will emit a new http_req_failed metric as well as tag HTTP metrics with expected_response: <bool>. By default, k6 will now fail requests that return HTTP 4xx/5xx response codes.
For example:
import http from 'k6/http';
// Set expected statuses globally for all requests.
http.setResponseCallback(http.expectedStatuses({min: 200, max: 399}, 418));
export default function () {
// This request will be marked as failed.
http.get('https://httpbin.test.k6.io/status/400');
// This request will be considered as "passed" because of the responseCallback override.
http.get('https://httpbin.test.k6.io/status/400', { responseCallback: http.expectedStatuses(400) });
}
Running this script will produce a summary like:
http_req_duration..............: avg=204.57ms min=203.31ms med=204.57ms max=205.82ms p(90)=205.57ms p(95)=205.7ms
{ expected_response:true }...: avg=203.31ms min=203.31ms med=203.31ms max=203.31ms p(90)=203.31ms p(95)=203.31ms
http_req_failed................: 50.00% ✓ 1 ✗ 1
Note the new http_req_duration sub-metric for expected responses only, and the new http_req_failed Rate metric. This new metric and metric tag have many potential use cases, and one of the most important ones is the ability to set better thresholds. For example:
'http_req_failed': ['rate<0.1'], i.e. fail the test if more than 10% of requests fail.'http_req_duration{expected_response:true}': ['p(95)<300', 'p(99.9)<500'] - fail the test if the 95th percentile HTTP request duration is above 300ms or the 99.9th percentile is above 500ms; specifying expected_response:true here may be important, because a lot of times failed requests may return more quickly than normal ones, thus skewing the results and wrongly satisfying the threshold.If the response callback is not specified, the default expected statuses will be {min: 200, max: 399}. The previous behavior of not emitting anything can be achieved by setting the callback to null, i.e. http.setResponseCallback(null). Additionally, the expected_response tag can be disabled by removing it from the default list of system tags, e.g. k6 run --system-tags 'proto,subproto,status,method,url,name,group,check,error,error_code,tls_version,scenario,service'.
The http.setResponseCallback() is planned to allow arbitrary JS functions to process responses in the future, but for now only the http.expectedStatuses() callback is supported.
--compatibility-mode=extended. So in v0.31.0 core.js has been dropped entirely, yielding some significant CPU and memory usage improvements. The actual numbers will depend on the use case, but for simple tests users can expect a memory drop of about 2MB per VU (from ~2.7MB to ~600KB), and a slight CPU decrease of about 5-10%. For more complex tests with a lot of JS code this benefit won't be as pronounced. Another benefit of this change is that initializing VUs and starting a test is substantially faster than before! (#1824)ArrayBuffer support in most internal modules, so now you can pass ArrayBuffer to http.file(), in k6/encoding and k6/crypto functions. This makes working with binary files more efficient as it doesn't require string translations. In upcoming versions we plan to expand this to the WebSocket module, as well as make some potentially breaking changes for APIs that currently return an array of integers or string (see the details in the Breaking Changes announcement below). (#1800)ca-certificates as a dependency. Thanks @Bablzz! (#1854)^C) will now properly propagate to any used outputs. (#1869)ext.loadimpact.name JS option or config, or the K6_CLOUD_NAME environment variable. (#1870)SharedArray introduced in v0.30.0 can now be iterated with forEach. (#1848)SharedArray was rewritten using goja.DynamicArray making it more performant and easier to reason about. (#1848)Promise is now undefined, and some unused Babel plugins like transform-es2015-for-of and transform-regenerator were also removed. This means that some workarounds like the ones mentioned here and here also won't work as is and will need additional polyfills and plugins to work properly.The following are not breaking changes in this release, but we'd like to announce them so users can prepare for them in upcoming releases (likely k6 v0.32.0).
ArrayBuffer changes in this release are backwards compatible and shouldn't cause any issues, but in v0.32.0 some JS APIs that currently return an array of integers or string for binary data will return ArrayBuffer instead. This is the case for open() when used with the 'b' argument, response bodies for requests that specify responseType: 'binary', crypto.randomBytes(), hasher.digest('binary'), and encoding.b64decode(). Response.json() and Response.html() will also probably stop working when used with requests that specify responseType: 'binary'. These changes shouldn't be a problem for most users that were simply using these values to pass them to other internal modules (e.g. opening a binary file and passing it to http.post()), but if the scripts modified the binary data or depended on the current array of integers or string values they will need to be adapted to use typed arrays instead. You can follow the discussion in PR #1841 and issue #1020.