release notes/v0.26.0.md
k6 v0.26.0 is here! :tada:
This release contains mostly bug fixes, though it also has several new features and enhancements! They include a new JS compatibility mode option, exporting the end-of-test summary to a JSON report file, speedups to the InfluxDB and JSON outputs, http.batch() improvements, a brand new CSV output, multiple layered HTTP response body decompression, being able to use console in the init context, a new optional column in the summary, and Docker improvements!
Thanks to @Sirozha1337, @openmohan, @MMartyn, @KajdeMunter, @dmitrytokarev and @dimatock for contributing to this release!
This adds a way to disable the automatic script transformation by Babel (v6.4.2) and loading of core-js (v2) polyfills, bundled in k6. With the new base compatibility mode, k6 will instead rely only on the goja runtime and what is built into k6.
This can be configured through the new --compatibility-mode CLI flag and the K6_COMPATIBILITY_MODE environment variable. The possible values currently are:
extended: this is the default and current compatibility mode, which uses Babel and core.js to achieve ES6+ compatibility.base: an optional mode that disables loading of Babel and core.js, running scripts with only goja's native ES5.1+ compatibility. If the test scripts don't require ES6 compatibility (e.g. they were previously transformed by Babel), this option can be used to reduce RAM usage during test runs.More info what this means can be found in the documentation.
Our benchmarks show a considerable drop in memory usage - around 80% for simple scripts, and around 50% in the case of 2MB script with a lot of static data in it. The CPU usage is mostly unchanged, except that k6 initializes test runs a lot faster. All of those benefits will be most noticeable if k6 is used with big number of VUs (1k+). More performance comparisons can be found in #1167.
This returns (from the very early days of k6) the ability to output the data from the end of test summary in a machine-readable JSON file.
This report can be enabled by the --summary-export <file_path> CLI flag or the K6_SUMMARY_EXPORT environment variable. The resulting JSON file will include data for all test metrics, checks and thresholds.
There is an entirely new csv output that can be enabled by using the --out csv CLI flag. There are two things that can be configured: the output file with K6_CSV_FILENAME (by default it's file.csv), and the interval of pushing metrics to disk, which is configured with K6_CSV_SAVE_INTERVAL (1 second by default). Both of those can be configured by the CLI as well: --out csv=somefile.csv will output to somefile.csv and --out csv=file_name=somefile.csv,save_interval=2s will output again to somefile.csv, but will flush the data every 2 seconds instead of every second.
The first line of the output is the names of columns and looks like:
metric_name,timestamp,metric_value,check,error,error_code,group,method,name,proto,status,subproto,tls_version,url,extra_tags
http_reqs,1573131887,1.000000,,,,,GET,http://httpbin.org/,HTTP/1.1,200,,,http://httpbin.org/,
http_req_duration,1573131887,116.774321,,,,,GET,http://httpbin.org/,HTTP/1.1,200,,,http://httpbin.org/,
http_req_blocked,1573131887,148.691247,,,,,GET,http://httpbin.org/,HTTP/1.1,200,,,http://httpbin.org/,
http_req_connecting,1573131887,112.593448,,,,,GET,http://httpbin.org/,HTTP/1.1,200,,,http://httpbin.org/,
All thanks to @Sirozha1337!
The JSON output no longer blocks the goroutine sending samples to the file, but instead (like all other outputs) buffers the samples and writes them at regular intervals (100ms and is currently not configurable). It also uses a slightly faster way of encoding the data, which should decrease the memory usage by a small amount.
Another improvement is the ability to compress the generated JSON file by simply adding .gz to the end of the file name. Compressed files are typically 30x smaller.
The InfluxDB output has been updated to use less memory and try to send smaller and consistent chunks of data to InfluxDB, in order to not drop packets and be more efficient. This is primarily done by sending data in parallel, as this seems to be better from a performance perspective, and more importantly, queuing data in separate packets, so that we don't send the data for a big time period all at once. Also, the used library was updated, which also decreased the memory usage.
Two new options were added:
K6_INFLUXDB_PUSH_INTERVAL - configures at what interval the collected data is queued to be sent to InfluxDB. By default this is "1s".K6_INFLUXDB_CONCURRENT_WRITES - configures the number of concurrent write calls to InfluxDB. If this limit is reached the next writes will be queued and made when a slot is freed. By default this is 10.console is now available in the init context (#982):This wasn't supported for the longest time, which made debugging things outside of VU code much harder, but now it's here! :tada:
In order to get this feature shipped in a timely manner, it currently has a known bug. The output of console calls in the init context will always be written to the stderr, even if the --console-output option is specified. This bug is tracked in https://github.com/loadimpact/k6/issues/1131
In v0.25.0 compressing bodies was added and it had support for multiple layered algorithms. Now this is also true for decompressing bodies when k6 gets them as responses.
count column in the end-of-test summary (#1143)The --summary-trend-stats now also recognizes count as a valid column and will output the count of samples in all Trend metrics. This could be especially useful for custom Trend metrics, since with them you no longer need to specify a separate accompanying Counter metric.
The example docker-compose that enabled easy running of InfluxDB+Grafana+k6 was refactored and all the images were updated to use the latest stable versions.
Thanks, @KajdeMunter!
Also the k6 Dockerfile Alpine version was bumped to 3.10. Thanks @dmitrytokarev!
http.batch() improvements and optimizations (#1259)We made several small improvements to the mechanism for executing multiple HTTP requests simultaneously from a single VU:
http.batch() should now be more efficient, especially for many requests, because of reduced locking, type conversions, and goroutine spawning.batchPerHost has been reduced from 0 (unlimited) to 6, to more closely match browser behavior. The default value for the batch option remains unchanged at 20.http.batch(arg), where arg is an array, would now return an array. Previously, this would have returned an object with integer keys, as explained in #767... Now http.batch() will return an array when you pass it an array, and return an object when you pass an object.setup and teardown timeouts, including hints on how to fix them. (#1173)open(), the resulting error message will now include the path to the specified folder. (#1238)k6 version output will now include more information - the git commit it was built from (in most cases), as well as the used Go version and architecture. (#1235)check if an uncaught error is thrown inside of it. (#1137)* when emitting HTTP metrics. (#1132)error handler on an error when closing the WebSocket, instead of calling with a null (#1118)GetBody in order to be able to get the body multiple times for a single request as needed in 308 redirects of posts and if the server sends GOAWAY with no error. (#1093)minIterationDuration for setup and teardown. (#1175)error_code metric tag values. (#1164)K6_ prefixed environment variables as k6 configuration, most notably DURATION and ITERATIONS. (#1215)Selection.map was not wrapping the nodes it was outputting, which lead to wrongly using the internal Goquery.Selection instead of k6's Selection. Thanks to @MMartyn for reporting this! (#1198)name tag. (#1220)batchPerHost of 20 wasn't propagated properly and was instead 0. (#1264)httpbin.org. (#1213)configdir. (#1162)envconfig as it was very old and the newer versions had fixes and features we want. (#1214)netext.NetTrail, instead of as a standalone one. Also cutting down on amount of bytes we sent to the cloud output. (#1203)master version (commit 007eef3) (#1259)-trimpath and CGO_ENABLED=0. Previously the GitHub release assets were built with CGO_ENABLED=0, making them unsuitable for non-glibc systems (like Alpine Linux). (#1244, #1245)The output of k6 version is different. It now contains not only the k6 version number, but also information about the git commit, build date, Go version and system architecture. For example, if previously the output of k6 version looked like k6 v0.25.1, now it is like this: k6 v0.26.0 (2019-12-16T10:58:59+0000/v0.26.0-0-gaeec9a7f, go1.13.5, linux/amd64). (#1235)
We had to make a few minor breaking changes in the course of improving http.batch() (#1259):
batchPerHost has been reduced from 0 (unlimited) to 6, to more closely match browser behavior. The default value for the batch option remains unchanged at 20.http.batch(arg), where arg is an array, would now return an array. Previously, this would have returned an object with integer keys, as explained in #767... Now http.batch() will return an array when you pass it an array, and return an object when you pass an object.