Cucumber

This documentation describes the technical aspects of our cucumber test suite.

See also: on how to write Cucumber tests.
See also: about Cucumber in the OSRM wiki.
See also: the Cucumber docs.

tl;dr

Run the Cucumber tests with:

bash

$ npm test

Single OSRM Configuration

An OSRM configuration consists of a routing algorithm and a data load method. OSRM currently supports the routing algorithms:

ch (Contraction Hierarchy), and
mld (Multi-Level-Dijkstra)

and the data load methods:

directly (load the files into memory),
mmap (use memory mapped files), and
datastore (use shared memory).

To test all scenarios with a single OSRM configuration, say:

bash

$ npx cucumber-js -p home -p mld -p mmap --parallel 8 --fail-fast

Explanations follow:

Profiles

Profiles are chosen with the -p commandline argument. Cucumber profiles allow you to change multiple configuration items with just one commandline argument. If you set more than one profile they are all merged into one configuration.

Note: Cucumber profiles should not be confused with OSRM profiles. Cucumber profiles are defined in cucumber.mjs. OSRM profiles reside in the profiles/*.lua files.

Our implementation offers following stock profiles. You should always use one base profile followed by zero or more additional profiles.

Name
home	Base profile to use on a developer machine
github	Base profile to use on the github CI server
ch	Additional profile that selects the CH algorithm
mld	Additional profile that selects the MLD algorithm
mmap	Additional profile that selects the mmap data load method
directly	Additional profile that selects the directly data load method
datastore	Additional profile that selects the datastore data load method
stress	Additional profile that selects only @stress tests
todo	Additional profile that selects only @todo tests
all	Additional profile that selects all tests

Arguments

Here is a description of all arguments you can pass to Cucumber. The interesting ones probably are: --fail-fast, --format, --parallel, and --tags.

Note: when using --parallel N make sure there are N contiguous free ports at the configured port number (eg. at ports 5000--5000+N).

All OSRM Configurations

We provide a shortcut to run all 6 configurations:

bash

$ npm test

This is how the tests are run on the CI server. You can pass the same arguments as mentioned above.

Cache

To speed up subsequent runs with the same parameters, the files generated by Cucumber and the by the OSRM extraction chain are held in a cache directory. This cache is located by default in test/cache and should be cleaned periodically:

bash

$ rm -rf test/cache

Configuration

The whole configuration is done in cucumber.mjs. You can either edit worldParameters in cucumber.mjs or use environment variables to override single defaults.

worldParameters	Environment Variable	Defaults to
	`CUCUMBER_TIMEOUT`	5000	Scenario timeout in ms.
`httpTimeout`	`CUCUMBER_HTTP_TIMEOUT`	2000	HTTP timeout in ms.
`testPath`	`CUCUMBER_TEST_PATH`	`test`	The test directory
`profilesPath`	`CUCUMBER_PROFILES_PATH`	`profiles`	The profiles directory
`logsPath`	`CUCUMBER_LOGS_PATH`	`test/logs`	The logs directory
`cachePath`	`CUCUMBER_CACHE_PATH`	`test/cache`	The cache directory
`buildPath`	`OSRM_BUILD_DIR`	`build`	Path to the binaries
`loadMethod`	`OSRM_LOAD_METHOD`	datastore	Data load method
`algorithm`	`OSRM_ALGORITHM`	ch	Routing algorithm
`ip`	`OSRM_IP`	127.0.0.1	IP Address
`port`	`OSRM_PORT`	5000	IP Port

The default Cucumber timeout can be changed by setting the environment variable CUCUMBER_TIMEOUT. This is discouraged, because the default timeout of 5 seconds is plenty for the problem sizes we are dealing with. The probable reasons for a test timing out are that osrm-routed died or that sync between osrm-datastore and osrm-routed was lost.

Other environment variables

OSRM_RASTER_SOURCE is set by 'Given the raster source' and is supposed to be read back in your profiles/*.lua profile by os.getenv('OSRM_RASTER_SOURCE').

OSRM_PROFILE See: Pull Request #4516

Tags

Single scenarios or whole feature files can be tagged. Tag names can be selected arbitrarily although it is best to conform to the tags already used. Eg. the tag @guidance can be used to run only those tests related to the guidance feature:

bash

$ npm test -- --tags @guidance

We also support following special tags:

Tag	A scenario thus tagged ...
`@isolated`	will not run while any other scenario is running in parallel
`@with_(datastore\|directly\|mmap)`	will be executed iff the load method matches
`@no_(datastore\|directly\|mmap)`	will be executed unless the load method matches
`@with_(ch\|mld)`	will be executed iff the algorithm matches
`@no_(ch\|mld)`	will be executed unless the algorithm matches
`@with_(linux\|darwin\|win32)`	will be executed iff the OS matches
`@no_(linux\|darwin\|win32)`	will be executed unless the OS matches

A test that calls osrm-datastore --spring-clean should not run concurrently with any other test, thus the tag @isolated should be applied. A test that runs or kills osrm-routed should not run while testing the datastore load method, and thus should be labeled with the tag @no_datastore.