Testing

Testing consists of executing components with known input, and validating that the output matches what is expected. It reduces errors in the components, and enables verifying that the component works in the currently environment. Testing is optional, but strongly recommended.

Writing a test case

Let’s write test cases for SimpleCSVFilter. Test cases are located in the testcases folder under the component folder. Each test case is a sub-folder of testcases and the name of the folder should describe the purpose of the test.

The component interface gives hints what kinds of tests could be written. For SimpleCSVFilter, we note that it operates in three “modes”: pass-through (input unchanged), inclusive column filtering, and exclusive column filtering. This proposes three main test cases.

For the pass-through test case, we construct the following input CSV file and place it in testcases/case1_passthrough/input/in.csv. The name of the file (in.csv) corresponds to the input port.

Gene    Value   QualityOK
gene01  1.5     1
gene02  2.7     0
gene03  5.8     0
gene99  3.2     1

We expect that the component produces an unchanged output CSV file, so we copy in.csv to testcases/case1_passthrough/expected-output/out.csv. Again, the file names corresponds to the output port.

Executing test cases

Test cases are executed using anduril test -b demo. Here, test is the sub-command for running component tests, and -b demo gives the name of our bundle. Anduril prints messages during the execution of the test, and whether the test succeeded or not.

Test case with two inputs

The second test case, inclusive column filtering, requires a second input file (columnNames.csv) that contains names of columns that are to be preserved. For our second test, let’s keep using the in.csv file from the first test, and use the following for testcases/case2_include/input/columnNames.csv

ColumnName
Gene
Value

Now, the expected output (testcases/case2_include/expected-output/out.csv) is:

Gene    Value
gene01  1.5
gene02  2.7
gene03  5.8
gene99  3.2

Executing a test case selectively

We can run all tests for the demo bundle using anduril test -b demo, but when working on a new test case, it is more convenient to selectively run just one case. This is done using anduril test -b demo -c SimpleCSVFilter -t case2_include.

Test case with parameters

For our third test case, exclusive filtering, we need to set the include parameter to false. We can set the parameters for the test case by using a properties file in testcases/case3_exclude/component.properties:

component.include=false

Test cases for composite components

Writing and executing tests for composite components (such as PreprocessGeneCSV) is done in exactly the same way as for atomic components: by placing them under testcases, and executing using anduril test -b demo -c PreprocessGeneCSV. One consideration is that it can be necessary to use input and expected output files without file extensions (e.g., out instead of out.csv) because extensions are obtained from the workflow topology; Anduril will notify if it cannot locate a file with an extension.

Test cases that expect failure

It is good practice to also develop test cases that expect the component to fail. This verifies that the component correctly rejects invalid input and parameters. Such test cases are written by placing an empty files called failure in the test case folder, and omitting the expected-output folder.