Classes of tests and naming convention

Owned by Olly Butters
Last updated: 01 Nov, 2019 by Stuart Wheater
2 min read


Rationale

Several classes of tests are created to bring a certain level of completeness in verifying  DataSHIELD functionalities. As a whole, the following aspects are checked:

  • The functions returns some expected and correct values. 
  • Some values are returns repetitively the same outcome
  • The outcomes for some combinations of arguments provides the expected behaviour
  • The disclosure control has not been breached

Syntax and imports

To check the files are syntactically correct and all the relevant imports have been declared, you can use the following function.

devtools::check()


Additional reading:

Devtools CRAN documentation

Summary of tests and naming convention

Files must be saved under the  /test/testthat folder. The file must start with the pattern of letters ``test'' to be executed by the continuous integration and by ``testthat" locally.  

Naming convention of test files
test-[type of short name]-[function name].R


Type of testShort nameNaming conventionDescription
System integrity_test-_-Tests to be run before tests to ensure overall system integrity
Mathematically correctmathtest-math-

Verify some mathematical properties hold for a function. This type of tests relies 

on the TESTING datasets available locally and the testing virtual machines. 

Expected valuesexpttest-expt-

Test the outcomes matches some expected values computed by R. This type of tests relies on the TESTING datasets available locally and the testing virtual machines.

Smokesmktest-smk-Is it invokable without catching fire?
Smoke expected valuessmk_expttest-smk_expt-Expected values style smoke tests, often only checking results are repeated.
Arguments checkingargtest-arg-Does it behave as expected when unexpected arguments are passed to functions?
Disclosure controldisctest-disc-Does it adhere to the disclosure control settings?

Testthat contexts

For the testthat tests we use the 'context' to use define types of test. The short name must be from the short name list above and follow the format below. We also allow a modifier to make it more specific (e.g. poisson or gaussian if the function has different behaviour between the two). Additionally we add a 'single' or 'multiple' tag to specify if it is testing a single remote source or multiple remote sources. The context are name using this notation across all the tests.

Naming convention of contexts
[name of the function]::[type of test]::[purpose]::[single or multiple]
Example contextsExample test file names

ds.mean::math::something::multiple

test-math-ds.mean.R
ds.mean::expt::something::singletest-expt-ds.mean.R

ds.mean::smk

test-smk-ds.mean.R

ds.mean::multiple

test-smk-ds.mean.R

ds.mean::smk::gaussian

test-smk-ds.mean.R

ds.mean::smk::gaussian::single

test-smk-ds.mean.R

Examples of tests

Writing some tests

DataSHIELD Wiki by DataSHIELD is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. Based on a work at http://www.datashield.ac.uk/wiki