Datashield user support is provided via the DataSHIELD Forum, DataSHIELD user training classes are offered on a fee basis.  Please enquire at datashield@newcastle.ac.uk for current prices. 

Assign functions do not return an output to the client, with the exception of error or status messages. Assign functions create new objects and store them server-side either because the objects are potentially disclosive, or because they consist of the individual-level data which, in DataSHIELD, is never seen by the analyst. These new objects can include:

• new transformed variables (e.g. mean centred or log transformed variables)
• a new variable of a modified class (e.g. a variable of class numeric may be converted into a factor which R can then model as having discrete categorical levels)
• a subset object (e.g. a table may be split into males and females).
  Assign functions return no output to the client except to indicate an error or useful messages about the object store on
  server-side.

Aggregate functions analyse the data server-side and return an output in the form of aggregate data (summary statistics that are not disclosive) to the client. The help page for each function tells us what is returned and when not to expect an output on client-side.

The DataSHIELD cloud training environment does not use fixed IP addresses, the client and opal training server addresses change each training session. As part of the user tutorial you learn how to build a DataSHIELD login dataframe. In a real world instance of DataSHIELD this is populated with secure certificates not text based usernames and passwords.

my_logindata

As part of the user tutorial you learn how to build a DataSHIELD login dataframe. In a real world instance of DataSHIELD this is populated with secure certificates not text based usernames and passwords.

my_logindata

You might see the following status message that you can ignore. The message refers to the blocking of functions within the package. The following objects are masked from ‘package:xxxx’

Only DataSHIELD developers will need to change the default value of the last argument, directory , of the datashield.login function.

Almost all functions in DataSHIELD can display split results (results separated for each study) or pooled results (results for all the studies combined). This can be done using the type='split' and type='combine' argument in each function. The majority of DataSHIELD functions have a default of type='combined'. The default for each function can be checked in the function help page.

ds.class(x='D$LAB_HDL')
#$study1
#[1] "numeric"

#$study2
#[1] "numeric"

ds.mean(x='D$LAB_HDL')
#$`Global mean`
#[1] 1.561957

ds.mean(x='LAB_HDL.c')
#$`Global mean`
#[1] -4.280051e-05

ds.table2D(x='D$DIS_DIAB', y='D$GENDER')
# $counts
# $counts$`pooled-D$DIS_DIAB(row)|D$GENDER(col)`
#          0    1 Total
# 0     2625 2549  5174
# 1       52   25    77
# Total 2677 2574  5251

# $rowPercent
# $rowPercent$`pooled-D$DIS_DIAB(row)|D$GENDER(col)`
#           0     1 Total
# 0     50.73 49.27   100
# 1     67.53 32.47   100
# Total 50.98 49.02   100

# $colPercent
# $colPercent$`pooled-D$DIS_DIAB(row)|D$GENDER(col)`
#            0      1  Total
# 0      98.06  99.03  98.53
# 1       1.94   0.97   1.47
# Total 100.00 100.00 100.00

# $chi2Test
# $chi2Test$`pooled-D$DIS_DIAB(row)|D$GENDER(col)`
# 	Pearson's Chi-squared test with Yates' continuity correction
# data:  pooledContingencyTable
# X-squared = 7.9078, df = 1, p-value = 0.004922
# $validity
# [1] "All tables are valid!"

Scatter plots are not possible in DataSHIELD because they display individual data points and are hence potentially disclosive. Instead it is possible to draw a contour plot or a heatmap plot if the aim is to visualise a correlation pattern.

In DataSHIELD histogram outliers are not shown as these are potentially disclosive. The text summary of the function printed to the client screen informs the user of the presence of classes (bins) with a count smaller than the minimal cell count set by data providers.

ds.histogram(x='D$LAB_HDL', type='split')
#Warning: study1: 1 invalid cells
#Warning: study2: 1 invalid cells

ds.contourPlot(x='D$LAB_TSC', y='D$LAB_HDL')
#study1: Number of invalid cells (cells with counts >0 and <5) is 83
#study2: Number of invalid cells (cells with counts >0 and <5) is 97

The functions ds.contourPlot and ds.heatmapPlot use the range (mimumum and maximum values) of the x and y vectors in the process of generating the graph. But in DataSHIELD the minimum and maximum values cannot be returned because they are potentially disclosive; hence what is actually returned for these plots is the 'obscured' minimum and maximum. As a consequence the number of invalid cells, in the grid density matrix used for the plot, reported after running the functions can change slightly from one run to another. In the above output for example the number of invalid cells are respectively 83 and 97 for study1 and study2. These figures can change if the command is ran again but we should not be alarmed by this as it does not affect the plot itself. It was a deliberate decision to ensure the real minimum and maximum values are not returned.

Sub-setting is particularly useful in statistical analyses to break down variables or tables of variables into groups for analysis. Repeated sub-setting, however, can lead to thinning of the data to individual-level records that are disclosive (e.g. the statistical mean of a single value point is the value itself). Therefore, DataSHIELD does not subset an object below the minimal cell count set by the data providers (typically this is ≤ 4 observations).

Running ds.subsetByClass on a data frame without specifying the categorical variable in the argument variables will create a subset of all categorical variables. If the data frame holds many categorical variables the number of subsets produces might be too large - many of which may not be of interest for the analysis.

The outvar argument can be a vector of several numeric variables that the size, mean and sd is calculated for. But the number of categorical variables in the argument covar is limited to 3. This, because beyond that limit continuous sub-setting may lead to tables that hold a number of observation lower than the minimum cell count set by the data providers.

ds.histogram('BMI25plus$PM_BMI_CONTINUOUS')