Local DataSHIELD Training Environment

Getting ready

Start the Opal Servers

• Open VirtualBox. Left click on an Opal test server and select the green Start arrow (or double click on the Opal test server name).
• The Opal test servers take a few minutes to boot. Leave them running for a few minutes to ensure Opal has started. You do not need to login.

Start R/RStudio and load packages

• Start R (or RStudio, if using).
• Check for dataSHIELD package updates
• The following relevant R packages are required for analysis
  • opal to login and logout
  • dsBaseClient , dsStatsClient , ds.GraphicsClient and dsModellingClient containing all DataSHIELD functions referred to in this tutorial.
• To load the R packages, type the library function into the command line as given in the example below:

#update dataSHIELD packages
update.packages(repos='http://cran.obiba.org') 
 
#load libraries
library(opal)
library(dsBaseClient)
library(dsStatsClient)
library(dsGraphicsClient)
library(dsModellingClient)

The output in R/RStudio will look as follows:

library(opal)
#Loading required package: RCurl
#Loading required package: bitops
#Loading required package: rjson
library(dsBaseClient)
#Loading required package: fields
#Loading required package: spam
#Loading required package: grid
library(dsStatsClient)
library(dsGraphicsClient)
library(dsModellingClient)

Login template

A Horizontal-DataSHIELD process starts with a login to one or more Opal servers that hold the data behind the data provider firewall. Formatting of the login details is required to log into Opal servers:

• Login details are held in a data frame (table). The example below shows the format of the login data frame for DataSHIELD training VMs which consists of two datasets held by two Opal servers. This login file logindata is built into the DataSHIELD test environment.

• The DataSHIELD test environment login details are loaded using the data function. Calling logindata allows users to view the login data on the screen:

data(logindata)
logindata

# server url                 user          password table
# study1 192.168.56.100:8080 administrator password CNSIM.CNSIM
# study2 192.168.56.101:8080 administrator password CNSIM.CNSIM

path<-"/FILEPATH/login.txt"

my_login<-read.table(path, sep="", header=TRUE)

Log in to the remote servers

• Create a variable called opals that calls the datashield.login function to log into the desired Opal servers. In the DataSHIELD test environment logindata is our login template for the test Opal servers.

opals <- datashield.login(logins=logindata,assign=TRUE)

• The output below indicates that each of the two test Opal servers study1 and study2 contain the same 11 variables listed in capital letters under Variables assigned: .

> opals <- datashield.login(logins=logindata,assign=TRUE)
Logging into the collaborating servers

  No variables have been specified.
  All the variables in the opal table
  (the whole dataset) will be assigned to R!

Assigining data:
study1...
study2...

Variables assigned:
study1--LAB_TSC, LAB_TRIG, LAB_HDL, LAB_GLUC_ADJUSTED, PM_BMI_CONTINUOUS, DIS_CVA, MEDI_LPD, DIS_DIAB, DIS_AMI, GENDER, PM_BMI_CATEGORICAL
study2--LAB_TSC, LAB_TRIG, LAB_HDL, LAB_GLUC_ADJUSTED, PM_BMI_CONTINUOUS, DIS_CVA, MEDI_LPD, DIS_DIAB, DIS_AMI, GENDER, PM_BMI_CATEGORICAL

Assign individual variables on login

Users can specify individual variables to assign to the server-side R session. It is best practice to first create a list of the Opal variables you want to analyse.

• The example below creates a new variable myvar that lists the Opal variables required for analysis: LAB_HDL and GENDER 
• The variables argument in the function datashield.login uses myvar , which then will call only this list.

myvar <- list('LAB_HDL', 'GENDER')
opals <- datashield.login(logins=logindata,assign=TRUE,variables=myvar)

#Logging into the collaborating servers

#Assigining data:
#study1...
#study2...

#Variables assigned:
#study1--LAB_HDL, GENDER
#study2--LAB_HDL, GENDER

• The example below uses the argument symbol in the datashield.login function to change the name of the data frame from D to mytable .

myvar <- list('LAB_HDL', 'GENDER')
opals <- datashield.login(logins=logindata,assign=TRUE,variables=myvar, symbol='mytable')

#Logging into the collaborating servers

#Assigining data:
#study1...
#study2...

#Variables assigned:
#study1--LAB_HDL, GENDER
#study2--LAB_HDL, GENDER

Basic statistics and data manipulations

Descriptive statistics: variable dimensions and class

It is possible to get some descriptive or exploratory statistics about the assigned variables held in the server-side R session such as number of participants at each data provider, number of participants across all data providers and number of variables. Identifying parameters of the data will facilitate your analysis.

• The dimensions of the assigned data frame D can be found using the ds.dim command in which type='split' is the default argument:

opals <- datashield.login(logins=logindata,assign=TRUE)
ds.dim(x='D')

The output of the command is shown below. It shows that in study1 there are 2163 individuals with 11 variables and in study2 there are 3088 individuals with 11 variables:

> opals <- datashield.login(logins=logindata,assign=TRUE)
Logging into the collaborating servers
  No variables have been specified. 
  All the variables in the opal table 
  (the whole dataset) will be assigned to R!
Assigning data:
study1...
study2...
Variables assigned:
study1--LAB_TSC, LAB_TRIG, LAB_HDL, LAB_GLUC_ADJUSTED, PM_BMI_CONTINUOUS, DIS_CVA, MEDI_LPD, DIS_DIAB, DIS_AMI, GENDER, PM_BMI_CATEGORICAL
study2--LAB_TSC, LAB_TRIG, LAB_HDL, LAB_GLUC_ADJUSTED, PM_BMI_CONTINUOUS, DIS_CVA, MEDI_LPD, DIS_DIAB, DIS_AMI, GENDER, PM_BMI_CATEGORICAL


> ds.dim(x='D')
$study1
[1] 2163   11

$study2
[1] 3088   11

• Use the type='combine' argument in the ds.dim function to identify the number of individuals (5251) and variables (11) pooled across all studies:

ds.dim('D', type='combine')
#$pooled.dimension
#[1] 5251   11

• To check the variables in each study are identical (as is required for pooled data analysis), use the ds.colnames function on the assigned data frame D :

ds.colnames(x='D')
#$study1
# [1] "LAB_TSC"            "LAB_TRIG"           "LAB_HDL"            "LAB_GLUC_ADJUSTED"  "PM_BMI_CONTINUOUS"  "DIS_CVA"
# [7] "MEDI_LPD"           "DIS_DIAB"           "DIS_AMI"            "GENDER"             "PM_BMI_CATEGORICAL"

#$study2
# [1] "LAB_TSC"            "LAB_TRIG"           "LAB_HDL"            "LAB_GLUC_ADJUSTED"  "PM_BMI_CONTINUOUS"  "DIS_CVA"
# [7] "MEDI_LPD"           "DIS_DIAB"           "DIS_AMI"            "GENDER"             "PM_BMI_CATEGORICAL"

• Use the ds.class function to identify the class (type) of a variable - for example if it is an integer, character, factor etc. This will determine what analysis you can run using this variable class. The example below defines the class of the variable LAB_HDL held in the assigned data frame D, denoted by the argument x='D$LAB_HDL'

ds.class(x='D$LAB_HDL')
#$study1
#[1] "numeric"

#$study2
#[1] "numeric"

====================

opals <- datashield.login(logins=login_remoteServer,assign=TRUE)

data(login_remoteServer)