Server-side logs

Owned by Former user (Deleted)
Last updated: 23 Apr, 2020 by Stuart Wheater
5 min read

Developers may find it useful to consult the following logs that are available on Opal servers

/var/log/opal/datashield.log
  • Datashield audit log. Records datashield user actions:
  • the datashield command inspection (line containing "parsing ...") and the validation failure (if any)
  • the datashield command execution (line containing "evaluating ..." or "assigning ...")
/var/log/opal/opal.log
  • General purpose log. Default shows only info, warn and error messages.
/var/log/opal/rest.log
  • Opal web services audit log: who accessed to which rest resource and what was the result
/var/log/opal/stdout.log
  • Opal process output
/var/lib/rserver/logs/Rserve.log
  • Rserve process output
/var/lib/rserver/logs/rserver-admin.log
  • The Rserve process is managed by a small java application (rserver-admin). This one is the general purpose log of this manager (mainly logs starts/stops of the Rserve process)
/var/log/rserver/stdout.log
  • reserver-admin process output

Server log of DataSHIELD analysis commands

Opal keeps a log of all the commands that are run in the server-side R environment. These are stored on the server in:

/var/log/opal/datashield.log

Example

DataSHIELD commands

From the client, we can run the following commands:

ds.dim(x='D')
ds.dim('D', type='combine')
ds.colnames(x='D')
ds.class(x='D$LAB_HDL')
ds.quantileMean(x='D$LAB_HDL')
ds.mean(x='D$LAB_HDL')
ds.log(x='D$LAB_HDL')
ds.log(x='D$LAB_HDL', newobj='log_lab_hdl')
ds.length (x='LAB_HDL_log')
ds.length(x='D$LAB_HDL')
ds.assign(toAssign='D$LAB_HDL-1.562', newobj='LAB_HDL.c')
ds.mean(x='LAB_HDL.c')
ds.table1D(x='D$GENDER')
ds.table2D(x='D$DIS_DIAB', y='D$GENDER')

Log output

This generates the following log output:

2016-01-28 15:00:19,210 INFO  datashield.user - SubjectCredentials 'administrator' creating symbol 'D' from opal data 'DASIM.DASIM'
2016-01-28 15:00:21,635 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'colnames(D)'
2016-01-28 15:00:21,905 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::colnames( D )'
2016-01-28 15:00:36,693 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("D")'
2016-01-28 15:00:36,940 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "D" )'
2016-01-28 15:00:37,090 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'class(D)'
2016-01-28 15:00:37,328 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::class( D )'
2016-01-28 15:00:37,487 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'dim(D)'
2016-01-28 15:00:37,752 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::dim( D )'
2016-01-28 15:00:37,889 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("D")'
2016-01-28 15:00:38,128 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "D" )'
2016-01-28 15:00:38,264 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'class(D)'
2016-01-28 15:00:38,500 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::class( D )'
2016-01-28 15:00:38,641 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'dim(D)'
2016-01-28 15:00:38,880 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::dim( D )'
2016-01-28 15:00:39,049 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("D")'
2016-01-28 15:00:39,288 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "D" )'
2016-01-28 15:00:39,415 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'class(D)'
2016-01-28 15:00:39,653 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::class( D )'
2016-01-28 15:00:39,807 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'colnames(D)'
2016-01-28 15:00:40,044 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::colnames( D )'
2016-01-28 15:00:40,212 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("D")'
2016-01-28 15:00:40,448 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "D" )'
2016-01-28 15:00:40,620 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'class(D$LAB_HDL)'
2016-01-28 15:00:40,860 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::class( D$LAB_HDL )'
2016-01-28 15:00:41,007 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("D")'
2016-01-28 15:00:41,248 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "D" )'
2016-01-28 15:00:41,378 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'class(D$LAB_HDL)'
2016-01-28 15:00:41,616 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::class( D$LAB_HDL )'
2016-01-28 15:00:41,758 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'quantileMeanDS(D$LAB_HDL)'
2016-01-28 15:00:41,996 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'dsBase::quantileMeanDS( D$LAB_HDL )'
2016-01-28 15:00:42,173 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'length(D$LAB_HDL)'
2016-01-28 15:00:42,412 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::length( D$LAB_HDL )'
2016-01-28 15:00:42,536 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'numNaDS(D$LAB_HDL)'
2016-01-28 15:00:42,772 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'dsBase::numNaDS( D$LAB_HDL )'
2016-01-28 15:00:42,902 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("D")'
2016-01-28 15:00:43,140 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "D" )'
2016-01-28 15:00:43,295 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'class(D$LAB_HDL)'
2016-01-28 15:00:43,532 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::class( D$LAB_HDL )'
2016-01-28 15:00:43,696 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'meanDS(D$LAB_HDL)'
2016-01-28 15:00:43,936 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'dsBase::meanDS( D$LAB_HDL )'
2016-01-28 15:00:44,077 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'NROW(D$LAB_HDL)'
2016-01-28 15:00:44,316 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::NROW( D$LAB_HDL )'
2016-01-28 15:00:44,432 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'numNaDS(D$LAB_HDL)'
2016-01-28 15:00:44,668 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'dsBase::numNaDS( D$LAB_HDL )'
2016-01-28 15:00:44,801 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("D")'
2016-01-28 15:00:45,040 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "D" )'
2016-01-28 15:00:45,182 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'class(D$LAB_HDL)'
2016-01-28 15:00:45,420 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::class( D$LAB_HDL )'
2016-01-28 15:00:45,538 INFO  datashield.user - SubjectCredentials 'administrator' creating symbol 'LAB_HDL_log' from R script 'log(D$LAB_HDL,2.71828182845905)'
2016-01-28 15:00:45,539 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'log(D$LAB_HDL,2.71828182845905)'
2016-01-28 15:00:45,700 INFO  datashield.user - SubjectCredentials 'administrator' assigning 'LAB_HDL_log' with base::log( D$LAB_HDL,2.71828182845905 )
2016-01-28 15:00:45,808 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("LAB_HDL_log")'
2016-01-28 15:00:46,048 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "LAB_HDL_log" )'
2016-01-28 15:00:46,217 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("D")'
2016-01-28 15:00:46,456 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "D" )'
2016-01-28 15:00:46,575 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'class(D$LAB_HDL)'
2016-01-28 15:00:46,812 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::class( D$LAB_HDL )'
2016-01-28 15:00:46,933 INFO  datashield.user - SubjectCredentials 'administrator' creating symbol 'log_lab_hdl' from R script 'log(D$LAB_HDL,2.71828182845905)'
2016-01-28 15:00:46,933 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'log(D$LAB_HDL,2.71828182845905)'
2016-01-28 15:00:47,088 INFO  datashield.user - SubjectCredentials 'administrator' assigning 'log_lab_hdl' with base::log( D$LAB_HDL,2.71828182845905 )
2016-01-28 15:00:47,167 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("log_lab_hdl")'
2016-01-28 15:00:47,404 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "log_lab_hdl" )'
2016-01-28 15:00:47,543 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("LAB_HDL_log")'
2016-01-28 15:00:47,784 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "LAB_HDL_log" )'
2016-01-28 15:00:47,933 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'class(LAB_HDL_log)'
2016-01-28 15:00:48,172 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::class( LAB_HDL_log )'
2016-01-28 15:00:48,315 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'length(LAB_HDL_log)'
2016-01-28 15:00:48,556 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::length( LAB_HDL_log )'
2016-01-28 15:00:48,718 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("D")'
2016-01-28 15:00:48,956 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "D" )'
2016-01-28 15:00:49,122 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'class(D$LAB_HDL)'
2016-01-28 15:00:49,360 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::class( D$LAB_HDL )'
2016-01-28 15:00:49,485 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'length(D$LAB_HDL)'
2016-01-28 15:00:49,720 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::length( D$LAB_HDL )'
2016-01-28 15:00:49,837 INFO  datashield.user - SubjectCredentials 'administrator' creating symbol 'LAB_HDL.c' from R script 'D$LAB_HDL-1.562'
2016-01-28 15:00:49,837 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'D$LAB_HDL-1.562'
2016-01-28 15:00:49,992 INFO  datashield.user - SubjectCredentials 'administrator' assigning 'LAB_HDL.c' with D$LAB_HDL-1.562
2016-01-28 15:00:50,091 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("LAB_HDL.c")'
2016-01-28 15:00:50,328 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "LAB_HDL.c" )'
2016-01-28 15:00:50,485 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("LAB_HDL.c")'
2016-01-28 15:00:50,720 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "LAB_HDL.c" )'
2016-01-28 15:00:50,831 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'class(LAB_HDL.c)'
2016-01-28 15:00:51,072 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::class( LAB_HDL.c )'
2016-01-28 15:00:51,202 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'meanDS(LAB_HDL.c)'
2016-01-28 15:00:51,444 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'dsBase::meanDS( LAB_HDL.c )'
2016-01-28 15:00:51,585 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'NROW(LAB_HDL.c)'
2016-01-28 15:00:51,820 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::NROW( LAB_HDL.c )'
2016-01-28 15:00:51,932 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'numNaDS(LAB_HDL.c)'
2016-01-28 15:00:52,172 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'dsBase::numNaDS( LAB_HDL.c )'
2016-01-28 15:00:52,310 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("D")'
2016-01-28 15:00:52,553 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "D" )'
2016-01-28 15:00:52,688 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'class(D$GENDER)'
2016-01-28 15:00:52,928 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::class( D$GENDER )'
2016-01-28 15:00:53,071 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'table1dDS(D$GENDER)'
2016-01-28 15:00:53,312 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'dsBase::table1dDS( D$GENDER )'
2016-01-28 15:00:53,473 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("D")'
2016-01-28 15:00:53,708 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "D" )'
2016-01-28 15:00:53,813 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'exists("D")'
2016-01-28 15:00:54,052 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::exists( "D" )'
2016-01-28 15:00:54,184 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'class(D$DIS_DIAB)'
2016-01-28 15:00:54,420 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::class( D$DIS_DIAB )'
2016-01-28 15:00:54,557 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'class(D$GENDER)'
2016-01-28 15:00:54,800 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'base::class( D$GENDER )'
2016-01-28 15:00:54,903 INFO  datashield.user - SubjectCredentials 'administrator' parsing 'table2dDS(D$DIS_DIAB,D$GENDER)'
2016-01-28 15:00:55,144 INFO  datashield.user - SubjectCredentials 'administrator' evaluating 'dsBase::table2dDS( D$DIS_DIAB,D$GENDER )'

DataSHIELD Wiki by DataSHIELD is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. Based on a work at http://www.datashield.ac.uk/wiki