DAX Command Line Tools¶
Table of Contents¶
List of the Tools¶
Each tool has a help option and some examples on how to use the tools. You can call each tool with no arguments to see the help.
XnatSetup¶
You can use the Xnatsetup (see below) command tool to setup your computer with the –basic options. It will do what is below automatically, but if you don’t want to do that, it can be setup manually.
This Xnat commands will use two thing :
install pyxnat and python packages on your computer (Check ‘Get started for Spiders’ on the wiki)
set your bashrc with the env variable to connect to Xnat with pyxnat :
export XNAT_HOST=http://xnat.vanderbilt.edu/xnat
export XNAT_USER=username
export XNAT_PASS=password
export PATH=/PathToMasimatlab/trunk/xnatspiders/Xnat_tools:$PATH
FYI : you can open the bashrc like :
vim (or nano or any editor you like) ~/.bashrc
and when you are done editing it, use :
. ~/.bashrc
You will after this be able to call the commands directly on your terminal.
One last thing, the Xnatupload will send you updates (errors and warnings) about the directory you are trying to upload on Xnat. If you want to receive this email, you need to set up two variables in your bashrc :
export EMAIL_ADDR=add@gmail.com
export EMAIL_PWS=passwordforthisemail
It will use this email address to send you email. It has to be a gmail address.
Xnatsetup is as you can guess a command tool to set up your computer. It will install the python package needed and ask for the variables that need to be set up. There are different kinds of setup :
basic to be able to use the XNAT command tools
advance to run spiders on your computer or ACCRE
redcap to use the spider to send data to redcap
cci package setup for ACCRE or if you need XnatUtils
ACCRE setup
#######################################################
# XNATSETUP
# XnatSetup is a command tool to set up on your computer the variables to use
# the tools/spiders.
# Developed by the masiLab Vanderbilt University, TN, USA.
# Operating system : Linux & Mac OS
# Shell : bash
# Requirements : python with pip & git
# Contact : benjamin.c.yvernault@vanderbilt.edu
# No Arguments given
# See the help bellow or Use "Xnatquery" -h
#######################################################
Usage: Xnatsetup [options]
What is the script doing : Set up your computer to use xnat.
*Basic installation (--basic) - Needed to use the Xnat command tools or any of
the next installations : install the python package httplib2, lxml, and pyxnat if not
already install & saving your username/host/password for XNAT.
*Advance installation (--advance ) - Needed to run the non-specific spiders :
Set up the Upload Directory, set up masimatlab path for Xnatrun, add the xnat
tools to your PATH, and add Spiders.py in your PYTHONPATH .
*Redcap installation (--redcap ) - Needed to use redcap spiders (send data to
redcap) : Install Pycap / pandas if not install and set up the URL for redcap .
*API installation (--api ) - Needed to use API package to run spiders on ACCRE
via jobs (Contains XnatUtils) : Install API if not install.
*ACCRE installation (--accre) - Setup the environment to use the
package/spiders/tools on ACCRE.
Options:
-h, --help show this help message and exit
--basic Use this options to set up the env variables to use
the Xnat tools and have the basic set up.
--advance Use this options to set up the env variables to run
spiders in general.
--redcap Use this options to set up the env variables to use
redcap spiders.
--api Use this options to set up the env variables to run
spiders on ACCRE via jobs.
--Accre Use this options if you are on Accre.
--NoSudo Use this options if you don't have sudo access and you
still want to install the package (check -d option).
-d INSTALLDIR, --installdir=INSTALLDIR
Use this options to specify a directory where the
python package need to be install. It works only if
you use --NoSudo option.
--tutorial Give you the step for the specific setup you are
asking.
Contact - benjamin.c.yvernault@vanderbilt.edu
XnatQuery¶
Xnatquery will show you the tree on xnat. Xnatquery is a tool to query objects on XNAT for each level. You can see which projects you have access to and see the hierarchy of data on your project. It has several options (accessible with -h or –help) :
################################################################
# XnatQuery #
# #
# Developed by the MASI Lab Vanderbilt University, TN, USA. #
# If issues, please start a thread here: #
# https://groups.google.com/forum/#!forum/vuiis-cci #
# Usage: #
# Query through XNAT at the level you want. #
# Examples: #
# Check the help for examples by running --help #
################################################################
----------------------------------------------------------------
usage: XnatQuery [-h] [--host HOST] [-u USERNAME] [-p PROJECT] [-s SUBJECT]
[-e SESSION] [-a ASSESSOR] [-c SCAN] [--all] [--me]
What is the script doing :
* Query on Xnat at any level.
Examples:
*Show all the projects you have access to:
Xnatquery --me
*Show all projects:
Xnatquery --all
*Query a specific level (example scan/assessors for a session):
Xnatquery -p PID -s 109873 -e 109873
*Query a specific level with all objects under it :
Xnatquery -p PID -s 109873 --all
optional arguments:
-h, --help show this help message and exit
--host HOST Host for XNAT. Default: env XNAT_HOST.
-u USERNAME, --username USERNAME
Username for XNAT.
-p PROJECT, --project PROJECT
project ID on Xnat or 'all' to see all the project.
-s SUBJECT, --subject SUBJECT
Subject label on Xnat
-e SESSION, --experiment SESSION
Session label on Xnat
-a ASSESSOR, --assessor ASSESSOR
Assessor/Process label on XNAT. E.G: VUSTP-x-VUSTP1-x-VUSTP1a-x-FS
-c SCAN, --scan SCAN Scan ID on Xnat.
--all Print all the objects on XNAT from the level you are at.
--me Give the projects ID that you have access.
Extra Examples
To get information on the project
Xnatquery -p projectID --info
To get all the subjects in this project
Xnatquery -p projectID
To get all the experiments in this project
Xnatquery -p projectID -s subject
Contact - benjamin.c.yvernault@vanderbilt.edu
XnatCheck¶
Xnatcheck is a quick way to check directly on your terminal if there is the resource you just created on all your project. You can check if there is a scan type or an assessor type as well with the options -s or -a. Options available (-h or -help):
################################################################
# XnatCheck #
# #
# Developed by the MASI Lab Vanderbilt University, TN, USA. #
# If issues, please start a thread here: #
# https://groups.google.com/forum/#!forum/vuiis-cci #
# Usage: #
# Check XNAT data (subject/session/scan/assessor/resource) #
# Examples: #
# Check the help for examples by running --help #
################################################################
----------------------------------------------------------------
usage: XnatCheck [-h] [--host HOST] [-u USERNAME] [-p PROJECTS]
[--filters FILTERS [FILTERS ...]]
[--delimiter DELIMITER_FILTER_RESOURSE] [--csv CSV_FILE]
[--format FORMAT] [--printfilters] [--printformat]
What is the script doing :
*Check object on XNAT (subject/session/scan/assessor/resources) specify by the options.
How to write a filter string:
- for resources filters, the string needs to follow this template:
variable_name=value--sizeoperatorValue--nbfoperatorValue--fpathsoperatorValue
By default, it will return the assessor that does have the resource if no other filter specify
- for other filters, the string needs to follow this template:
variable_name=Value
operator can be different than =. Look at the table in --printfilters
Use --printfilters to see the different variables available
Examples:
*See format variables:
Xnatcheck --printformat
*See filter variables:
Xnatcheck --printfilters
*Get list of T1,DTI scans that have a resource called NIFTI:
Xnatcheck -p PID --filters type=T1,DTI assessor_res=NIFTI
*Get list of fMRIQA assessors that have a resource called PDF:
Xnatcheck -p PID --filters proctype=fMRIQA assessor_res=PDF
*Get list of assessors except fMRIQA that have a resource called PDF :
Xnatcheck -p PID --filters proctype!=fMRIQA assessor_res=PDF
*Get list of project sessions that do not have a resource called testing:
Xnatcheck -p PID --filters session_label=VUSTP1a,VUSTP2b,VUSTP3a session_res!=testing
*Get list of project fMRIQA and VBMQA that used more than 45mb and less than 1hour:
Xnatcheck -p PID1,PID2 --filters proctype=fMRIQA,VBMQA procstatus=COMPLETE "memused>45mb" "walltimeused<1:00:00" --format assessor_label,procnode,memused,walltimeused
optional arguments:
-h, --help show this help message and exit
--host HOST Host for XNAT. Default: env XNAT_HOST.
-u USERNAME, --username USERNAME
Username for XNAT.
-p PROJECTS, --project PROJECTS
Project(s) ID on XNAT
--filters FILTERS [FILTERS ...]
List of filters separated by a space to apply to the search.
--delimiter DELIMITER_FILTER_RESOURSE
Resource filters delimiter. By default: --.
--csv CSV_FILE File path to save the CSV output.
--format FORMAT Header for the csv. format: list of variables name comma-separated
--printfilters Print available filters.
--printformat Print available format for display.
Extra Examples
To return all the scans where there is no NIFTI on the project BLSA
Xnatcheck -p BLSA -r NIFTI
To return all the assessors where there is no PDF on the project BLSA
Xnatcheck -p BLSA -r PDF -l 1
To return all the subjects/experiments where there is no fMRIQA assessor on the project BLSA
Xnatcheck -p BLSA -a fMRIQA
To return all the subjects/experiments where there is no fMRIQA assessor on the project BLSA and check for the one that exists if there is a PDF resource
Xnatcheck -p BLSA -a fMRIQA -r PDF
Contact - benjamin.c.yvernault@vanderbilt.edu
XnatDownload¶
Xnatdownload will download all the resources that you asked for in a directory. Xnatdownload provides bulk download of data from XNAT with specific filters applied. It provides options to narrow your download to only what you need. This tool will generate a folder per project in your -d directory with two files: download_commandLine.txt and download_report.csv with the description of what you downloaded. It has several options (accessible with -h or -help) :
########################################################
# XNATDOWNLOAD
#
# Developed by the masiLab Vanderbilt University, TN, USA.
# If issues, email benjamin.c.yvernault@vanderbilt.edu
# Parameters :
# No Arguments given
# See the help bellow or Use "Xnatdownload" -h
########################################################
usage: Xnatdownload [-h] [--host HOST] [-u USERNAME] [-p PROJECT]
[-d DIRECTORY] [-D] [--subj SUBJECT] [--sess SESSION]
[-s SCANTYPE] [-a ASSESSORTYPE] [--WOS WITHOUTS]
[--WOP WITHOUTA] [--quality QUALITIES] [--status STATUS]
[--qcstatus QCSTATUS] [-c CSVFILE] [--rs RESOURCESS]
[--ra RESOURCESA] [--selectionS SELECTIONSCAN]
[--selectionP SELECTIONASSESSOR] [--overwrite] [--update]
[--fullRegex] [-o OUTPUTFILE] [-i] [-b BIDS_DIR] [-xt]
[--bond_dir BOND_DIR]
What is the script doing :
*Download filtered data from XNAT to your local computer using the different OPTIONS.
Examples:
*Download all resources for all scans/assessors in a project:
Xnatdownload -p PID -d /tmp/downloadPID -s all --rs all -a all --ra all
*Download NIFTI for T1,fMRI:
Xnatdownload -p PID -d /tmp/downloadPID -s T1,fMRI --rs NIFTI
*Download only the outlogs for fMRIQA assessors that failed:
Xnatdownload -p PID -d /tmp/downloadPID -a fMRIQA --status JOB_FAILED --ra OUTLOG
*Download PDF for assessors that Needs QA:
Xnatdownload -p PID -d /tmp/downloadPID -a all --qcstatus="Needs QA" --ra OUTLOG
*Download NIFTI for T1 for some sessions :
Xnatdownload -p PID -d /tmp/downloadPID --sess 109309,189308 -s all --rs NIFTI
*Download same data than previous line but overwrite the data:
Xnatdownload -p PID -d /tmp/downloadPID --sess 109309,189308 -s all --rs NIFTI --overwrite
*Download data described by a csvfile (follow template) :
Xnatdownload -d /tmp/downloadPID -c upload_sheet.csv
*Transform the XnatDownload data in BIDS format for all sessions, scantype and resources:
Xnatdownload -p PID --sess all -d /tmp/downloadPID -s all --rs all --bids /tmp/BIDS_dataset
*Transform the XnatDownload data in BIDS format for all sessions, scantype and resources with xnat tag:
Xnatdownload -p PID --sess all -d /tmp/downloadPID -s all --rs all --bids /tmp/BIDS_dataset -xt
*Transform the XnatDownload data in BIDS format for all sessions, scantype and resources with xnat tag and perform bond:
Xnatdownload -p PID --sess all -d /tmp/downloadPID -s all --rs all --bids /tmp/BIDS_dataset -xt --bond /tmp/BOND_dir
optional arguments:
-h, --help show this help message and exit
--host HOST Host for XNAT. Default: using $XNAT_HOST.
-u USERNAME, --username USERNAME
Username for XNAT. Default: using $XNAT_USER.
-p PROJECT, --project PROJECT
Project(s) ID on Xnat
-d DIRECTORY, --directory DIRECTORY
Directory where the data will be download
-D, --oneDirectory Data will be downloaded in the same directory. No sub-
directory.
--subj SUBJECT filter scans/assessors by their subject label. Format:
a comma separated string (E.G: --subj VUSTP2,VUSTP3).
--sess SESSION filter scans/assessors by their session label. Format:
a comma separated string (E.G: --sess VUSTP2b,VUSTP3a)
-s SCANTYPE, --scantype SCANTYPE
filter scans by their types (required to download
scans). Format: a comma separated string (E.G : -s
T1,MPRAGE,REST). To download all types, set to 'all'.
-a ASSESSORTYPE, --assessortype ASSESSORTYPE
filter assessors by their types (required to download
assessors). Format: a comma separated string (E.G : -a
fMRIQA,dtiQA_v2,Multi_Atlas). To download all types,
set to 'all'.
--WOS WITHOUTS filter scans by their types and removed the one with
the specified types. Format: a comma separated string
(E.G : --WOS T1,MPRAGE,REST).
--WOP WITHOUTA filter assessors by their types and removed the one
with the specified types. Format: a comma separated
string (E.G : --WOP fMRIQA,dtiQA).
--quality QUALITIES filter scans by their quality. Format: a comma
separated string (E.G: --quality
usable,questionable,unusable).
--status STATUS filter assessors by their job status. Format: a comma
separated string.
--qcstatus QCSTATUS filter assessors by their quality control status.
Format: a comma separated string.
-c CSVFILE, --csvfile CSVFILE
CSV file with the following header: object_type,projec
t_id,subject_label,session_type,session_label,as_label
. object_type must be 'scan' or 'assessor' and
as_label the scan ID or assessor label.
--rs RESOURCESS Resources you want to download for scans. E.g : --rs
NIFTI,PAR,REC.
--ra RESOURCESA Resources you want to download for assessors. E.g :
--ra OUTLOG,PDF,PBS.
--selectionS SELECTIONSCAN
Download from only one selected scan.By default : no
selection. E.G : project-x-subject-x-session-x-scan
--selectionP SELECTIONASSESSOR
Download from only one selected processor.By default :
no selection. E.G : assessor_label
--overwrite Overwrite the previous data downloaded with the same
command.
--update Update the files from XNAT that have been downloaded
with the newest version if there is one (not working
yet).
--fullRegex Use full regex for filtering data.
-o OUTPUTFILE, --output OUTPUTFILE
Write the display in a file giving to this OPTIONS.
-i, --ignore Ignore reading of the csv report file
-b BIDS_DIR, --bids BIDS_DIR
Directory to store the XNAT to BIDS curated data
-xt, --xnat_tag Download BIDS data with XNAT subjID and sessID
--bond_dir BOND_DIR Download the Key groups and Param groups in BIDS data to BOND_DIR
Example
Downloads in /home/benjamin/temp the resources NIFTI and PDF for all the scan fMRI and the assessor fMRIQA for the project BLSA
Xnatdownload -p BLSA -d /home/benjamin/temp/ -a fMRIQA -s fMRI -r NIFTI,PDF
Contact - benjamin.c.yvernault@vanderbilt.edu
XnatUpload¶
Xnatupload will create subject/experiment/scan/resources for a project on XNAT and upload the data into the project from a folder. Xnatupload provides bulk upload of data to a project on XNAT. You need to provide a specific CSV file with the following header:
object_type,project_id,subject_label,session_type,session_label,as_label,as_type,as_description,quality,resource,fpath
where:
as_label corresponds to assessor or scan label
as_type corresponds to proctype or scantype
as_description corresponds to procstatus or series description for the scan
quality corresponds to qastatus or quality for scan
It should be similar to this (project in the example is CIBS-TEST):
object_type,project_id,subject_label,session_type,session_label,as_label,as_type,as_description,quality,resource,fpath scan,CIBS-TEST,CIBS-TEST_01,MR,CIBS-TEST_01,401,BRAIN2_3DT1,BRAIN2_3DT1,usable,NIFTI,/Users/<USER>/Downloads/Archive/DICOM_T1W_3D_TFE.nii.gz
Methods
Warning: the project must already exist on XNAT. You can add a new project via the XNAT web GUI. Follow one of the three methods to upload:
Number 1 : all the files are in one directory but they need to be rename like this projectID-x-subjectID-x-experimentID-x-scanID-x-scantype-x-resourcename.extention. Fastest methode but only one file can be upload in a resource.
Number 2 : you don’t need to rename all the files but you need to give a specific structure to your directory : folder/subjectID/experimentID/scanID-x-scantype/ResourceID/ and put the resources corresponding in it. E.G : TempDir/BLSA_0000/BLSA_0000_0/scan2-x-fMRI/NIFTI/nifti.nii.gz. It will not be as fast as the first methode but you can upload more than one file to a resources.
Option -o : There is a third choice. If you want to upload files to Xnat on a scan and you don’t want to create anything, you can use this options -o. It’s for only upload. It’s using something like the first methodes : put all the files into one folder with a special name : projectID-x-subjectID-x-experimentID-x-scanID-x-resourcename.extention for assessor, assessor_label-resourcename.extension for assessor (Reminder : assessor_label = projectID-x-subjectID-x-experimentID-x-scanID-x-process_name or projectID-x-subjectID-x-experimentID-x-processname).
################################################################
# XnatUpload #
# #
# Developed by the MASI Lab Vanderbilt University, TN, USA. #
# If issues, please start a thread here: #
# https://groups.google.com/forum/#!forum/vuiis-cci #
# Usage: #
# Print a detailed report from XNAT projects. #
# Examples: #
# Check the help for examples by running --help #
################################################################
IMPORTANT WARNING FOR ALL USERS ABOUT XNAT:
session_label needs to be unique for each session.
Two subjects can NOT have the same session_label
----------------------------------------------------------------
usage: XnatUpload [-h] [--host HOST] [-u USERNAME] -c CSV_FILE
[--sess SESSION_TYPE] [--report] [--force] [--delete]
[--deleteAll] [--noextract] [--printmodality]
[-o OUTPUT_FILE] [-b BIDS_DIR] [-p PROJECT]
What is the script doing :
* Upload data to XNAT following the csv file information.
csv header:
object_type,project_id,subject_label,session_type,session_label,
as_label,as_type,as_description,quality,resource,fpath
IMPORTANT: YOU NEED TO CREATE THE PROJECT ON XNAT BEFORE UPLOADING.
Examples:
* See Session type:
Xnatupload --printmodality
* Simple upload:
Xnatupload -c upload_sheet.csv
* Upload everything with a session type:
Xnatupload -c upload_sheet.csv --sess PET
* Check the upload:
Xnatupload -c upload_sheet.csv --report
* Force upload:
Xnatupload -c upload_sheet.csv --force
* Upload with delete resource before uploading:
Xnatupload -c upload_sheet.csv --delete
* Upload with delete every resources for the object (SCAN/ASSESSOR) before uploading:
Xnatupload -c upload_sheet.csv --deleteAll
* Upload BIDS data to XNAT format for scan
Xnatupload -b /tmp/bidsDataset -p PID
* Check BIDS data to XNAT
Xnatupload -b /tmp/bidsDataset -p PID --report
* Force upload BIDS data to upload XNAT
Xnatupload -b /tmp/bidsDataset -p PID --force
optional arguments:
-h, --help show this help message and exit
--host HOST Host for XNAT. Default: env XNAT_HOST.
-u USERNAME, --username USERNAME
Username for XNAT.
-c CSV_FILE, --csv CSV_FILE
CSV file with the information for uploading data to XNAT. Header: object_type,project_id,subject_label,session_type,session_label,as_label,as_type,as_description,as_quality,resource,fpath
--sess SESSION_TYPE Session type on Xnat. Use printmodality to see the options.
--report Print a report to verify inputs.
--force Force the upload and remove previous resources.
--delete Delete resource files prior to upload.
--deleteAll Delete all resources in object prior to upload.
--noextract Do not extract the zip files on XNAT when uploading a folder.
--printmodality Display the different modality available on XNAT for a session.
-o OUTPUT_FILE, --output OUTPUT_FILE
File path to store the script logs.
-b BIDS_DIR, --bids BIDS_DIR
BIDS Directory to convert to XNAT and then upload
-p PROJECT, --project PROJECT
Project for BIDS XNAT upload
Extra Examples
Shows on the terminal what kind of data the command is going to upload and where with method 1
Xnatupload -d /Path/to/directory --report --up1
Uploads the files in the directory with the first method
Xnatupload -p projectID -d /Path/to/directory -up1 -sess MR
Uploads the files in the directory with the second method
Xnatupload -p projectID -d /Path/to/directory --up2 --sess CT
Uploads (only, no creation of subject/exp/scan) all the files from the directory into Xnat even if there is already a resources (options -force)
Xnatupload -d /Path/to/directory -o -T 1 --force
Contact - benjamin.c.yvernault@vanderbilt.edu
XnatReport¶
Xnatreport will give you a report on one ore more projects. It will show all the subjects/sessions/scans/assessors/resources for the projects chosen. It has several options (accessible with -h or -help) :
################################################################
# XnatReport #
# #
# Developed by the MASI Lab Vanderbilt University, TN, USA. #
# If issues, please start a thread here: #
# https://groups.google.com/forum/#!forum/vuiis-cci #
# Usage: #
# Print a detailed report from XNAT projects. #
# Examples: #
# Check the help for examples by running --help #
################################################################
----------------------------------------------------------------
usage: XnatReport [-h] [--host HOST] [-u USERNAME] [-p PROJECTS] [-c CSV_FILE]
[--format FORMAT] [--printformat]
What is the script doing :
* Create a report about Xnat projects.
Examples:
*Report of a project:
Xnatreport -p PID
*Report with a specific format:
Xnatreport -p PID --format object_type,session_id,session_label,age
*print the format available:
Xnatreport --printformat
*Save report in a csv:
Xnatreport -p PID -c report.csv
optional arguments:
-h, --help show this help message and exit
--host HOST Host for XNAT. Default: env XNAT_HOST.
-u USERNAME, --username USERNAME
Username for XNAT.
-p PROJECTS, --project PROJECTS
List of project ID on Xnat separate by a coma
-c CSV_FILE, --csvfile CSV_FILE
csv fullpath where to save the report.
--format FORMAT Header for the csv. format: variables name separated by comma.
--printformat Print available variables names for the option --format.
Extra Examples
Creates a report for BLSA and CTONS and will print it on the screen/terminal
Xnatreport -p BLSA,CTONS
Sends the report on BLSA and CTONS to your email address as a csv file. You need to set to variables gmail address and password used to sent the email in your bashrc
Xnatreport -p BLSA,CTONS --csv -e email@email.com
Writes the report as a “.csv” file that can be open with Excel. If not path specify, /tmp is the place where the report is save. -t will do the same but in a text file
Xnatreport -p BLSA,CTONS --csv
Contact - benjamin.c.yvernault@vanderbilt.edu
XnatSwitchProcessStatus¶
XnatSwitchProcessStatus is one of the most powerful and used of the Xnat_tools. It allows the user to switch/set the procstatus (job status) for a specific proctype (type of assessor) in a project. XnatSwitchProcessStatus allows the user to change the status of several type of assessors in a project that have a specific type or just for all of them.
################################################################
# XnatSwitchProcessStatus #
# #
# Developed by the MASI Lab Vanderbilt University, TN, USA. #
# If issues, please start a thread here: #
# https://groups.google.com/forum/#!forum/vuiis-cci #
# Usage: #
# Change assessor job/quality control status. #
# Examples: #
# Check the help for examples by running --help #
################################################################
----------------------------------------------------------------
usage: XnatSwitchProcessStatus [-h] [--host HOST] [-u USERNAME]
[--select SELECT] [-x TXT_FILE] [-p PROJECTS]
[--subj SUBJECTS] [--sess SESSIONS] [-s STATUS]
[-f FORMER_STATUS] [-t PROCTYPES]
[-n NEED_INPUTS] [-d] [--qc] [--printstatus]
[--fullRegex] [--restart] [--rerun] [--init]
[--rerundiskq]
What is the script doing :
*Switch/Set the status for assessors on XNAT selected by the proctype.
Examples:
*See status managed by DAX:
XnatSwitchProcessStatus --printstatus
*Set all fMRIQA to a specific status Error for a project:
XnatSwitchProcessStatus -p PID -s Error -t fMRIQA
*Set all Multi_Atlas that have the status JOB_FAILED to NEED_TO_RUN to have the processes run again:
XnatSwitchProcessStatus -p PID -f JOB_FAILED -t Multi_Atlas -s NEED_TO_RUN
*Set all VBMQA to NEED_TO_RUN for a project and delete resources:
XnatSwitchProcessStatus -p PID -s NEED_TO_RUN -t VBMQA -d
*Set all VBMQA to NEED_TO_RUN, delete resources, and set linked assessors fMRI_Preprocess to NEED_INPUTS:
XnatSwitchProcessStatus -p PID -s NEED_TO_RUN -t VBMQA -d -n fMRI_Preprocess
*Set all dtiQA_v2 qa status to Passed for a project:
XnatSwitchProcessStatus -p PID -s Passed -t dtiQA_v2 --qc
*Set FreeSurfer for a specific project/subject to NEED_INPUTS:
XnatSwitchProcessStatus -p PID --subj 123 -s NEED_INPUTS -t FreeSurfer
optional arguments:
-h, --help show this help message and exit
--host HOST Host for XNAT. Default: env XNAT_HOST.
-u USERNAME, --username USERNAME
Username for XNAT.
--select SELECT Give the assessor label that you want to change the status.
-x TXT_FILE, --txtfile TXT_FILE
File txt. Each line represents the label of the assessor which need to change status.
-p PROJECTS, --project PROJECTS
Project ID on XNAT or list of Project ID
--subj SUBJECTS Change Status for only this subject/list of subjects.
--sess SESSIONS Change Status for only this session/list of sessions.
-s STATUS, --status STATUS
Status you want to set on the Processes. E.G: 'NEED_TO_RUN'
-f FORMER_STATUS, --formerStatus FORMER_STATUS
Change assessors with this former status. E.G: 'JOB_FAILED'
-t PROCTYPES, --type PROCTYPES
Assessor process type you want the status to changed.
-n NEED_INPUTS, --Needinputs NEED_INPUTS
Assessor process type that need to change to NEED_INPUTS because the assessors from -t you changed are inputs to those assessors.
-d, --deleteR Delete the resources on the assessor.
--qc Change the quality control status on XNAT.
--printstatus Print status used by DAX to manage assessors.
--fullRegex Use full regex for filtering data.
--restart Restart the assessors by switching the status for all assessors found to NEED_TO_RUN and delete previous resources.
--rerun Rerun the assessors by switching status to NEED_TO_RUN for assessors that failed and delete previous resources.
--init Init the assessors by switching status to NEED_INPUTS for assessors that have been set to NO_DATA.
--rerundiskq Rerun the assessor that have the status JOB_FAILED: switching status to NEED_INPUTS from JOB_FAILED and delete previous resources.
Extra Examples
Changes the status for dtiQA_v2 and Freesurfer that have a Failed status to NeedToRun in project BLSA
XnatSwitchProcessStatus -p BLSA -f Failed -s NeedToRun -t dtiQA_v2,FreeSurfer
Changes the status for dtiQA_v2 and Freesurfer that have a Failed status to NeedToRun in project BLSA and it will delete all the resources on the assessor
XnatSwitchProcessStatus -p BLSA -f Failed -s NeedToRun -t dtiQA_v2,FreeSurfer -d
Changes the status for the specific FreeSurfer assessor in BLSA_0000_00 session to NeedToRun and delete the resources
XnatSwitchProcessStatus --select BLSA-x-BLSA_0000-x-BLSA_0000_00-x-FreeSurfer -s NeedToRun -d
Contact - benjamin.c.yvernault@vanderbilt.edu
XnatProcessUpload¶
Xnatprocessupload allows you to upload data for an assessor (you can’t do it that with Xnatupload). You only need to give the path to the folder where the data are. If the assessor doesn’t exist, it will create one. You need to organize the data like this :
One folder per assessor you want to upload, the name of the folder needs to be the name of the assessor (Remember: assessor label = projectID-x-subjectID-x-sessionID-x-(scanID if running on a only a scan)-x-processname)
Put one folder for each resources you want to upload within the assessor folder with the name folder equal to the resource name.
Put the file you want to upload in it.
########################################################
# XNATPROCESSUPLOAD
#
# Developed by the masiLab Vanderbilt University, TN, USA.
# If issues, email benjamin.c.yvernault@vanderbilt.edu
# Parameters :
# No Arguments given
# Use "Xnatprocessupload -h" to see the options
########################################################
Usage: Xnatprocessupload [options]
What is the script doing : Upload Data on Xnat from a Directory as an Assessor.
Options:
-h, --help show this help message and exit
-d FOLDER_PATH, --directory=FOLDER_PATH
Directory containing the different assessors folders that you want to upload.
--force Force the upload.
Contact - benjamin.c.yvernault@vanderbilt.edu
XnatSubjectUpdate¶
Xnatsubjectupdate changes the last update date on XNAT to nothing. It will make the automatic process (in cci package when it’s setup) to run again on this subject.
########################################################
# XNATSUBJECTUPDATE
#
# Developed by the masiLab Vanderbilt University, TN, USA.
# If issues, email benjamin.c.yvernault@vanderbilt.edu
# Parameters :
# No Arguments given
# See the help bellow or Use "Xnatsubjectupdate" -h
########################################################
Usage: Xnatsubjectupdate [options]
What is the script doing : Query on Xnat at any level.
Options:
-h, --help show this help message and exit
-p PROJECT_ID, --project=PROJECT_ID
One project ID on Xnat.
-s SUBJECT_LABELS, --subject=SUBJECT_LABELS
Subject label on Xnat or list of them.
Contact - benjamin.c.yvernault@vanderbilt.edu
RedCapReport¶
Redcapreport is a powertool to extract data from REDCap. It will download the data and put it into a csv file. You can specify different options to have a precise download.
################################################################
# RedCapReport #
# #
# Developed by the MASI Lab Vanderbilt University, TN, USA. #
# If issues, please start a thread here: #
# https://groups.google.com/forum/#!forum/vuiis-cci #
# Usage: #
# Create REDCap report for a redcap project. #
# Examples: #
# Check the help for examples by running --help #
################################################################
----------------------------------------------------------------
usage: RedCapReport [-h] -k KEY [-c CSVFILE] [-x TXTFILE] [-p PROJECT]
[-s SUBJECT] [-e SESSION] [-a ASSESSOR] [-t PROCTYPE]
[-f PROCFILE] [-l LIBRARIES] [-F] [-L] [--all]
What is the script doing :
*Extract data from REDCap as a csv file.
Examples:
*Save the data in a csv file: Redcapreport -k KEY -c extract_redcap.csv
*print the libraries name: Redcapreport -k KEY -L
*print all fields name and label: Redcapreport -k KEY -F
*Extract values for all record: Redcapreport -k KEY --all
*Filter for specific project/subject/session/assessor type:
Redcapreport -k KEY -p PID -s 109387 -e 109387_1,109387_2 -t FS,TRACULA_v1,dtiQA_v2
*Extract for specific assessor: Redcapreport -k KEY -p PID -a PID-x-109387-x-109387_1-x-FS
*Extract for specific libraries type: Redcapreport -k KEY -p PID -l library_name
*Extract only the fields described in the txt file: Redcapreport -k KEY -x fields.txt
optional arguments:
-h, --help show this help message and exit
-k KEY, --key KEY API Token for REDCap project.
-c CSVFILE, --csvfile CSVFILE
csv file path where the report will be save.
-x TXTFILE, --txtfile TXTFILE
txt file path with per line, the name of the variable
on REDCap you want to extract.
-p PROJECT, --project PROJECT
Extract values for processes for the projects chosen.
E.G: project1,project2
-s SUBJECT, --subject SUBJECT
Extract values for processes for the subjects chosen.
E.G: subject1,subject2
-e SESSION, --session SESSION
Extract values for processes for the sessions chosen.
E.G: session1,session2
-a ASSESSOR, --assessor ASSESSOR
Extract values for processors chosen. E.G:
processor1,processor2
-t PROCTYPE, --proctype PROCTYPE
Extract values for processes types chosen. E.G:
fMRIQA,dtiQA
-f PROCFILE, --procfile PROCFILE
file path with each line one processor label. Extract
values for processes types chosen.
-l LIBRARIES, --libraries LIBRARIES
Extract values for only the libraries specify. Check
the project for the libraries name. Switch spaces by
'_' and everything lower case. E.G:
dti_quality_assurance. By default: all libraries
-F, --fields Print all field names and labels
-L, --printlib Print all libraries names for the project.
--all Extract values for all records.
Contact - benjamin.c.yvernault@vanderbilt.edu
XnatCheckLogin¶
XnatCheckLogin allows the user to check that environment variables are set appropriately. It will let you know in a few seconds if your logins are good or not.
usage: XnatCheckLogin [-h] [--host HOST]
Set and Check the logins for XNAT.
optional arguments:
-h, --help show this help message and exit
--host HOST Host for XNAT.
Xnatinfo¶
Xnatinfo is the tool to get fast statistics information on a project (number of subjects/sessions/scans/assessors and the status of the assessors). There is only one way to call Xnatinfo:
################################################################
# Xnatinfo #
# #
# Developed by the MASI Lab Vanderbilt University, TN, USA. #
# If issues, please start a thread here: #
# https://groups.google.com/forum/#!forum/vuiis-cci #
# Usage: #
# Display information on a XNAT project. #
# Examples: #
# Check the help for examples by running --help #
################################################################
----------------------------------------------------------------
usage: Xnatinfo [-h] [--host HOST] [-u USERNAME] [-x OUTPUT_FILE] [-f] [-r]
[--ignoreUnusable] [--ignoreScans]
project
What is the script doing :
* Generate a report for a XNAT project displaying scans/assessors
information.
Examples:
* See the information for project TEST:
Xnatinfo TEST
positional arguments:
project Project ID on XNAT
optional arguments:
-h, --help show this help message and exit
--host HOST Host for XNAT. Default: env XNAT_HOST.
-u USERNAME, --username USERNAME
Username for XNAT.
-x OUTPUT_FILE, --filetxt OUTPUT_FILE
Path to a txt file to save the report
-f, --failed Add this flag to print out failed jobs
-r, --running Add this flag to print out running jobs
--ignoreUnusable Ignore print statement of unusable scans
--ignoreScans Ignore print statement of scans
Xnatsessionupdate¶
Xnatsessionupdate resets the last update date on XNAT on a session. It will force DAX update scripts to update the session. This tool is for advanced users and managers of projects on XNAT.
################################################################
# XnatSessionUpdate #
# #
# Developed by the MASI Lab Vanderbilt University, TN, USA. #
# If issues, please start a thread here: #
# https://groups.google.com/forum/#!forum/vuiis-cci #
# Usage: #
# Reset sessions to be seen by the nex dax_update. #
# Examples: #
# Check the help for examples by running --help #
################################################################
----------------------------------------------------------------
usage: XnatSessionUpdate [-h] [--host HOST] [-u USERNAME] -p PROJECTS
[-s SESSION] [-n] [-x TXT_FILE] [-a]
What is the script doing :
* Reset sessions last update date to update the sessions during
the next dax_update.
Examples:
*Reset all sessions:
Xnatsessionupdate -p PID --all
*Reset some sessions :
Xnatsessionupdate -p PID -s 109374,109348
*Reset for the sessions that have assessors NEED_INPUTS:
Xnatsessionupdate -p PID -n
optional arguments:
-h, --help show this help message and exit
--host HOST Host for XNAT. Default: env XNAT_HOST.
-u USERNAME, --username USERNAME
Username for XNAT.
-p PROJECTS, --project PROJECTS
Projects ID on Xnat.
-s SESSION, --session SESSION
Session label on Xnat or list of them.
-n, --needinputs Change the subject last update date for all the subject with processes that have a job status equal to NEED_INPUTS.
-x TXT_FILE, --txtfile TXT_FILE
File txt with at each line the label of the assessor or just the Session label where the Subject date need to be changed. E.G for label: project-x-subject-x-experiment-x-scan-x-process_name.
-a, --all Change for all sessions.
BIDSMapping¶
BIDSMapping tool allows the user to create, update or replace rules/mapping at the project level on XNAT. These rules are essential as they entail the link between scan type or series description on XNAT to the BIDS datatype, task type and repetition time. XnatToBids function uses these mapping at the project to transform XNAT data into the BIDS compliant data with BIDS filenames and folder structure.
################################################################
# BIDSMAPPING #
# #
# Developed by the MASI Lab Vanderbilt University, TN, USA. #
# If issues, please start a thread here: #
# https://groups.google.com/forum/#!forum/vuiis-cci #
# Usage: #
# Upload rules/mapping to Project level on XNAT. #
# Examples: #
# Check the help for examples by running --help #
################################################################
usage: use "BIDSMapping --help" for more information
What is the script doing :
*Uploads BIDS datatype, tasktype and repitition time mapping to XNAT project level using the different OPTIONS.
Examples:
*Create a new datatype mapping for scan_type of XNAT scans:
BIDSMapping -p PID --xnatinfo scan_type --type datatype --create /tmp/projectID_datataype.csv
*The correct format for /tmp/projectID_datataype.csv
scan_type,datatype
Resting State,func
*Create a new datatype mapping for series_description of XNAT scans:
BIDSMapping -p PID --xnatinfo series_description --type datatype --create /tmp/projectID_datataype.csv
*Create a new tasktype mapping for scan_type of XNAT scans:
BIDSMapping -p PID --xnatinfo scan_type --type tasktype --create /tmp/projectID_tasktype.csv
*Replace tasktype mapping for scan_type of XNAT scans: (It removes the old mapping and upload the new mapping)
BIDSMapping -p PID --xnatinfo scan_type --type tasktype --replace /tmp/projectID_tasktype.csv
*Update tasktype mapping for scan_type of XNAT scans: (This is ONLY add new mapping rules, CANT remove rules use --replace to remove and add mapping rules)
BIDSMapping -p PID --xnatinfo scan_type --type tasktype --update /tmp/projectID_tasktype.csv
*Create default datatype mapping for scan_type of XNAT scans: (There is no default for series_description use --create)
BIDSMapping -p PID --xnatinfo scan_type --type datatype --create_default
*Download the current mapping on XNAT:
BIDSMapping -p PID --xnatinfo scan_type --type datatype --download /tmp/download.csv
*Download the scan_types on project on XNAT:
BIDSMapping -p PID --template /tmp/scan_type_template.csv
optional arguments:
-h, --help show this help message and exit
--host HOST Host for XNAT. Default: using $XNAT_HOST.
-u USERNAME, --username USERNAME
Username for XNAT. Default: using $XNAT_USER.
-o LOGFILE, --logfile LOGFILE
Write the display/output in a file given to this OPTIONS.
-p PROJECT, --project PROJECT
Project to create/update BIDS mapping file
-t TYPE, --type TYPE The type of mapping either datatype, tasktype or repetition_time_sec
-x XNATINFO, --xnatinfo XNATINFO
The type of xnat info to use for mapping either scan_type or series_description
-c CREATE, --create CREATE
Create the given BIDS new mapping file at project level. (EG. --create <mappingfile>.csv)
Default create creates the default mapping at project file. (EG. --create)
csvfile EG:
scan_type,datatype
T1W/3D/TFE,anat
Resting State,func
-cd, --create_default
Default create creates the default mapping at project file. (EG. --create_default)
-ud UPDATE, --update UPDATE
Update the existing BIDS mapping file at project level. (EG. --update <mappingfile>.csv)
This option can only add rules
-rp REPLACE, --replace REPLACE
Replace the existing BIDS mapping file at project level. (EG. --replace <mappingfile>.csv)
This option can remove and add new rules
-rv REVERT, --revert REVERT
Revert to an old mapping from a specific date/time. (EG: --revert 10-17-19-21:32:15
or --revert 10-17-19). Check the LOGFILE at project level for the date
-d DOWNLOAD, --download DOWNLOAD
Downloads the current BIDS mapping file (EG: --download <foldername>)
-tp TEMPLATE, --template TEMPLATE
Default mapping template (EG: --template <template file>)
For a walkthrough tutorial of BIDSMapping check out https://dax.readthedocs.io/en/latest/BIDS_walkthrough.html Contact - praitayini.kanakaraj@vanderbilt.edu
XnatBOND¶
XnatBOND takes in a BIDS directory and detects the Key and Parameter Groups. This tool can be used to Modifying Key and Parameter Group Assignment. For more details on the package used look at https://bids-bond.readthedocs.io/en/latest/readme.html
################################################################
# XnatBond #
# #
# Developed by the MASI Lab Vanderbilt University, TN, USA. #
# If issues, please start a thread here: #
# https://groups.google.com/forum/#!forum/vuiis-cci #
# Usage: #
# Generate and alternate key params in BIDS using BOND #
# Examples: #
# Check the help for examples by running --help #
################################################################
usage: XnatBOND [-h] --bids_dir BIDS_DIR [-b BOND_DIR] [-m keyparam_edited keyparam_files new_keyparam_prefix] [-o LOGFILE]
What is the script doing :
*Generate the csv files that have the summary of key groups and param groups from the
bidsdata and modify them in the bids data.
Examples:
*Generate orginial key and parameter groups:
XnatBOND --bids_dir BIDS_DIR --bond_dir BOND_DIR
*Update the key and parameter groups:
XnatBOND --bids_dir BIDS_DIR --modify_keyparam
optional arguments:
-h, --help show this help message and exit
--bids_dir BIDS_DIR BIDS data directory.
-b BOND_DIR, --bond_dir BOND_DIR
BOND data directory.
-m keyparam_edited keyparam_files new_keyparam_prefix, --modify_keyparam keyparam_edited keyparam_files new_keyparam_prefix
Values to modify the keyparam in bids.
-o LOGFILE, --logfile LOGFILE
Write the display/output in a file given to this OPTIONS.