DAX Command Line Tools ====================== Table of Contents ~~~~~~~~~~~~~~~~~ 1. `List of the Tools <#list-of-the-tools>`__ 1. `XnatSetup <#xnatsetup>`__ 2. `XnatQuery <#xnatquery>`__ 3. `XnatCheck <#xnatcheck>`__ 4. `XnatDownload <#xnatdownload>`__ 5. `XnatUpload <#xnatupload>`__ 6. `XnatReport <#xnatreport>`__ 7. `XnatSwitchProcessStatus <#xnatswitchprocessstatus>`__ 8. `XnatSetup <#xnatsetup>`__ 9. `XnatProcessUpload <#xnatprocessupload>`__ 10. `XnatSubjectUpdate <#xnatsubjectupdate>`__ 11. `RedCapReport <#redcapreport>`__ 12. `XnatCheckLogin <#xnatchecklogin>`__ 13. `Xnatinfo <#xnatinfo>`__ 14. `Xnatsessionupdate <#xnatsessionupdate>`__ 15. `BIDSMapping <#bidsmapping>`__ 16. `XnatBOND <#xnatbond>`__ List of the Tools ~~~~~~~~~~~~~~~~~ Each tool has a help option and some examples on how to use the tools. You can call each tool with no arguments to see the help. ---------- XnatSetup ---------- You can use the Xnatsetup (see below) command tool to setup your computer with the --basic options. It will do what is below automatically, but if you don't want to do that, it can be setup manually. This Xnat commands will use two thing : - install pyxnat and python packages on your computer (Check 'Get started for Spiders' on the wiki) - set your bashrc with the env variable to connect to Xnat with pyxnat : :: export XNAT_HOST=http://xnat.vanderbilt.edu/xnat export XNAT_USER=username export XNAT_PASS=password export PATH=/PathToMasimatlab/trunk/xnatspiders/Xnat_tools:$PATH FYI : you can open the bashrc like : :: vim (or nano or any editor you like) ~/.bashrc and when you are done editing it, use : :: . ~/.bashrc You will after this be able to call the commands directly on your terminal. One last thing, the Xnatupload will send you updates (errors and warnings) about the directory you are trying to upload on Xnat. If you want to receive this email, you need to set up two variables in your bashrc : :: export EMAIL_ADDR=add@gmail.com export EMAIL_PWS=passwordforthisemail It will use this email address to send you email. It has to be a gmail address. Xnatsetup is as you can guess a command tool to set up your computer. It will install the python package needed and ask for the variables that need to be set up. There are different kinds of setup : - basic to be able to use the XNAT command tools - advance to run spiders on your computer or ACCRE - redcap to use the spider to send data to redcap - cci package setup for ACCRE or if you need XnatUtils - ACCRE setup :: ####################################################### # XNATSETUP # XnatSetup is a command tool to set up on your computer the variables to use # the tools/spiders. # Developed by the masiLab Vanderbilt University, TN, USA. # Operating system : Linux & Mac OS # Shell : bash # Requirements : python with pip & git # Contact : benjamin.c.yvernault@vanderbilt.edu # No Arguments given # See the help bellow or Use "Xnatquery" -h ####################################################### Usage: Xnatsetup [options] What is the script doing : Set up your computer to use xnat. *Basic installation (--basic) - Needed to use the Xnat command tools or any of the next installations : install the python package httplib2, lxml, and pyxnat if not already install & saving your username/host/password for XNAT. *Advance installation (--advance ) - Needed to run the non-specific spiders : Set up the Upload Directory, set up masimatlab path for Xnatrun, add the xnat tools to your PATH, and add Spiders.py in your PYTHONPATH . *Redcap installation (--redcap ) - Needed to use redcap spiders (send data to redcap) : Install Pycap / pandas if not install and set up the URL for redcap . *API installation (--api ) - Needed to use API package to run spiders on ACCRE via jobs (Contains XnatUtils) : Install API if not install. *ACCRE installation (--accre) - Setup the environment to use the package/spiders/tools on ACCRE. Options: -h, --help show this help message and exit --basic Use this options to set up the env variables to use the Xnat tools and have the basic set up. --advance Use this options to set up the env variables to run spiders in general. --redcap Use this options to set up the env variables to use redcap spiders. --api Use this options to set up the env variables to run spiders on ACCRE via jobs. --Accre Use this options if you are on Accre. --NoSudo Use this options if you don't have sudo access and you still want to install the package (check -d option). -d INSTALLDIR, --installdir=INSTALLDIR Use this options to specify a directory where the python package need to be install. It works only if you use --NoSudo option. --tutorial Give you the step for the specific setup you are asking. Contact - benjamin.c.yvernault@vanderbilt.edu --------- XnatQuery --------- Xnatquery will show you the tree on xnat. Xnatquery is a tool to query objects on XNAT for each level. You can see which projects you have access to and see the hierarchy of data on your project. It has several options (accessible with -h or --help) : :: ################################################################ # XnatQuery # # # # Developed by the MASI Lab Vanderbilt University, TN, USA. # # If issues, please start a thread here: # # https://groups.google.com/forum/#!forum/vuiis-cci # # Usage: # # Query through XNAT at the level you want. # # Examples: # # Check the help for examples by running --help # ################################################################ ---------------------------------------------------------------- usage: XnatQuery [-h] [--host HOST] [-u USERNAME] [-p PROJECT] [-s SUBJECT] [-e SESSION] [-a ASSESSOR] [-c SCAN] [--all] [--me] What is the script doing : * Query on Xnat at any level. Examples: *Show all the projects you have access to: Xnatquery --me *Show all projects: Xnatquery --all *Query a specific level (example scan/assessors for a session): Xnatquery -p PID -s 109873 -e 109873 *Query a specific level with all objects under it : Xnatquery -p PID -s 109873 --all optional arguments: -h, --help show this help message and exit --host HOST Host for XNAT. Default: env XNAT_HOST. -u USERNAME, --username USERNAME Username for XNAT. -p PROJECT, --project PROJECT project ID on Xnat or 'all' to see all the project. -s SUBJECT, --subject SUBJECT Subject label on Xnat -e SESSION, --experiment SESSION Session label on Xnat -a ASSESSOR, --assessor ASSESSOR Assessor/Process label on XNAT. E.G: VUSTP-x-VUSTP1-x-VUSTP1a-x-FS -c SCAN, --scan SCAN Scan ID on Xnat. --all Print all the objects on XNAT from the level you are at. --me Give the projects ID that you have access. **Extra Examples** - To get information on the project :: Xnatquery -p projectID --info - To get all the subjects in this project :: Xnatquery -p projectID - To get all the experiments in this project :: Xnatquery -p projectID -s subject Contact - benjamin.c.yvernault@vanderbilt.edu --------- XnatCheck --------- Xnatcheck is a quick way to check directly on your terminal if there is the resource you just created on all your project. You can check if there is a scan type or an assessor type as well with the options -s or -a. Options available (-h or -help): :: ################################################################ # XnatCheck # # # # Developed by the MASI Lab Vanderbilt University, TN, USA. # # If issues, please start a thread here: # # https://groups.google.com/forum/#!forum/vuiis-cci # # Usage: # # Check XNAT data (subject/session/scan/assessor/resource) # # Examples: # # Check the help for examples by running --help # ################################################################ ---------------------------------------------------------------- usage: XnatCheck [-h] [--host HOST] [-u USERNAME] [-p PROJECTS] [--filters FILTERS [FILTERS ...]] [--delimiter DELIMITER_FILTER_RESOURSE] [--csv CSV_FILE] [--format FORMAT] [--printfilters] [--printformat] What is the script doing : *Check object on XNAT (subject/session/scan/assessor/resources) specify by the options. How to write a filter string: - for resources filters, the string needs to follow this template: variable_name=value--sizeoperatorValue--nbfoperatorValue--fpathsoperatorValue By default, it will return the assessor that does have the resource if no other filter specify - for other filters, the string needs to follow this template: variable_name=Value operator can be different than =. Look at the table in --printfilters Use --printfilters to see the different variables available Examples: *See format variables: Xnatcheck --printformat *See filter variables: Xnatcheck --printfilters *Get list of T1,DTI scans that have a resource called NIFTI: Xnatcheck -p PID --filters type=T1,DTI assessor_res=NIFTI *Get list of fMRIQA assessors that have a resource called PDF: Xnatcheck -p PID --filters proctype=fMRIQA assessor_res=PDF *Get list of assessors except fMRIQA that have a resource called PDF : Xnatcheck -p PID --filters proctype!=fMRIQA assessor_res=PDF *Get list of project sessions that do not have a resource called testing: Xnatcheck -p PID --filters session_label=VUSTP1a,VUSTP2b,VUSTP3a session_res!=testing *Get list of project fMRIQA and VBMQA that used more than 45mb and less than 1hour: Xnatcheck -p PID1,PID2 --filters proctype=fMRIQA,VBMQA procstatus=COMPLETE "memused>45mb" "walltimeused<1:00:00" --format assessor_label,procnode,memused,walltimeused optional arguments: -h, --help show this help message and exit --host HOST Host for XNAT. Default: env XNAT_HOST. -u USERNAME, --username USERNAME Username for XNAT. -p PROJECTS, --project PROJECTS Project(s) ID on XNAT --filters FILTERS [FILTERS ...] List of filters separated by a space to apply to the search. --delimiter DELIMITER_FILTER_RESOURSE Resource filters delimiter. By default: --. --csv CSV_FILE File path to save the CSV output. --format FORMAT Header for the csv. format: list of variables name comma-separated --printfilters Print available filters. --printformat Print available format for display. **Extra Examples** - To return all the scans where there is no NIFTI on the project BLSA :: Xnatcheck -p BLSA -r NIFTI - To return all the assessors where there is no PDF on the project BLSA :: Xnatcheck -p BLSA -r PDF -l 1 - To return all the subjects/experiments where there is no fMRIQA assessor on the project BLSA :: Xnatcheck -p BLSA -a fMRIQA - To return all the subjects/experiments where there is no fMRIQA assessor on the project BLSA and check for the one that exists if there is a PDF resource :: Xnatcheck -p BLSA -a fMRIQA -r PDF Contact - benjamin.c.yvernault@vanderbilt.edu ------------ XnatDownload ------------ Xnatdownload will download all the resources that you asked for in a directory. Xnatdownload provides bulk download of data from XNAT with specific filters applied. It provides options to narrow your download to only what you need. This tool will generate a folder per project in your -d directory with two files: download_commandLine.txt and download_report.csv with the description of what you downloaded. It has several options (accessible with -h or -help) : :: ######################################################## # XNATDOWNLOAD # # Developed by the masiLab Vanderbilt University, TN, USA. # If issues, email benjamin.c.yvernault@vanderbilt.edu # Parameters : # No Arguments given # See the help bellow or Use "Xnatdownload" -h ######################################################## usage: Xnatdownload [-h] [--host HOST] [-u USERNAME] [-p PROJECT] [-d DIRECTORY] [-D] [--subj SUBJECT] [--sess SESSION] [-s SCANTYPE] [-a ASSESSORTYPE] [--WOS WITHOUTS] [--WOP WITHOUTA] [--quality QUALITIES] [--status STATUS] [--qcstatus QCSTATUS] [-c CSVFILE] [--rs RESOURCESS] [--ra RESOURCESA] [--selectionS SELECTIONSCAN] [--selectionP SELECTIONASSESSOR] [--overwrite] [--update] [--fullRegex] [-o OUTPUTFILE] [-i] [-b BIDS_DIR] [-xt] [--bond_dir BOND_DIR] What is the script doing : *Download filtered data from XNAT to your local computer using the different OPTIONS. Examples: *Download all resources for all scans/assessors in a project: Xnatdownload -p PID -d /tmp/downloadPID -s all --rs all -a all --ra all *Download NIFTI for T1,fMRI: Xnatdownload -p PID -d /tmp/downloadPID -s T1,fMRI --rs NIFTI *Download only the outlogs for fMRIQA assessors that failed: Xnatdownload -p PID -d /tmp/downloadPID -a fMRIQA --status JOB_FAILED --ra OUTLOG *Download PDF for assessors that Needs QA: Xnatdownload -p PID -d /tmp/downloadPID -a all --qcstatus="Needs QA" --ra OUTLOG *Download NIFTI for T1 for some sessions : Xnatdownload -p PID -d /tmp/downloadPID --sess 109309,189308 -s all --rs NIFTI *Download same data than previous line but overwrite the data: Xnatdownload -p PID -d /tmp/downloadPID --sess 109309,189308 -s all --rs NIFTI --overwrite *Download data described by a csvfile (follow template) : Xnatdownload -d /tmp/downloadPID -c upload_sheet.csv *Transform the XnatDownload data in BIDS format for all sessions, scantype and resources: Xnatdownload -p PID --sess all -d /tmp/downloadPID -s all --rs all --bids /tmp/BIDS_dataset *Transform the XnatDownload data in BIDS format for all sessions, scantype and resources with xnat tag: Xnatdownload -p PID --sess all -d /tmp/downloadPID -s all --rs all --bids /tmp/BIDS_dataset -xt *Transform the XnatDownload data in BIDS format for all sessions, scantype and resources with xnat tag and perform bond: Xnatdownload -p PID --sess all -d /tmp/downloadPID -s all --rs all --bids /tmp/BIDS_dataset -xt --bond /tmp/BOND_dir optional arguments: -h, --help show this help message and exit --host HOST Host for XNAT. Default: using $XNAT_HOST. -u USERNAME, --username USERNAME Username for XNAT. Default: using $XNAT_USER. -p PROJECT, --project PROJECT Project(s) ID on Xnat -d DIRECTORY, --directory DIRECTORY Directory where the data will be download -D, --oneDirectory Data will be downloaded in the same directory. No sub- directory. --subj SUBJECT filter scans/assessors by their subject label. Format: a comma separated string (E.G: --subj VUSTP2,VUSTP3). --sess SESSION filter scans/assessors by their session label. Format: a comma separated string (E.G: --sess VUSTP2b,VUSTP3a) -s SCANTYPE, --scantype SCANTYPE filter scans by their types (required to download scans). Format: a comma separated string (E.G : -s T1,MPRAGE,REST). To download all types, set to 'all'. -a ASSESSORTYPE, --assessortype ASSESSORTYPE filter assessors by their types (required to download assessors). Format: a comma separated string (E.G : -a fMRIQA,dtiQA_v2,Multi_Atlas). To download all types, set to 'all'. --WOS WITHOUTS filter scans by their types and removed the one with the specified types. Format: a comma separated string (E.G : --WOS T1,MPRAGE,REST). --WOP WITHOUTA filter assessors by their types and removed the one with the specified types. Format: a comma separated string (E.G : --WOP fMRIQA,dtiQA). --quality QUALITIES filter scans by their quality. Format: a comma separated string (E.G: --quality usable,questionable,unusable). --status STATUS filter assessors by their job status. Format: a comma separated string. --qcstatus QCSTATUS filter assessors by their quality control status. Format: a comma separated string. -c CSVFILE, --csvfile CSVFILE CSV file with the following header: object_type,projec t_id,subject_label,session_type,session_label,as_label . object_type must be 'scan' or 'assessor' and as_label the scan ID or assessor label. --rs RESOURCESS Resources you want to download for scans. E.g : --rs NIFTI,PAR,REC. --ra RESOURCESA Resources you want to download for assessors. E.g : --ra OUTLOG,PDF,PBS. --selectionS SELECTIONSCAN Download from only one selected scan.By default : no selection. E.G : project-x-subject-x-session-x-scan --selectionP SELECTIONASSESSOR Download from only one selected processor.By default : no selection. E.G : assessor_label --overwrite Overwrite the previous data downloaded with the same command. --update Update the files from XNAT that have been downloaded with the newest version if there is one (not working yet). --fullRegex Use full regex for filtering data. -o OUTPUTFILE, --output OUTPUTFILE Write the display in a file giving to this OPTIONS. -i, --ignore Ignore reading of the csv report file -b BIDS_DIR, --bids BIDS_DIR Directory to store the XNAT to BIDS curated data -xt, --xnat_tag Download BIDS data with XNAT subjID and sessID --bond_dir BOND_DIR Download the Key groups and Param groups in BIDS data to BOND_DIR **Example** - Downloads in /home/benjamin/temp the resources NIFTI and PDF for all the scan fMRI and the assessor fMRIQA for the project BLSA :: Xnatdownload -p BLSA -d /home/benjamin/temp/ -a fMRIQA -s fMRI -r NIFTI,PDF Contact - benjamin.c.yvernault@vanderbilt.edu ---------- XnatUpload ---------- Xnatupload will create subject/experiment/scan/resources for a project on XNAT and upload the data into the project from a folder. Xnatupload provides bulk upload of data to a project on XNAT. You need to provide a specific CSV file with the following header: - object_type,project_id,subject_label,session_type,session_label,as_label,as_type,as_description,quality,resource,fpath where: - as_label corresponds to assessor or scan label - as_type corresponds to proctype or scantype - as_description corresponds to procstatus or series description for the scan - quality corresponds to qastatus or quality for scan It should be similar to this (project in the example is CIBS-TEST): object_type,project_id,subject_label,session_type,session_label,as_label,as_type,as_description,quality,resource,fpath scan,CIBS-TEST,CIBS-TEST_01,MR,CIBS-TEST_01,401,BRAIN2_3DT1,BRAIN2_3DT1,usable,NIFTI,/Users//Downloads/Archive/DICOM_T1W_3D_TFE.nii.gz **Methods** Warning: the project must already exist on XNAT. You can add a new project via the XNAT web GUI. Follow one of the three methods to upload: - Number 1 : all the files are in one directory but they need to be rename like this projectID-x-subjectID-x-experimentID-x-scanID-x-scantype-x-resourcename.extention. Fastest methode but only one file can be upload in a resource. - Number 2 : you don't need to rename all the files but you need to give a specific structure to your directory : folder/subjectID/experimentID/scanID-x-scantype/ResourceID/ and put the resources corresponding in it. E.G : TempDir/BLSA_0000/BLSA_0000_0/scan2-x-fMRI/NIFTI/nifti.nii.gz. It will not be as fast as the first methode but you can upload more than one file to a resources. - Option -o : There is a third choice. If you want to upload files to Xnat on a scan and you don't want to create anything, you can use this options -o. It's for only upload. It's using something like the first methodes : put all the files into one folder with a special name : projectID-x-subjectID-x-experimentID-x-scanID-x-resourcename.extention for assessor, assessor_label-resourcename.extension for assessor (Reminder : assessor_label = projectID-x-subjectID-x-experimentID-x-scanID-x-process_name or projectID-x-subjectID-x-experimentID-x-processname). :: ################################################################ # XnatUpload # # # # Developed by the MASI Lab Vanderbilt University, TN, USA. # # If issues, please start a thread here: # # https://groups.google.com/forum/#!forum/vuiis-cci # # Usage: # # Print a detailed report from XNAT projects. # # Examples: # # Check the help for examples by running --help # ################################################################ IMPORTANT WARNING FOR ALL USERS ABOUT XNAT: session_label needs to be unique for each session. Two subjects can NOT have the same session_label ---------------------------------------------------------------- usage: XnatUpload [-h] [--host HOST] [-u USERNAME] -c CSV_FILE [--sess SESSION_TYPE] [--report] [--force] [--delete] [--deleteAll] [--noextract] [--printmodality] [-o OUTPUT_FILE] [-b BIDS_DIR] [-p PROJECT] What is the script doing : * Upload data to XNAT following the csv file information. csv header: object_type,project_id,subject_label,session_type,session_label, as_label,as_type,as_description,quality,resource,fpath IMPORTANT: YOU NEED TO CREATE THE PROJECT ON XNAT BEFORE UPLOADING. Examples: * See Session type: Xnatupload --printmodality * Simple upload: Xnatupload -c upload_sheet.csv * Upload everything with a session type: Xnatupload -c upload_sheet.csv --sess PET * Check the upload: Xnatupload -c upload_sheet.csv --report * Force upload: Xnatupload -c upload_sheet.csv --force * Upload with delete resource before uploading: Xnatupload -c upload_sheet.csv --delete * Upload with delete every resources for the object (SCAN/ASSESSOR) before uploading: Xnatupload -c upload_sheet.csv --deleteAll * Upload BIDS data to XNAT format for scan Xnatupload -b /tmp/bidsDataset -p PID * Check BIDS data to XNAT Xnatupload -b /tmp/bidsDataset -p PID --report * Force upload BIDS data to upload XNAT Xnatupload -b /tmp/bidsDataset -p PID --force optional arguments: -h, --help show this help message and exit --host HOST Host for XNAT. Default: env XNAT_HOST. -u USERNAME, --username USERNAME Username for XNAT. -c CSV_FILE, --csv CSV_FILE CSV file with the information for uploading data to XNAT. Header: object_type,project_id,subject_label,session_type,session_label,as_label,as_type,as_description,as_quality,resource,fpath --sess SESSION_TYPE Session type on Xnat. Use printmodality to see the options. --report Print a report to verify inputs. --force Force the upload and remove previous resources. --delete Delete resource files prior to upload. --deleteAll Delete all resources in object prior to upload. --noextract Do not extract the zip files on XNAT when uploading a folder. --printmodality Display the different modality available on XNAT for a session. -o OUTPUT_FILE, --output OUTPUT_FILE File path to store the script logs. -b BIDS_DIR, --bids BIDS_DIR BIDS Directory to convert to XNAT and then upload -p PROJECT, --project PROJECT Project for BIDS XNAT upload **Extra Examples** - Shows on the terminal what kind of data the command is going to upload and where with method 1 :: Xnatupload -d /Path/to/directory --report --up1 - Uploads the files in the directory with the first method :: Xnatupload -p projectID -d /Path/to/directory -up1 -sess MR - Uploads the files in the directory with the second method :: Xnatupload -p projectID -d /Path/to/directory --up2 --sess CT - Uploads (only, no creation of subject/exp/scan) all the files from the directory into Xnat even if there is already a resources (options -force) :: Xnatupload -d /Path/to/directory -o -T 1 --force Contact - benjamin.c.yvernault@vanderbilt.edu ---------- XnatReport ---------- Xnatreport will give you a report on one ore more projects. It will show all the subjects/sessions/scans/assessors/resources for the projects chosen. It has several options (accessible with -h or -help) : :: ################################################################ # XnatReport # # # # Developed by the MASI Lab Vanderbilt University, TN, USA. # # If issues, please start a thread here: # # https://groups.google.com/forum/#!forum/vuiis-cci # # Usage: # # Print a detailed report from XNAT projects. # # Examples: # # Check the help for examples by running --help # ################################################################ ---------------------------------------------------------------- usage: XnatReport [-h] [--host HOST] [-u USERNAME] [-p PROJECTS] [-c CSV_FILE] [--format FORMAT] [--printformat] What is the script doing : * Create a report about Xnat projects. Examples: *Report of a project: Xnatreport -p PID *Report with a specific format: Xnatreport -p PID --format object_type,session_id,session_label,age *print the format available: Xnatreport --printformat *Save report in a csv: Xnatreport -p PID -c report.csv optional arguments: -h, --help show this help message and exit --host HOST Host for XNAT. Default: env XNAT_HOST. -u USERNAME, --username USERNAME Username for XNAT. -p PROJECTS, --project PROJECTS List of project ID on Xnat separate by a coma -c CSV_FILE, --csvfile CSV_FILE csv fullpath where to save the report. --format FORMAT Header for the csv. format: variables name separated by comma. --printformat Print available variables names for the option --format. **Extra Examples** - Creates a report for BLSA and CTONS and will print it on the screen/terminal :: Xnatreport -p BLSA,CTONS - Sends the report on BLSA and CTONS to your email address as a csv file. You need to set to variables gmail address and password used to sent the email in your bashrc :: Xnatreport -p BLSA,CTONS --csv -e email@email.com - Writes the report as a ".csv" file that can be open with Excel. If not path specify, /tmp is the place where the report is save. -t will do the same but in a text file :: Xnatreport -p BLSA,CTONS --csv Contact - benjamin.c.yvernault@vanderbilt.edu ----------------------- XnatSwitchProcessStatus ----------------------- XnatSwitchProcessStatus is one of the most powerful and used of the Xnat_tools. It allows the user to switch/set the procstatus (job status) for a specific proctype (type of assessor) in a project. XnatSwitchProcessStatus allows the user to change the status of several type of assessors in a project that have a specific type or just for all of them. :: ################################################################ # XnatSwitchProcessStatus # # # # Developed by the MASI Lab Vanderbilt University, TN, USA. # # If issues, please start a thread here: # # https://groups.google.com/forum/#!forum/vuiis-cci # # Usage: # # Change assessor job/quality control status. # # Examples: # # Check the help for examples by running --help # ################################################################ ---------------------------------------------------------------- usage: XnatSwitchProcessStatus [-h] [--host HOST] [-u USERNAME] [--select SELECT] [-x TXT_FILE] [-p PROJECTS] [--subj SUBJECTS] [--sess SESSIONS] [-s STATUS] [-f FORMER_STATUS] [-t PROCTYPES] [-n NEED_INPUTS] [-d] [--qc] [--printstatus] [--fullRegex] [--restart] [--rerun] [--init] [--rerundiskq] What is the script doing : *Switch/Set the status for assessors on XNAT selected by the proctype. Examples: *See status managed by DAX: XnatSwitchProcessStatus --printstatus *Set all fMRIQA to a specific status Error for a project: XnatSwitchProcessStatus -p PID -s Error -t fMRIQA *Set all Multi_Atlas that have the status JOB_FAILED to NEED_TO_RUN to have the processes run again: XnatSwitchProcessStatus -p PID -f JOB_FAILED -t Multi_Atlas -s NEED_TO_RUN *Set all VBMQA to NEED_TO_RUN for a project and delete resources: XnatSwitchProcessStatus -p PID -s NEED_TO_RUN -t VBMQA -d *Set all VBMQA to NEED_TO_RUN, delete resources, and set linked assessors fMRI_Preprocess to NEED_INPUTS: XnatSwitchProcessStatus -p PID -s NEED_TO_RUN -t VBMQA -d -n fMRI_Preprocess *Set all dtiQA_v2 qa status to Passed for a project: XnatSwitchProcessStatus -p PID -s Passed -t dtiQA_v2 --qc *Set FreeSurfer for a specific project/subject to NEED_INPUTS: XnatSwitchProcessStatus -p PID --subj 123 -s NEED_INPUTS -t FreeSurfer optional arguments: -h, --help show this help message and exit --host HOST Host for XNAT. Default: env XNAT_HOST. -u USERNAME, --username USERNAME Username for XNAT. --select SELECT Give the assessor label that you want to change the status. -x TXT_FILE, --txtfile TXT_FILE File txt. Each line represents the label of the assessor which need to change status. -p PROJECTS, --project PROJECTS Project ID on XNAT or list of Project ID --subj SUBJECTS Change Status for only this subject/list of subjects. --sess SESSIONS Change Status for only this session/list of sessions. -s STATUS, --status STATUS Status you want to set on the Processes. E.G: 'NEED_TO_RUN' -f FORMER_STATUS, --formerStatus FORMER_STATUS Change assessors with this former status. E.G: 'JOB_FAILED' -t PROCTYPES, --type PROCTYPES Assessor process type you want the status to changed. -n NEED_INPUTS, --Needinputs NEED_INPUTS Assessor process type that need to change to NEED_INPUTS because the assessors from -t you changed are inputs to those assessors. -d, --deleteR Delete the resources on the assessor. --qc Change the quality control status on XNAT. --printstatus Print status used by DAX to manage assessors. --fullRegex Use full regex for filtering data. --restart Restart the assessors by switching the status for all assessors found to NEED_TO_RUN and delete previous resources. --rerun Rerun the assessors by switching status to NEED_TO_RUN for assessors that failed and delete previous resources. --init Init the assessors by switching status to NEED_INPUTS for assessors that have been set to NO_DATA. --rerundiskq Rerun the assessor that have the status JOB_FAILED: switching status to NEED_INPUTS from JOB_FAILED and delete previous resources. **Extra Examples** - Changes the status for dtiQA_v2 and Freesurfer that have a Failed status to NeedToRun in project BLSA :: XnatSwitchProcessStatus -p BLSA -f Failed -s NeedToRun -t dtiQA_v2,FreeSurfer - Changes the status for dtiQA_v2 and Freesurfer that have a Failed status to NeedToRun in project BLSA and it will delete all the resources on the assessor :: XnatSwitchProcessStatus -p BLSA -f Failed -s NeedToRun -t dtiQA_v2,FreeSurfer -d - Changes the status for the specific FreeSurfer assessor in BLSA_0000_00 session to NeedToRun and delete the resources :: XnatSwitchProcessStatus --select BLSA-x-BLSA_0000-x-BLSA_0000_00-x-FreeSurfer -s NeedToRun -d Contact - benjamin.c.yvernault@vanderbilt.edu ----------------- XnatProcessUpload ----------------- Xnatprocessupload allows you to upload data for an assessor (you can't do it that with Xnatupload). You only need to give the path to the folder where the data are. If the assessor doesn't exist, it will create one. You need to organize the data like this : 1) One folder per assessor you want to upload, the name of the folder needs to be the name of the assessor (Remember: assessor label = projectID-x-subjectID-x-sessionID-x-(scanID if running on a only a scan)-x-processname) 2) Put one folder for each resources you want to upload within the assessor folder with the name folder equal to the resource name. 3) Put the file you want to upload in it. :: ######################################################## # XNATPROCESSUPLOAD # # Developed by the masiLab Vanderbilt University, TN, USA. # If issues, email benjamin.c.yvernault@vanderbilt.edu # Parameters : # No Arguments given # Use "Xnatprocessupload -h" to see the options ######################################################## Usage: Xnatprocessupload [options] What is the script doing : Upload Data on Xnat from a Directory as an Assessor. Options: -h, --help show this help message and exit -d FOLDER_PATH, --directory=FOLDER_PATH Directory containing the different assessors folders that you want to upload. --force Force the upload. Contact - benjamin.c.yvernault@vanderbilt.edu ----------------- XnatSubjectUpdate ----------------- Xnatsubjectupdate changes the last update date on XNAT to nothing. It will make the automatic process (in cci package when it's setup) to run again on this subject. :: ######################################################## # XNATSUBJECTUPDATE # # Developed by the masiLab Vanderbilt University, TN, USA. # If issues, email benjamin.c.yvernault@vanderbilt.edu # Parameters : # No Arguments given # See the help bellow or Use "Xnatsubjectupdate" -h ######################################################## Usage: Xnatsubjectupdate [options] What is the script doing : Query on Xnat at any level. Options: -h, --help show this help message and exit -p PROJECT_ID, --project=PROJECT_ID One project ID on Xnat. -s SUBJECT_LABELS, --subject=SUBJECT_LABELS Subject label on Xnat or list of them. Contact - benjamin.c.yvernault@vanderbilt.edu ------------ RedCapReport ------------ Redcapreport is a powertool to extract data from REDCap. It will download the data and put it into a csv file. You can specify different options to have a precise download. :: ################################################################ # RedCapReport # # # # Developed by the MASI Lab Vanderbilt University, TN, USA. # # If issues, please start a thread here: # # https://groups.google.com/forum/#!forum/vuiis-cci # # Usage: # # Create REDCap report for a redcap project. # # Examples: # # Check the help for examples by running --help # ################################################################ ---------------------------------------------------------------- usage: RedCapReport [-h] -k KEY [-c CSVFILE] [-x TXTFILE] [-p PROJECT] [-s SUBJECT] [-e SESSION] [-a ASSESSOR] [-t PROCTYPE] [-f PROCFILE] [-l LIBRARIES] [-F] [-L] [--all] What is the script doing : *Extract data from REDCap as a csv file. Examples: *Save the data in a csv file: Redcapreport -k KEY -c extract_redcap.csv *print the libraries name: Redcapreport -k KEY -L *print all fields name and label: Redcapreport -k KEY -F *Extract values for all record: Redcapreport -k KEY --all *Filter for specific project/subject/session/assessor type: Redcapreport -k KEY -p PID -s 109387 -e 109387_1,109387_2 -t FS,TRACULA_v1,dtiQA_v2 *Extract for specific assessor: Redcapreport -k KEY -p PID -a PID-x-109387-x-109387_1-x-FS *Extract for specific libraries type: Redcapreport -k KEY -p PID -l library_name *Extract only the fields described in the txt file: Redcapreport -k KEY -x fields.txt optional arguments: -h, --help show this help message and exit -k KEY, --key KEY API Token for REDCap project. -c CSVFILE, --csvfile CSVFILE csv file path where the report will be save. -x TXTFILE, --txtfile TXTFILE txt file path with per line, the name of the variable on REDCap you want to extract. -p PROJECT, --project PROJECT Extract values for processes for the projects chosen. E.G: project1,project2 -s SUBJECT, --subject SUBJECT Extract values for processes for the subjects chosen. E.G: subject1,subject2 -e SESSION, --session SESSION Extract values for processes for the sessions chosen. E.G: session1,session2 -a ASSESSOR, --assessor ASSESSOR Extract values for processors chosen. E.G: processor1,processor2 -t PROCTYPE, --proctype PROCTYPE Extract values for processes types chosen. E.G: fMRIQA,dtiQA -f PROCFILE, --procfile PROCFILE file path with each line one processor label. Extract values for processes types chosen. -l LIBRARIES, --libraries LIBRARIES Extract values for only the libraries specify. Check the project for the libraries name. Switch spaces by '_' and everything lower case. E.G: dti_quality_assurance. By default: all libraries -F, --fields Print all field names and labels -L, --printlib Print all libraries names for the project. --all Extract values for all records. Contact - benjamin.c.yvernault@vanderbilt.edu -------------- XnatCheckLogin -------------- XnatCheckLogin allows the user to check that environment variables are set appropriately. It will let you know in a few seconds if your logins are good or not. :: usage: XnatCheckLogin [-h] [--host HOST] Set and Check the logins for XNAT. optional arguments: -h, --help show this help message and exit --host HOST Host for XNAT. -------- Xnatinfo -------- Xnatinfo is the tool to get fast statistics information on a project (number of subjects/sessions/scans/assessors and the status of the assessors). There is only one way to call Xnatinfo: :: ################################################################ # Xnatinfo # # # # Developed by the MASI Lab Vanderbilt University, TN, USA. # # If issues, please start a thread here: # # https://groups.google.com/forum/#!forum/vuiis-cci # # Usage: # # Display information on a XNAT project. # # Examples: # # Check the help for examples by running --help # ################################################################ ---------------------------------------------------------------- usage: Xnatinfo [-h] [--host HOST] [-u USERNAME] [-x OUTPUT_FILE] [-f] [-r] [--ignoreUnusable] [--ignoreScans] project What is the script doing : * Generate a report for a XNAT project displaying scans/assessors information. Examples: * See the information for project TEST: Xnatinfo TEST positional arguments: project Project ID on XNAT optional arguments: -h, --help show this help message and exit --host HOST Host for XNAT. Default: env XNAT_HOST. -u USERNAME, --username USERNAME Username for XNAT. -x OUTPUT_FILE, --filetxt OUTPUT_FILE Path to a txt file to save the report -f, --failed Add this flag to print out failed jobs -r, --running Add this flag to print out running jobs --ignoreUnusable Ignore print statement of unusable scans --ignoreScans Ignore print statement of scans ----------------- Xnatsessionupdate ----------------- Xnatsessionupdate resets the last update date on XNAT on a session. It will force DAX update scripts to update the session. This tool is for advanced users and managers of projects on XNAT. :: ################################################################ # XnatSessionUpdate # # # # Developed by the MASI Lab Vanderbilt University, TN, USA. # # If issues, please start a thread here: # # https://groups.google.com/forum/#!forum/vuiis-cci # # Usage: # # Reset sessions to be seen by the nex dax_update. # # Examples: # # Check the help for examples by running --help # ################################################################ ---------------------------------------------------------------- usage: XnatSessionUpdate [-h] [--host HOST] [-u USERNAME] -p PROJECTS [-s SESSION] [-n] [-x TXT_FILE] [-a] What is the script doing : * Reset sessions last update date to update the sessions during the next dax_update. Examples: *Reset all sessions: Xnatsessionupdate -p PID --all *Reset some sessions : Xnatsessionupdate -p PID -s 109374,109348 *Reset for the sessions that have assessors NEED_INPUTS: Xnatsessionupdate -p PID -n optional arguments: -h, --help show this help message and exit --host HOST Host for XNAT. Default: env XNAT_HOST. -u USERNAME, --username USERNAME Username for XNAT. -p PROJECTS, --project PROJECTS Projects ID on Xnat. -s SESSION, --session SESSION Session label on Xnat or list of them. -n, --needinputs Change the subject last update date for all the subject with processes that have a job status equal to NEED_INPUTS. -x TXT_FILE, --txtfile TXT_FILE File txt with at each line the label of the assessor or just the Session label where the Subject date need to be changed. E.G for label: project-x-subject-x-experiment-x-scan-x-process_name. -a, --all Change for all sessions. ----------- BIDSMapping ----------- BIDSMapping tool allows the user to create, update or replace rules/mapping at the project level on XNAT. These rules are essential as they entail the link between scan type or series description on XNAT to the BIDS datatype, task type and repetition time. XnatToBids function uses these mapping at the project to transform XNAT data into the BIDS compliant data with BIDS filenames and folder structure. :: ################################################################ # BIDSMAPPING # # # # Developed by the MASI Lab Vanderbilt University, TN, USA. # # If issues, please start a thread here: # # https://groups.google.com/forum/#!forum/vuiis-cci # # Usage: # # Upload rules/mapping to Project level on XNAT. # # Examples: # # Check the help for examples by running --help # ################################################################ usage: use "BIDSMapping --help" for more information What is the script doing : *Uploads BIDS datatype, tasktype and repitition time mapping to XNAT project level using the different OPTIONS. Examples: *Create a new datatype mapping for scan_type of XNAT scans: BIDSMapping -p PID --xnatinfo scan_type --type datatype --create /tmp/projectID_datataype.csv *The correct format for /tmp/projectID_datataype.csv scan_type,datatype Resting State,func *Create a new datatype mapping for series_description of XNAT scans: BIDSMapping -p PID --xnatinfo series_description --type datatype --create /tmp/projectID_datataype.csv *Create a new tasktype mapping for scan_type of XNAT scans: BIDSMapping -p PID --xnatinfo scan_type --type tasktype --create /tmp/projectID_tasktype.csv *Replace tasktype mapping for scan_type of XNAT scans: (It removes the old mapping and upload the new mapping) BIDSMapping -p PID --xnatinfo scan_type --type tasktype --replace /tmp/projectID_tasktype.csv *Update tasktype mapping for scan_type of XNAT scans: (This is ONLY add new mapping rules, CANT remove rules use --replace to remove and add mapping rules) BIDSMapping -p PID --xnatinfo scan_type --type tasktype --update /tmp/projectID_tasktype.csv *Create default datatype mapping for scan_type of XNAT scans: (There is no default for series_description use --create) BIDSMapping -p PID --xnatinfo scan_type --type datatype --create_default *Download the current mapping on XNAT: BIDSMapping -p PID --xnatinfo scan_type --type datatype --download /tmp/download.csv *Download the scan_types on project on XNAT: BIDSMapping -p PID --template /tmp/scan_type_template.csv optional arguments: -h, --help show this help message and exit --host HOST Host for XNAT. Default: using $XNAT_HOST. -u USERNAME, --username USERNAME Username for XNAT. Default: using $XNAT_USER. -o LOGFILE, --logfile LOGFILE Write the display/output in a file given to this OPTIONS. -p PROJECT, --project PROJECT Project to create/update BIDS mapping file -t TYPE, --type TYPE The type of mapping either datatype, tasktype or repetition_time_sec -x XNATINFO, --xnatinfo XNATINFO The type of xnat info to use for mapping either scan_type or series_description -c CREATE, --create CREATE Create the given BIDS new mapping file at project level. (EG. --create .csv) Default create creates the default mapping at project file. (EG. --create) csvfile EG: scan_type,datatype T1W/3D/TFE,anat Resting State,func -cd, --create_default Default create creates the default mapping at project file. (EG. --create_default) -ud UPDATE, --update UPDATE Update the existing BIDS mapping file at project level. (EG. --update .csv) This option can only add rules -rp REPLACE, --replace REPLACE Replace the existing BIDS mapping file at project level. (EG. --replace .csv) This option can remove and add new rules -rv REVERT, --revert REVERT Revert to an old mapping from a specific date/time. (EG: --revert 10-17-19-21:32:15 or --revert 10-17-19). Check the LOGFILE at project level for the date -d DOWNLOAD, --download DOWNLOAD Downloads the current BIDS mapping file (EG: --download ) -tp TEMPLATE, --template TEMPLATE Default mapping template (EG: --template