DPABISurfSlurm: High Performance Computing Version of DPABISurf

Submitted by YAN Chao-Gan on

 

 

DPABISurfSlurm is a high performance computing version of DPABISurf! You can use it on High Performance Computing (HPC, or supercomputing) center. The only requirement is Singularity!

 

1.     Download the DPABISurfSlurm scripts to ${DPABISurfSlurmDIR}: https://github.com/Chaogan-Yan/DPABI/tree/master/DPABISurf/DPABISurfSlurm.

2.     Download the singularity files: singularity pull dpabisurfslurm.sif docker://cgyan/dpabisurfslurm:latest and singularity pull freesurfer.sif docker://cgyan/freesurfer:latest. Or download from here. Place them in the directory under ${SingularityDIR}.

3.     Prepare your data in BIDS format in the BIDS directory under ${DATADIR}. You should also have a subjects.txt under ${DATADIR}. E.g.,

BIDS

subjects.txt

     sub-Sub001

     sub-Sub002

     sub-Sub003

4.     In the *.slurm files under ${DPABISurfSlurmDIR}, you should revise this line:

#SBATCH -p normal #指定分区名称

normal is the partition you will use by Slurm. Ask your HPC administrator for the partition name you can use, and then revise “normal” to that partition name.

You may also need to revise this line to correctly load the singularity module, please confirm with your HPC administrator.

module load singularity/3.8.0

5.     Copy DPABISurf_runSlurm.sh and SetEnv.sh from ${DPABISurfSlurmDIR} to your ${DATADIR}.

6.     In DPABISurf_runSlurm.sh, you should define:

export DPABISurfSlurmDIR="/work1/yancg/Scripts/DPABISurfSlurm"

# Should have the DPABISurfSlurm files

export SetEnvScriptDir="/work1/yancg/Data/Test/Test"

# Should have SetEnv.sh

# You should also go into SetEnv.sh to define your parameters!!!

export DATADIR="/work1/yancg/Data/Test/Test"

# Should have BIDS and subjects.txt

7.     In SetEnv.sh, you should define:

export DATADIR="/work1/yancg/Data/Test/Test"

# Should have BIDS and subjects.txt

export SingularityDIR="/work1/yancg/Soft/Singularity"

# Should have dpabisurfslurm.sif. You can get it by singularity pull dpabisurfslurm.sif docker://cgyan/dpabisurfslurm:latest

# Should have freesurfer.sif. You can get it by singularity pull freesurfer.sif docker://cgyan/freesurfer:latest

export FreeSurferLicenseDIR="/work1/yancg/Soft/FreeSurferLicense"

# Should have license.txt from FreeSurfer

export RemoveFirstTimePoints="5"

# Set up Number of time points needs to be removed

export FunctionalSessionNumber="1"

# Set up Number of Functional Sessions

8.     Make a directory to save the log files under ${DATADIR} and cd to that directory.

9.     source ../DPABISurf_runSlurm.sh. Submit your tasks! You can use squeue to check the status of running.

10.  You will have all you want at ${DATADIR}/ResultsOrganized.tar.gz.

 

During the processing, you may encounter some subjects failed processing, you may find these commands helpful for your checking: https://github.com/Chaogan-Yan/DPABI/blob/master/DPABISurf/DPABISurfSlurm/RefCommandsForFailedSubjects.txt