Build a bash script that loops over variables and submits SLURM jobs

This function builds a bash script functioning as an array job that loops over a set of variables with pre-specified values.

job_loop(
  loops,
  name,
  create_shell = FALSE,
  partition = "shared",
  memory = "10G",
  cores = 1L,
  tc = 20,
  email = "ALL",
  logdir = "logs"
)

Arguments

loops

A named list where each of the elements are character vectors. The names of loops specify the variables used for the loops and the contents specify the options to loop through for each variable.

name

A character(1) vector giving either the name or path (relative or absolute) to the shell script to create.

create_shell

A logical(1) vector specifying whether to create a shell file for the script.

partition

A character(1) vector with the name of the SLURM partition. Check how busy a given partition is by running squeue -p [partition].

memory

character(1): the amount of memory in total to request, as a number and unit accepted by the '--mem' SLURM parameter (e.g. '10G')

cores

The number of cores to request. Note that the total memory your job will request is cores multiplied by memory.

tc

If task_num is specified, this option controls the number of concurrent tasks.

email

The email reporting option for the email address ("BEGIN", "END", "FAIL", or "ALL")

logdir

The directory to contain the log, as an absolute or relative path.

Value

A length-2 list of character vectors containing the contents of the R and shell scripts

See also

Other shell-script creation and submission functions: array_submit(), job_single()

Author

Nicholas J. Eagles

Leonardo Collado-Torres

Examples

job_loop(
    loops = list(region = c("DLPFC", "HIPPO"), feature = c("gene", "exon", "tx", "jxn")),
    name = "bsp2_test_array",
    cores = 2
)