Submit an array job with a specified set of task IDs

Source: R/array_submit.R

Given a bash script that specifies the --array sbatch option (that is, an array job), this function overwrites (temporarily if restore = TRUE) the script in place and resubmits (when submit = TRUE) the array with the specified task_ids. If this array was created with job_single, task_ids may be ommitted and failed tasks are automatically inferred. This function is intended to help re-run failed tasks of a large array job that was previously submitted.

array_submit(
  name,
  task_ids = NULL,
  submit = FALSE,
  restore = TRUE,
  verbose = FALSE
)

Arguments

name

A character(1) vector giving either the name or path (relative or absolute) to the shell or R script to create.

task_ids

An optional numeric vector specifying which (relative) task IDs to resubmit (e.g. c(1, 4, 6)). If NULL, the task IDs will be inferred by scraping the log file for the job ID for the array job as originally submitted, and using job_report() to pull failed task IDs

submit

A logical(1) vector determining whether to actually submit the tasks or not using qsub.

restore

A logical(1) vector determining whether to restore the script to the original state.

verbose

A logical(1) vector specifying whether to print details about how failed tasks were determined (applicable when task_ids is NULL).

Value

The absolute path to the shell script to create.

See also

Other shell-script creation and submission functions: job_loop(), job_single()

Author

Leonardo Collado-Torres

Nicholas J. Eagles

Examples


## Choose a script name
job_name <- paste0("array_submit_example_", Sys.Date())

## Create an array job script to use for this example
job_single(
    name = job_name,
    create_shell = TRUE,
    task_num = 100
)
#> 2024-09-20 18:18:32.779989 creating the logs directory at:  logs
#> 2024-09-20 18:18:32.781164 creating the shell file array_submit_example_2024-09-20.sh
#> To submit the job use: sbatch array_submit_example_2024-09-20.sh

## Now we can submit the job for a set of task IDs (or omit 'task_ids'
## to automatically grab those same failed task IDs)
array_submit(
    name = job_name,
    task_ids = c(1, 6, 8:20, 67),
    submit = FALSE
)