10 Help

10.1 Common Errors

BiocMAP should be configured so that fundamental issues related to pipeline function do not arise. If you encounter an error and believe it to be a design flaw in BiocMAP, you can always submit a github issue. However, please take a look at the following common issues:

A job/ process is not given enough memory or time: pipeline runs on large samples or datasets may require more memory or a higher time limit. When reported correctly, the pipeline will indicate an error status of 140 (for SGE or SLURM environments); however, memory issues can take many forms, and related error messages are not always clear. In this example case, the process FilterAlignments failed due to insufficient memory, but indicated a general error status (1):

How Nextflow may report memory-related errors

Attempt to provide the process more memory in your config. In this case the configuration for FilterAlignments looks like this (for SGE users):
```
withName: FilterAlignments {
    cpus = 2
    penv = 'local'
    memory = 16.GB
    clusterOptions = '-l h_fsize=800G'
}
```
Note that disk space may also be the limitation. See the configuration section for more info.
The pipeline keeps halting during alignment: In particular, the output log shows the error message "No GPUs are available.". This can have a few causes– if you are running the pipeline on an SGE/SLURM-based cluster, BiocMAP assumes that when GPUs are fully occupied, pending jobs will be held in queue, and will not be run until those resources become available. When this assumption is not met, some potential workarounds (which also apply to local runs/ without a cluster) are:
1. Set the max_gpus variable in your config file for the first module to 1, if you’ve raised it above this default value.
2. Increase the number of CPUs required for the AlignReads process in your config file (see “Specifying Options for your Cluster”. In particular, make sure the ratio of total CPUs available to this value exceeds the number of GPUs available. For example, if you are running the pipeline locally, have 2 GPUs and 8 CPU cores, set cpus = 4 in the AlignReads process.
3. If the above solutions don’t work, a last-case fix could be to lower queueSize in the executor scope of your config file for the first module to the number of GPUs available to you. Note that this limits the number of concurrent processes, and can thus greatly slow pipeline execution.
```
executor {
    name = 'sge'
    queueSize = 8  // By default, this is set to 40
    submitRateLimit = '1 sec'
    exitReadTimeout = '40 min'
}
```
Nextflow has trouble communicating with your cluster: a less common issue can occur on slower clusters, related to nextflow failing to poll your grid scheduler (like SGE or SLURM) for information it needs about the jobs that are running. This can show up in an error message like: ProcessFilterAlignments (Prefix: Sample_FE2P1_blood)terminated for an unknown reason -- Likely it has been terminated by the external system. We have found that raising the exitReadTimeout to a large value (such as 40 minutes) solves this issue, but consider raising it further if needed.
```
executor {
    name = 'sge'
    queueSize = 40
    submitRateLimit = '1 sec'
    exitReadTimeout = '40 min'
}
```

10.2 Resuming

An important feature of BiocMAP (because it is based on nextflow) is the ability to resume pipeline execution if an error occurs for any reason. To resume, you must add the -resume flag to your “main” script, determined here, and rerun. Otherwise, the default is to restart the entire pipeline, regardless of how much progress was made!

10.3 Docker help

For those who wish to use docker to manage BiocMAP software dependencies, we provide a brief set-up guide.

Install docker

A set of instructions for different operating systems are available on the Docker site.

Create a docker group

sudo addgroup docker

Add user to docker group

sudo usermod -aG docker <your_user>

Checking installation

Log out and log back in to ensure your user is running with the correct permissions.

Test Docker installation by running:

docker run hello-world