10 Help

10.1 Common Errors

PopTop should be configured so that fundamental issues related to pipeline function do not arise. If you encounter an error and believe it to be a design flaw in PopTop, you can always submit a github issue. However, please take a look at the following common issues:

• A job/ process is not given enough memory or time: pipeline runs on large samples or datasets may require more memory or a higher time limit. When reported correctly, the pipeline will indicate an error status; however, memory issues can take many forms, and related error messages are not always clear. In this example case, the process CleanPlink failed due to insufficient memory, but indicated a general error status (1):

How Nextflow may report memory-related errors

Attempt to provide the process more memory in your config. In this case the configuration for CleanPlink looks like this:

 withName: CleanPlink {
        cpus = 1
        clusterOptions = '-l mf=5G,h_vmem=5G'
        module = 'plink/1.90b6.6'
    }

Note that disk space may also be the limitation. See the configuration section for more info.

• A backslash is missing in the main script: in each main script, options passed to nextflow are by default split across several lines for clarity. When changing around options in this script, it is easy to omit a backslash (\), which is required for the script to interpret a series of lines of options as a single command. In many cases, the pipeline will run successfully– at least for a large fraction of the workflow– but many desired options may be completely ignored. Here is an example of the mistake in a modified version of the run_pipeline.sh script:

#  The "--output" line has no "\" at the end, and the
#  "-resume" option will be ignored!
nextflow main.nf \
    -profile jhpce \
    -w "./work" \
    --output "./out" 
    -resume

10.2 Resuming

An important feature of PopTop (because it is based on nextflow) is the ability to resume pipeline execution if an error occurs for any reason. To resume, you must add the -resume flag to your “main” script, determined here, and rerun. Otherwise, the default is to restart the entire pipeline, regardless of how much progress was made!

10.2.1 Trouble resuming?

In some cases, nextflow will improperly determine which processes have finished, when a pipeline run halts with an error. This can be related to how your filesystem communicates with nextflow. We have found that specifying cache = 'lenient' in the process section of the config fixes issues with resuming (such as re-running a process which had actually completed). See nextflow’s documentation on this. In your config, this could look like:

process {
  time = { 10.hour * task.attempt }
  errorStrategy = { task.exitStatus == 140 ? 'retry' : 'terminate' }
  maxRetries = 1
  // this can be placed here or anywhere inside the 'process' brackets starting
  // at the top of this code chunk
  cache = 'lenient'
  withName: PullAssemblyFasta {
    cpus = 1
    clusterOptions = '-l mf=2G,h_vmem=2G'
  }

This is the default in jhpce.config, but may be needed on other cluster types as well.