Computer setup

JHPCE

Since most analyses we run involve resources larger than those in our laptops, we typically used a high performance computing environment. Particularly, JHPCE. One of the first things you’ll do is get a JHPCE account. Once you do, you’ll want to get familiarized with many parts of JHPCE. You can see the archive of questions people have asked through bithelp here.

• Setup password-less login through ssh keys.
• Environment modules are useful so you don’t have to install some common software from stratch, such as R which you can load using module load conda_R.
• Use LIBD modules which you can setup following these instructions. jhpce_mod_source and jhpce_module_config are our GitHub repositories for our LIBD modules.
• Edit your bashrc file for a nicer terminal experience blog post.
• sgejobs (documentation): contains helper functions for writing and interacting with SGE jobs at JHPCE.
• Array and Sequential Cluster Jobs. See also sgejobs::job_single().
• Setting up your computer for bioinformatics/biostatistics and a compedium of resources 2012 blog post by Leo. Includes some Mac and Windows tools.
• Login to the cluster, request a node and change to your project directory in a single command 2013 blog post by Leo.
• me: Bad rm, don’t delete stuff I didn’t want to delete! (rm: well, I do what you tell me to do!) 2012 blog post by Leo.
• Automatically coloring your R output in the terminal using colorout blog post by Leo on colorout which you can access through jalvesaq/colorout nowadays and install with remotes::install_github('jalvesaq/colorout').
• Check Leo’s JHPCE configuration files here.

There were a lot of miscellaneous things about using the cluster that might save some confusion if you knew now:

• there’s a 100G storage max on your home directory /users/[yourusername]/, which will likely fill up very quickly. Most of us have directories under the /dcl01 and /dcl02 filesystems, where there is far more space.
• you generally will want to include the -cwd flag in bash scripts. By default, it will default to dumping output files in your home directory, regardless of where you submit the script.
• for scripts/jobs where you are generating large files, you will need to change the max writeable file size. This is the h_fsize flag for bash scripts, which defaults to 10G. See https://jhpce.jhu.edu/knowledge-base/how-to/.

Windows

• TODO: WinSCP, putty, etc
• git for windows which includes Git Bash. Check rstudio issue 2224 for specifics on how to install Git Bash such that it will work with RStudio terminal.

Mac

• Cyberduck for accessing files remotely. With it, you can open any R files at JHPCE with RStudio in your laptop, then open a terminal from RStudio so you can execute your code.
• Alfred for quickly finding files among many other powerful workflows.
• Textmate setup (Mac only) LIBD rstats 2018 blog post by Leo; it’s an alternative to using RStudio. The blog post also explains rmate which now also exists as a LIBD module that you can load with module load rmate at JHPCE.
• CriticMarkup.
• How to sync your Mac’s Documents, Desktop, and any other folder to Dropbox.
• Mac keyboard shortcuts.