If you are working at LIBD or with large data, it’s very likely that it won’t fit in your laptop and that you’ll be using the terminal to interact with a high performance computing cluster (like JHPCE) or server. Some small edits to your bash configuration file can make your terminal experience much more enjoyable and hopefully boost your productivity. The edits described below work for any OS. On Windows, I’m assuming that you are using
git bash or a similar terminal program.
The way we can control our terminal appearance and some behavior is through the
.bashrc file. That file typically gets read once when loading a new terminal window and that is where we can save some shortcuts we like to use, alter the colors of our terminal, change the behavior of the up and down arrow keys, etc.
First, we need to learn where to locate this file. On all OS (Mac, Windows, Linux) machines/servers, the
.bashrc file typically lives at
~/.bashrc. For my Mac for example that is
/home/lcollado/.bashrc. For my Windows machine, that’s
/c/Users/Leonardo/.bashrc. Now, the dot before the file makes it a hidden file. A quick search can help you find the options for your computer that lets you see these hidden files. From a terminal window, I typically use this bash command to show all the hidden files (that’s from the
## List files in human readable format including hidden files ls -lha
You might have a
~/.bashrc and a
~/.bash_profile already. If not, lets create simple ones. You can use the
touch bash command for make a new file (
touch ~/.bashrc), or you could use this R code:
Next open them with your text editor (say Notepad++, TextMate 2, RStudio, among others) and paste the following contents.
# Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi # In my Mac one I also have this: if [ -f ~/.profile ]; then . ~/.profile fi
# Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc fi
Control your bash history
Lets start adding features to our terminal experience by editing the
~/.bashrc file. I typically include comments
# describing what the code is doing and where I learned how to do \(X\). The first part is controlling your bash history. I want to have a longer history than what is included by default and where duplicates are deleted.
# http://www.biostat.jhsph.edu/~afisher/ComputingClub/webfiles/KasperHansenPres/IntermediateUnix.pdf # https://unix.stackexchange.com/questions/48713/how-can-i-remove-duplicates-in-my-bash-history-preserving-order export HISTCONTROL=ignoreboth:erasedups export HISTSIZE=10000 shopt -s histappend shopt -s cmdhist
Change the up and down arrows
The next change will save you a lot of time! Plus it goes nicely with the bash history changes we just made. Normally, the up and down arrow let you select previous commands from your bash history (up) or select one of your latest commands (down, after having used up). The following changes make it so that the up arrow searches only commands that start with exactly the letters you had already typed.
Lets say that you just requested a compute node with
qrsh and you have an empty line.
If you use the
up arrow, you can navigate your command history. So far, this is the same as the default
up arrow behavior.
Lets say that I want to change directory to one of my recent projects. So I type
cd / in the terminal window (without hitting enter).
Next I use the
up arrow, and it only finds for me commands that start with
cd /, including this long one.
Did you like this? Well, add the following code to your
# Auto-complete command from history # http://lindesk.com/2009/04/customize-terminal-configuration-setting-bash-cli-power-user/ export INPUTRC=~/.inputrc
~/.inputrc file has the following contents:
#Page up/page down "\e[B": history-search-forward "\e[A": history-search-backward $include /etc/inputrc
As an added benefit, the up and down arrows will now have this improved behavior when you run
R inside a terminal, although it’s limited to your current R history: actually, I guess that you could change your .Rprofile to load the previous R history.
Interactive deleting of files
In a terminal, you normally delete files with
rm, but you can make an alias (shortcut) so that when you are deleting files with
rmi you will get asked to confirm whether you want to delete the file or not. This can be useful if you are using some patterns for finding the files that you are trying to delete but want to make sure that the pattern didn’t catch other files you want to keep.
# http://superuser.com/questions/384769/alias-rm-rm-i-considered-harmful alias rmi='rm -i'
Change the command prompt
You can also control the command prompt. That is, the parts that are shown before you start typing in your terminal. I like keeping it short, so it only shows me the parent directory instead of the full path, plus a small version for the time (hh:mm) in a 12 hour clock. This is sometimes useful if I run some commands and later on want to get a quick idea if any of them took a while to run (specially if I was not looking at the terminal).
# Change command prompt # http://www.cyberciti.biz/tips/howto-linux-unix-bash-shell-setup-prompt.html # http://www.cyberciti.biz/faq/bash-shell-change-the-color-of-my-shell-prompt-under-linux-or-unix/ # https://bbs.archlinux.org/viewtopic.php?id=48910 # previous in enigma2: "[\u@\h \W]\$ " # previously in mac: "\h:\W \u\$ " export PS1="\[\e[0;33m\]\A \W \$ \[\e[m\]"
You can change the colors of your terminal. For example, do you want directories to be shown in blue and/or bold font while executable files are shown in red. This goes in hand with the
ls --color=auto shortcut to make sure that the colors are used (Mac: you might need
brew install coreutils as described in this blog post). The following lines of my
~/.bashrc file include some old history of the colors and how I use to have other options.
# colors # http://norbauer.com/notebooks/code/notes/ls-colors-and-terminal-app # used BSD pattern ExGxFxDxBxEgEdxbxgxhxd on http://geoff.greer.fm/lscolors/ # that tool does not specify the colors, which I did by looking manually at # http://blog.twistedcode.org/2008/04/lscolors-explained.html # and the norbauer.com site previously mentioned alias ls="ls --color=auto" #export LS_COLORS="di=1;34;40:ln=1;36;40:so=1;35;40:pi=1;93;40:ex=1;31;40:bd=1;34;46:cd=1;34;43:su=0;41:sg=0;46:tw=0;47:ow=0;43" ## After switching to RStudio: # https://askubuntu.com/questions/466198/how-do-i-change-the-color-for-directories-with-ls-in-the-console export LS_COLORS="di=0;32:ln=0;36:so=0;35:pi=0;93:ex=0;31:bd=0;34;46:cd=0;34;43:su=0;41:sg=0;46:tw=0;47:ow=0;43:fi=0;33"
Mac extra lines:
# Uncomment below for Mac and comment the two previous commands #export CLICOLOR=1 #export LSCOLORS="ExGxFxDxBxEgEdxbxgxhxd" ## Actually from https://superuser.com/questions/183876/how-do-i-get-ls-color-auto-to-work-on-mac-os-x # brew install coreutils # then change the aliast to use gls instead of ls # that way I can use the same config file =) alias ls="gls --color=auto"
I use the same
LS_COLORS now on my Mac too, but you don’t need to.
We got this note from Mark Miller, admin of JHPCE:
One quick note on your page. You mention setting colors for the ls output, which is great. One thing we (and others) have found is that, for a directory on a Lustre filesystem (/dcl01 or /dcl02), using “ls –colors=auto” or “ls -al” on a directory with lots (thousands+) of files in it can be super slow. With these options, the ls command needs to iterate through each file in the directory, and query the lustre server for each and every file to retrieve information about the file in order to determine what color to display. So, if you’re regularly using directories on Lustre that have lots of files in them, and your “ls” command it taking too long, we recommend using “ls –color=none”. https://wikis.nyu.edu/display/NYUHPC/Lustre+FAQ https://groups.google.com/forum/#!topic/lustre-discuss-list/3afjd4j2Q-g
Shortcuts for main project directories
We’ve seen several aliases (shortcuts) already such as the one for
ls --color=auto which is the one I use the most. But I also use aliases for changing to the root directories that I use the most.
alias labold="cd /dcl01/lieber/ajaffe/lab" alias lab="cd /dcl01/ajaffe/data/lab"
Actually, we were supposed to just use the new disk here and I should have probably chosen better names to differentiate the two.
The next terminal window you open after editing the
~/.bashrc file will have all your new features enabled.
Sometimes you might need to export other environment variables, such as
RMATE_PORT described in the
rmate setup post.
Repeat this process
You can/should repeat this process for other
~/.bashrc files you interact with. In my case, that would be:
~/.bashrcin my Mac laptop
~/.bashrcat my JHPCE home
~/.bashrcin my Windows laptop
This blog post was made possible thanks to:
## Session info ----------------------------------------------------------------------------------------------------------
## setting value ## version R version 3.5.1 Patched (2018-10-14 r75439) ## system x86_64, darwin15.6.0 ## ui X11 ## language (EN) ## collate en_US.UTF-8 ## tz America/New_York ## date 2018-10-26
## Packages --------------------------------------------------------------------------------------------------------------
## package * version date source ## backports 1.1.2 2017-12-13 cran (@1.1.2) ## base * 3.5.1 2018-10-15 local ## bibtex 0.4.2 2017-06-30 CRAN (R 3.5.0) ## BiocStyle * 2.8.2 2018-05-30 Bioconductor ## blogdown 0.8 2018-07-15 CRAN (R 3.5.0) ## bookdown 0.7 2018-02-18 CRAN (R 3.5.0) ## colorout * 1.2-0 2018-05-03 Github (jalvesaq/colorout@c42088d) ## compiler 3.5.1 2018-10-15 local ## datasets * 3.5.1 2018-10-15 local ## devtools * 1.13.6 2018-06-27 cran (@1.13.6) ## digest 0.6.18 2018-10-10 CRAN (R 3.5.0) ## evaluate 0.12 2018-10-09 CRAN (R 3.5.0) ## graphics * 3.5.1 2018-10-15 local ## grDevices * 3.5.1 2018-10-15 local ## htmltools 0.3.6 2017-04-28 cran (@0.3.6) ## httr 1.3.1 2017-08-20 CRAN (R 3.5.0) ## jsonlite 1.5 2017-06-01 CRAN (R 3.5.0) ## knitcitations * 1.0.8 2017-07-04 CRAN (R 3.5.0) ## knitr 1.20 2018-02-20 cran (@1.20) ## lubridate 1.7.4 2018-04-11 CRAN (R 3.5.0) ## magrittr 1.5 2014-11-22 cran (@1.5) ## memoise 1.1.0 2017-04-21 CRAN (R 3.5.0) ## methods * 3.5.1 2018-10-15 local ## plyr 1.8.4 2016-06-08 cran (@1.8.4) ## R6 2.3.0 2018-10-04 CRAN (R 3.5.0) ## Rcpp 0.12.19 2018-10-01 CRAN (R 3.5.1) ## RefManageR 1.2.0 2018-04-25 CRAN (R 3.5.0) ## rmarkdown 1.10 2018-06-11 CRAN (R 3.5.0) ## rprojroot 1.3-2 2018-01-03 cran (@1.3-2) ## stats * 3.5.1 2018-10-15 local ## stringi 1.2.4 2018-07-20 CRAN (R 3.5.0) ## stringr 1.3.1 2018-05-10 CRAN (R 3.5.0) ## tools 3.5.1 2018-10-15 local ## utils * 3.5.1 2018-10-15 local ## withr 2.1.2 2018-03-15 CRAN (R 3.5.0) ## xfun 0.3 2018-07-06 CRAN (R 3.5.0) ## xml2 1.2.0 2018-01-24 CRAN (R 3.5.0) ## yaml 2.2.0 2018-07-25 CRAN (R 3.5.0)