Edit your bashrc file for a nicer terminal experience
If you are working at LIBD or with large data, it’s very likely that it won’t fit in your laptop and that you’ll be using the terminal to interact with a high performance computing cluster (like JHPCE) or server. Some small edits to your bash configuration file can make your terminal experience much more enjoyable and hopefully boost your productivity. The edits described below work for any OS. On Windows, I’m assuming that you are using git bash or a similar terminal program.
The way we can control our terminal appearance and some behavior is through the .bashrc file. That file typically gets read once when loading a new terminal window and that is where we can save some shortcuts we like to use, alter the colors of our terminal, change the behavior of the up and down arrow keys, etc.
.bashrc file
First, we need to learn where to locate this file. On all OS (Mac, Windows, Linux) machines/servers, the .bashrc file typically lives at ~/.bashrc. For my Mac for example that is /home/lcollado/.bashrc. For my Windows machine, that’s /c/Users/Leonardo/.bashrc. Now, the dot before the file makes it a hidden file. A quick search can help you find the options for your computer that lets you see these hidden files. From a terminal window, I typically use this bash command to show all the hidden files (that’s from the a option).
## List files in human readable format including hidden files
ls -lhaInitial files
You might have a ~/.bashrc and a ~/.bash_profile already. If not, lets create simple ones. You can use the touch bash command for make a new file (touch ~/.bashrc), or you could use this R code:
file.create('~/.bashrc')
file.create('~/.bash_profile')Next open them with your text editor (say Notepad++, TextMate 2, RStudio, among others) and paste the following contents.
~/.bash_profile contents
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# In my Mac one I also have this:
if [ -f ~/.profile ]; then
. ~/.profile
fiMinimal ~/.bashrc contents
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fiControl your bash history
Lets start adding features to our terminal experience by editing the ~/.bashrc file. I typically include comments # describing what the code is doing and where I learned how to do \(X\). The first part is controlling your bash history. I want to have a longer history than what is included by default and where duplicates are deleted.
# http://www.biostat.jhsph.edu/~afisher/ComputingClub/webfiles/KasperHansenPres/IntermediateUnix.pdf
# https://unix.stackexchange.com/questions/48713/how-can-i-remove-duplicates-in-my-bash-history-preserving-order
export HISTCONTROL=ignoreboth:erasedups
export HISTSIZE=10000
shopt -s histappend
shopt -s cmdhistChange the up and down arrows
The next change will save you a lot of time! Plus it goes nicely with the bash history changes we just made. Normally, the up and down arrow let you select previous commands from your bash history (up) or select one of your latest commands (down, after having used up). The following changes make it so that the up arrow searches only commands that start with exactly the letters you had already typed.
Lets say that you just requested a compute node with qrsh and you have an empty line.
If you use the up arrow, you can navigate your command history. So far, this is the same as the default up arrow behavior.
Lets say that I want to change directory to one of my recent projects. So I type cd / in the terminal window (without hitting enter).
Next I use the up arrow, and it only finds for me commands that start with cd /, including this long one.
Did you like this? Well, add the following code to your ~/.bashrc file
# Auto-complete command from history
# http://lindesk.com/2009/04/customize-terminal-configuration-setting-bash-cli-power-user/
export INPUTRC=~/.inputrcwhere ~/.inputrc file has the following contents:
#Page up/page down
"\e[B": history-search-forward
"\e[A": history-search-backward
$include /etc/inputrcAs an added benefit, the up and down arrows will now have this improved behavior when you run R inside a terminal, although it’s limited to your current R history: actually, I guess that you could change your .Rprofile to load the previous R history.
Interactive deleting of files
In a terminal, you normally delete files with rm, but you can make an alias (shortcut) so that when you are deleting files with rmi you will get asked to confirm whether you want to delete the file or not. This can be useful if you are using some patterns for finding the files that you are trying to delete but want to make sure that the pattern didn’t catch other files you want to keep.
# http://superuser.com/questions/384769/alias-rm-rm-i-considered-harmful
alias rmi='rm -i'Change the command prompt
You can also control the command prompt. That is, the parts that are shown before you start typing in your terminal. I like keeping it short, so it only shows me the parent directory instead of the full path, plus a small version for the time (hh:mm) in a 12 hour clock. This is sometimes useful if I run some commands and later on want to get a quick idea if any of them took a while to run (specially if I was not looking at the terminal).
# Change command prompt
# http://www.cyberciti.biz/tips/howto-linux-unix-bash-shell-setup-prompt.html
# http://www.cyberciti.biz/faq/bash-shell-change-the-color-of-my-shell-prompt-under-linux-or-unix/
# https://bbs.archlinux.org/viewtopic.php?id=48910
# previous in enigma2: "[\u@\h \W]\$ "
# previously in mac: "\h:\W \u\$ "
export PS1="\[\e[0;33m\]\A \W \$ \[\e[m\]"Colors
You can change the colors of your terminal. For example, do you want directories to be shown in blue and/or bold font while executable files are shown in red. This goes in hand with the ls --color=auto shortcut to make sure that the colors are used (Mac: you might need brew install coreutils as described in this blog post). The following lines of my ~/.bashrc file include some old history of the colors and how I use to have other options.
# colors
# http://norbauer.com/notebooks/code/notes/ls-colors-and-terminal-app
# used BSD pattern ExGxFxDxBxEgEdxbxgxhxd on http://geoff.greer.fm/lscolors/
# that tool does not specify the colors, which I did by looking manually at
# http://blog.twistedcode.org/2008/04/lscolors-explained.html
# and the norbauer.com site previously mentioned
alias ls="ls --color=auto"
#export LS_COLORS="di=1;34;40:ln=1;36;40:so=1;35;40:pi=1;93;40:ex=1;31;40:bd=1;34;46:cd=1;34;43:su=0;41:sg=0;46:tw=0;47:ow=0;43"
## After switching to RStudio:
# https://askubuntu.com/questions/466198/how-do-i-change-the-color-for-directories-with-ls-in-the-console
export LS_COLORS="di=0;32:ln=0;36:so=0;35:pi=0;93:ex=0;31:bd=0;34;46:cd=0;34;43:su=0;41:sg=0;46:tw=0;47:ow=0;43:fi=0;33"Mac extra lines:
# Uncomment below for Mac and comment the two previous commands
#export CLICOLOR=1
#export LSCOLORS="ExGxFxDxBxEgEdxbxgxhxd"
## Actually from https://superuser.com/questions/183876/how-do-i-get-ls-color-auto-to-work-on-mac-os-x
# brew install coreutils
# then change the aliast to use gls instead of ls
# that way I can use the same config file =)
alias ls="gls --color=auto"I use the same LS_COLORS now on my Mac too, but you don’t need to.
UPDATE
We got this note from Mark Miller, admin of JHPCE:
One quick note on your page. You mention setting colors for the ls output, which is great. One thing we (and others) have found is that, for a directory on a Lustre filesystem (/dcl01 or /dcl02), using “ls –colors=auto” or “ls -al” on a directory with lots (thousands+) of files in it can be super slow. With these options, the ls command needs to iterate through each file in the directory, and query the lustre server for each and every file to retrieve information about the file in order to determine what color to display. So, if you’re regularly using directories on Lustre that have lots of files in them, and your “ls” command it taking too long, we recommend using “ls –color=none”. https://wikis.nyu.edu/display/NYUHPC/Lustre+FAQ https://groups.google.com/forum/#!topic/lustre-discuss-list/3afjd4j2Q-g
Shortcuts for main project directories
We’ve seen several aliases (shortcuts) already such as the one for ls --color=auto which is the one I use the most. But I also use aliases for changing to the root directories that I use the most.
alias labold="cd /dcl01/lieber/ajaffe/lab"
alias lab="cd /dcl01/ajaffe/data/lab"Actually, we were supposed to just use the new disk here and I should have probably chosen better names to differentiate the two.
The next terminal window you open after editing the ~/.bashrc file will have all your new features enabled.
Extra
Sometimes you might need to export other environment variables, such as RMATE_PORT described in the rmate setup post.
Repeat this process
You can/should repeat this process for other ~/.bashrc files you interact with. In my case, that would be:
~/.bashrcin my Mac laptop~/.bashrcat my JHPCE home~/.bashrcin my Windows laptop
Acknowledgments
This blog post was made possible thanks to:
References
[1] C. Boettiger. knitcitations: Citations for ‘Knitr’ Markdown Files. R package version 1.0.8. 2017. URL: https://CRAN.R-project.org/package=knitcitations.
[2] A. Oleś, M. Morgan and W. Huber. BiocStyle: Standard styles for vignettes and other Bioconductor documents. R package version 2.8.2. 2018. URL: https://github.com/Bioconductor/BiocStyle.
[3] H. Wickham, J. Hester and W. Chang. devtools: Tools to Make Developing R Packages Easier. R package version 1.13.6. 2018. URL: https://CRAN.R-project.org/package=devtools.
[4] Y. Xie, A. P. Hill and A. Thomas. blogdown: Creating Websites with R Markdown. ISBN 978-0815363729. Boca Raton, Florida: Chapman and Hall/CRC, 2017. URL: https://github.com/rstudio/blogdown.
Reproducibility
## Session info ----------------------------------------------------------------------------------------------------------## setting value
## version R version 3.5.1 Patched (2018-10-14 r75439)
## system x86_64, darwin15.6.0
## ui X11
## language (EN)
## collate en_US.UTF-8
## tz America/New_York
## date 2018-10-26## Packages --------------------------------------------------------------------------------------------------------------## package * version date source
## backports 1.1.2 2017-12-13 cran (@1.1.2)
## base * 3.5.1 2018-10-15 local
## bibtex 0.4.2 2017-06-30 CRAN (R 3.5.0)
## BiocStyle * 2.8.2 2018-05-30 Bioconductor
## blogdown 0.8 2018-07-15 CRAN (R 3.5.0)
## bookdown 0.7 2018-02-18 CRAN (R 3.5.0)
## colorout * 1.2-0 2018-05-03 Github (jalvesaq/colorout@c42088d)
## compiler 3.5.1 2018-10-15 local
## datasets * 3.5.1 2018-10-15 local
## devtools * 1.13.6 2018-06-27 cran (@1.13.6)
## digest 0.6.18 2018-10-10 CRAN (R 3.5.0)
## evaluate 0.12 2018-10-09 CRAN (R 3.5.0)
## graphics * 3.5.1 2018-10-15 local
## grDevices * 3.5.1 2018-10-15 local
## htmltools 0.3.6 2017-04-28 cran (@0.3.6)
## httr 1.3.1 2017-08-20 CRAN (R 3.5.0)
## jsonlite 1.5 2017-06-01 CRAN (R 3.5.0)
## knitcitations * 1.0.8 2017-07-04 CRAN (R 3.5.0)
## knitr 1.20 2018-02-20 cran (@1.20)
## lubridate 1.7.4 2018-04-11 CRAN (R 3.5.0)
## magrittr 1.5 2014-11-22 cran (@1.5)
## memoise 1.1.0 2017-04-21 CRAN (R 3.5.0)
## methods * 3.5.1 2018-10-15 local
## plyr 1.8.4 2016-06-08 cran (@1.8.4)
## R6 2.3.0 2018-10-04 CRAN (R 3.5.0)
## Rcpp 0.12.19 2018-10-01 CRAN (R 3.5.1)
## RefManageR 1.2.0 2018-04-25 CRAN (R 3.5.0)
## rmarkdown 1.10 2018-06-11 CRAN (R 3.5.0)
## rprojroot 1.3-2 2018-01-03 cran (@1.3-2)
## stats * 3.5.1 2018-10-15 local
## stringi 1.2.4 2018-07-20 CRAN (R 3.5.0)
## stringr 1.3.1 2018-05-10 CRAN (R 3.5.0)
## tools 3.5.1 2018-10-15 local
## utils * 3.5.1 2018-10-15 local
## withr 2.1.2 2018-03-15 CRAN (R 3.5.0)
## xfun 0.3 2018-07-06 CRAN (R 3.5.0)
## xml2 1.2.0 2018-01-24 CRAN (R 3.5.0)
## yaml 2.2.0 2018-07-25 CRAN (R 3.5.0)