Welcome back!
If you were here last week and you have already cloned the github repository,
you can update the phs directory with all of my new updates by navigating to
the phs
directory (cd phs
), and then running git pull
. If this doesn't
work, run git reset --hard HEAD~
, after which you can run git pull
(which
forces you to forget any changes you have made). Now, if you list the files
(ls
) in this directory you should see a new week_4
folder. Move into this
directory.
Last time, we installed LaTeX. We need to install a few other packages for this time, too. In a separate terminal, run the commands:
On Windows (in Windows Subsystem for Linux) or Ubuntu:
sudo apt-get install texlive-latex-recommended texlive-latex-extra texlive-font-utils pandoc evince
On Mac:
- First install
homebrew
, a package manager for Mac OSX (details athttp://brew.sh
), with/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
- Install
mactex
, a package containing LaTeX and many of its commonly used modules, withbrew cask install mactex
- Install
pandoc
withbrew install pandoc
. - Restart your terminal
Notice the new piece of software pandoc
, which is capable of converting files
between various markup formats. We will use it today to convert Markdown files
directly into PDF files.
Once this download is complete, ensure you have latex
and pdflatex
installed with:
% which latex
/usr/bin/latex
% which pdflatex
/usr/bin/pdflatex
Last week, we learned how to modify our .bashrc
file that configures our bash
experience. However, when opening Terminal in Mac OS, bash
looks for the file .bash_profile
instead of .bashrc
(as it does in
Linux/WSL). Therefore, Mac users should ensure that their .bash_profile
is
'linked' to their .bashrc
, which is accomplished by opening your
.bash_profile
in vim with vim ~/.bash_profile
, and adding the following
lines of text (from
Stack Exchange):
if [ -f ~/.bashrc ]; then
source ~/.bashrc
fi
For those of you who are unfamiliar,
this type of logical structre is called an if
statement.
In the first line, the single brackets []
literally invoke the
test
command (see man test
), which returns true
or false
depending on the
expression. If the conditional [ -f ~/.bashrc ]
is determined to be true, then the contents of the if
statement are
subsequently evaluated.
Here we are asking if a file exists (-f
) called ~/.bashrc
.
The content of the if
statement executes the .bashrc
file-- remember, it is just a bash
script, full of bash commands, and is therefore executable! The final line is
the bash syntax for ending an if statement (fi
is if backwards).
Whenever a login shell is entered (such as when Terminal is opened in Mac OS),
.bash_profile
will be called, and the above code will call the text in
.bashrc
. Therefore, you can now enter text into the .bashrc
and it will
customize your bash session as you would expect.
Files in your home directory that start with a .
(they are 'hidden') and end
in rc
are configuration files for given program. Today we will modify a
.bashrc
, which gives options for bash (which is probably the shell you are
using). You can ensure this with echo $0
(every line you enter into the shell
is literally passed to the bash
function (or whatever this command returned),
and the shell organizes the input and output in a user-friendly command line).
Your .bashrc
file is executed line by line in bash every time you start up
bash or open a terminal, which effectively personalizes your shell.
A sample .bashrc
with some useful commands can be found in this week_4
directory under the file name my_bashrc
.
Open up my example file my_bashrc
in this directory with vim my_bashrc
. Once open, open your actual .bashrc
file with :vsplit ~/.bashrc
in vim. Switch between the two sides of vim with Ctrl+w Ctrl+w
or by using
Ctrl-h
and Ctrl-l
.
Yank entire lines with yy
; or sweep out whole lines with V
, then sweep with jk
, then
yank with y
. Then switch windows with Ctrl+w Ctrl+w
, then paste with p
.
Or, if you want, you can go into insert mode (i
) and write the commands by
hand.
I recommend copying and pasting the first block of functions into your
.bashrc
(lines 3 through 26). After you are done editing, write and quit all
files with :wqa
in vim. To use the new .bashrc
, either run source ~/.bashrc
, or restart your terminal.
Let's take a closer look at one of these functions, vv
. When you call it, it
makes a new variable called temp
that is the output of a series of piped
commands. Piece by piece, ls -tr
lists files in time-reversed order, so
that the most recently modified files are at the end. *.+(tex|md|py)
only
lists the files that end in .tex
, .md
, or .py
files (these are just suggestions
for type of text
files people generally work with-- you can change this as needed). We pipe this
output into tail, and take the final entry (i.e. the name of the file that was
more recently modified). We store this file as the variable temp
, and then
state it in the terminal, and open it in vim. Try out the different pieces of
this command in the command line.
This function, vv
, is the sort of time-saving macro or function you can
already write that can improve your efficiency.
To test this, make a new file with echo SOME TEXT >> my_file.tex
, and run the
command vv
in the command line.
I also include some other files in this my_bashrc
that you can add. The S
and s
are functions that allow you to 'bookmark' a directory and teleport
there. vmd
will make a template file for markdown, which we will get to
later.
Last time we used some script files, which we made executable from anywhere in our computer. Let's review what we did last time, and you can check and ensure that your machine is up to date:
- We created a
bin
folder in our home directory, in which we placed executable scripts. Do you have this directory? Check withls ~/bin
-- if this gives an error, then you do not have it. - We added this bin to the
$PATH
bash local variable, but including a line in our.bashrc
that saysPATH=$PATH:/home/dillon/bin
, where/home/dillon
on your machine should be your home directory (checkecho $HOME
if you're not sure what that is)- Check that this worked by looking at your
$PATH
variable with the commandecho $PATH
. At the end of it, you should see<other text>:/home/dillon/bin
. - Remind yourself that the
.bashrc
is the configuration file that is executed every time you open bash. Therefore, this reassignment of the$PATH
variable occurs every time you open your computer.
- Check that this worked by looking at your
- We have some files in the
bin
from last week. They are included in this week's directory as well. Ensure they are in your path withwhich compile_article
andwhich clean_up_files
. - Lastly, we will be viewing graphics in this session, so if you are running
Windows, start your X graphics server, and then restart your terminals:
- If you followed the guide last week and installed an X server, launch the
program VcXsrv (assuming you have the line
export DISPLAY=localhost:0.0
in your.bashrc
already) - If you are on a lab computer, launch the program XLaunch and accept the default settings (press Next three times + Finish)
- If you followed the guide last week and installed an X server, launch the
program VcXsrv (assuming you have the line
Now, recall that you can make a .tex
file into a .pdf
file by running the
command pdflatex
. For example,
% pdflatex skeleton_document.tex
This generated unnecessary output files (.aux, .log) that we can delete with
clean_up_files
.
Some files are more complicated to compile (such as those that use bibTeX, a
way to easily store references and cite articles. You can compile any latex
file with the compile_article
command, too:
% compile_article skeleton_document.tex
Run lt
to list your files in time-reversed order (the .pdf
file should be
the newest file). Now you can open lais_example.pdf
using your favorite pdf
viewer (open
in Mac, evince
or mupdf
in Ubuntu or Windows (assuming your
X server is configured)).
LaTeX is widely used, and accordingly has extensive documentation for nearly every need. In general, (as with most things) the fastest way to learn how to use LaTeX is google: en.wikibooks.org/wiki/LaTeX/ is an excellent resource for basic issues, and StackExchange typically has solutions to any more advanced or specific questions.
Let's now look at a simple LaTeX report template that uses BibTeX, a
bibliography manager. A simple report file is in bibtex_file.tex
, which uses
the BibTex file bibtex_file.bib
. Each of these files are comprised of text,
and are compiled with LaTeX. The commands necessary to compile this article are
in compile_article
; you can either run this bash script (./compile_article bibtex_file.tex
) or you can type out each command in the bash script, replacing
the $filename
by bibtex_file
. The commands of this script are pdflatex
--
which we used earlier-- and bibtex
, which is a command used to get your
references right. When using bibtex
, you need to compile the article multiple
times (as in the script) in order for the program to learn what references go
where correctly. Also, this script runs clean_up_files
at the end, which
removes any unnecessary files that are made during the compilation process.
Open both the .tex
and the .bib
file in vim, using vim bibtex_file.tex
and then :tabe bibtex_file.bib
in vim. Switch between these two files with
gt
, which stands for goto tab. Note the format of the .bib
file.
This citation is typically available for any scientific journal, look for the
'download citation' link on the article website.
As we have seen, it is rather formulaic to compile and display latex files. But
it would be annoying if every time we made a modification to our latex file, we
had to exit vim, display the pdf, and enter vim again. We can make the
compilation and display of these files very streamlined by modifying your
~/.vimrc
.
Just like your .bashrc
file is run every time bash opens, your .vimrc
file
runs every time vim opens. In vim you use commands all the time-- even :wq
is
a command. Just like we coded useful macros into our .bashrc
file to save
time, we can code useful macros into our .vimrc
to save time. Open up the
my_vimrc
file in the week_4
directory with vim my_vimrc
. As before, open
up your actual .vimrc
file in the same window with :vsplit ~/.vimrc
. Your
.vimrc
may be empty, if you have never used it before. Switch windows with
Ctrl+w Ctrl+w
.
I recommend copying and pasting the first 8 lines into your .vimrc
. Do this
with block visual mode V
, sweeping out the desired section (j
and k
),
yanking with y
, changing windows with Ctrl+w Ctrl+w
, and pasting with p
.
Let's explore one of these commands:
autocmd BufRead,BufNewFile *.tex let @w=':wa
:!compile_article %
'
Notice that this looks different on the github than it does in vim. This is
because we are using a special character (in vim it looks like ^M
but
actually it is the literal Enter key,
, which we can't see on github but
which we can see in vim. To make this character, press (in insert mode)
Ctrl-V
followed by Enter
(try it yourself!). Ctrl-V
allows for any
non-standard key to be 'printed' in this way, including Backspace, Ctrl+keys,
and others. This is why the above formatting of the README on github looks
kind of weird.
In this command, we are telling Vim to run the command in quotes (:wa...%^M
)
every time we press the key @
then w
(not together). The command calls
compile_article
, the script we currently placed in our bin
, in order to
build a document. The autocmd BufRead,BufNewFile *.tex
tells Vim to use this
definition for @w
whenever a .tex
file is opened.
This command will literally be fed into Vim character-by-character. When you
press @w
, first a :
will be passed to Vim, putting it in command mode, then
a wa
will be fed to it, typing the command wa
for you (and writing=saving
all of the files you have open). When it reaches the
^M
, the Enter
key is fed to Vim, running that command for you. So we are
writing out all open files with the first :wa
.
The second bit of the command opens up command mode again, :
, then uses the
!
to tell Vim that the rest of the command is a shell command, not a Vim
command (try running :!ls
, for example). The %
will be replaced by the
filename of the document you are currently editing.
Lines 7 and 8 of my_vimrc
allow us top open the file through vim. In the same
way as before, it uses the editor evince
to open the proper pdf
file. In
Mac, you should replace evince
by open
(which is the default Mac graphics
viewer). For all users, fee free to replace this command with whichever PDF viewer
you prefer (mupdf
for instance).
Again, the formatting is weird due to the special character ^M
-- view this
file in vim in order to copy and paste. Here, we've simply added another macro
for Vim to use when we open .tex
files. This macro calls the command evince %:r.pdf &>/dev/null &
. The %:r
will
be replaced with the filename minus the extension (report.tex
-> report
),
so that %:r.pdf
will be replaced by the filename minus the initial extension
plus .pdf
(report.tex
-> report.pdf
). The &>/dev/null
tells the shell
to throw away output (any minor errors that evince
has, etc), and the final
&
tells evince
to run in the background, so that Vim will stay open and
visible.
Now compiling articles and viewing them are hotkeyed in vim as @w
and @o
.
Try it out by opening skeleton_document.tex
in vim, and running @w
, and
then @o
.
BEWARE: If you have a compilation error, you won't return from vim until you
ignore it (press X
three times-- you need to press it three times since we
are using pdflatex three times so that bibliographies are properly compiled).
One alternative to LaTeX is Markdown. Markdown is like fancy .txt-- it has
basic formatting. However, there is a powerful tool called pandoc
that can
convert markdown into any other format: it can convert it into LaTeX, or html,
or Word, and it will still format the document nicely.
As an example, look at the text in basic_markdown.md
. We will convert it into
a .tex
document, and then into a .pdf
like we did before. We will use
pandoc
, which should be already installed. If it isn't, run:
- Ubuntu/Windows:
sudo apt install pandoc
- Mac:
brew install pandoc
Now we will convert a standalone file (-s
) from (-f
) markdown to (-t
)
latex to a particular output file (-o
) with the command
% pandoc -s -f markdown -t latex -o basic_markdown.tex basic_markdown.md
Now you have the converted LaTeX file basic_markdown.tex-- take a look at it
with less
or tail
, and you will see that markdown inserts a huge preamble,
but then cuts and pastes the markdown text between the \begin{document}
and
\end{document}
tags. Now we can convert this .tex
file into a .pdf
with
our earlier compile_article
command:
% compile_article basic_markdown.tex
Take a look at the output pdf.
Pandoc will also convert markdown files into, for example, word documents (.docx). Try the following:
% pandoc -s -f markdown -t docx -o basic_markdown.docx basic_markdown.md
Now open basic_markdown.docx
in Word or Libreoffice, and you will see that it
is your standard Word editor. Therefore, you can use vim
to make a first
draft (create and edit text), and then convert it to Word and format it there
(or share it with your groupmates or advisor who may be LaTeX-illiterate). The
-t
option has many formats-- even .html
! Try
% pandoc -s -f markdown -t html -o basic_markdown.html basic_markdown.md
and open the html file in your web browser with chromium basic_markdown.html
or firefox basic_markdown.html
(as before, Windows users will need to
navigate to this file with Windows explorer).
Alternatively, we can compile our markdown file into a pdf in one step with the
build_pdf
command:
% ./build_pdf basic_markdown
Note here that we don't include the file extension for this program. Open
build_pdf
in vim and look through its contents-- every line is just a bash
command, and the commands are spun together in a script.
As before, we can
make a macro in vim
to run this build_pdf
program automatically. The
careful reader will know that we already did this, when we earlier altered our
.vimrc
! All that is left to be done is to move this file to your
bin with cp build_pdf ~/bin
(or wherever your bin
is).
Now open basic_markdown.md
in vim, and try using @w
to compile and @o
to
open the pdf file. This is an easy way to turn standard text that even includes
LaTeX formatting into a nice output pdf, without the need to remember the
often-confusing LaTeX syntax.
Finally, open the file beamer_in_markdown.md
. Compile it with @w
and open
it with @o
. This is a beamer (latex-style) file, useful for presentations,
made in markdown! This workflow makes it easy to produce high-quality slides for
presentations with only a few lines of text.
This workflow also makes markdown extremely useful
for casual note taking. For example, You can annotate
papers that you read in a papers.md
document with headers and bullet points.
Alternatively, you can
make notes for skype meetings with a collaborator using pandoc
to turn your
markdown into a beamer document (it turns different headings into new slides).
In fact, the document you are reading right now is actually a README.md
file
created in markdown!
Let's get some practice modifying documents in vim. Open
skeleton_document.tex
in vim, and scroll down to line 24. Autocompile it
regularly, open it as a pdf, and then every time you compile it refresh the pdf
to see how you changed it. This is how latex is meant to be done. Get practice
by doing the following tasks (autocompiling each time):
- Modify some of the math on lines 16 and 18
- Follow the instructions and fix the typos on lines 24-33
- Add an item to the itemize environment (line 49-ish) with your favorite type of ice cream
- Make a sub-sub-list by making a new enumerate environment within the existing ones on line 58
- Break the Pythagorean theorem by adding an extra constant (line 65-ish)
- Add a new row to the matrix (line 75-ish)
Make your own CV! Open the file cv_simple.tex
in vim, and find all of the
fake names (Ann Ersha) with /Ann Ersha
, then Enter
, and scroll between
instances with n
(forwards) or N
(backwards). For each one, replace the
name with your name. Compile and open the file.
Or: You can change all of the names at once using the regular expression:
:%s/Ann Ersha/My Name Here/gc
This will look for every instance of Ann Ersha, and ask if you want to change
it-- type y
for yes, or n
for no.
Then:
- update your email
- update your research experience (what labs have you been in)
- update your affiliations
This is basically identical to my CV format (the format is given is res.cls
),
and in my opinion looks very professional. People will be able to tell it is
made in latex, too, and they will be impressed!
Often, journals only have a .ris
(RIS) type of citations (like
pmcid-PMC5080291.ris
in this directory; cat
this file to see what it looks
like compared to the .bib
file). Using shell commands, we can convert these
RIS citations into BIB citations. We will need some commands in the bibutils
package. Download it with sudo apt install bibutils
(Windows/Ubuntu) or
brew install bibutils
(Mac).
Now we can take a RIS citation and convert it to .xml
, then take the .xml
citation and convert it to .bib
. To do this, run
% cat pmcid-PMC5080291.ris | ris2xml | xml2bib
@Article{Edwards2016,
author="Edwards, Adrianne N.
and Karim, Samiha T.
and Pascual, Ricardo A.
and Jowhar, Lina M.
and Anderson, Sarah E.
and McBride, Shonna M.",
title="Chemical and Stress Resistances of Clostridium difficile Spores and Vegetative Cells",
journal="Front Microbiol",
year="2016",
month="Oct",
day="26",
publisher="Frontiers Media S.A.",
volume="7",
pages="1698",
<--- MANY MORE LINES --->
url="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5080291/"
}
Now we can take this output and pipe it into your .bib
entry, with cat pmcid-PMC5080291.ris | ris2xml | xml2bib >> bibtex_file.bib
. Make sure you use
the redirect >>
instead of the overwrite >
! Once you do this your .bib
file may have a weird <feff> character before your BiBTeX entry, but you can
delete it in the usual way in vim with x
. I often delete the abstract
field of the
BiBTeX entry with dd
, but it is up to you. (An alternative to redirecting
this output is to copy it into the bibtex_file.bib
file-- for me, I do this by
highlighting the text in the shell with a mouse, then middle-clicking in insert
mode in vim)
Now we have a new .bib
entry, so let's use it. Write a line in the
bibtex_file.tex
that includes the command \cite{Edwards2016}
, then
recompile this file with compile_article bibtex_file.tex
, and view the
resulting pdf to ensure your citation worked properly. With this method, you
will never need to make your own citations ever again for scientific articles!
I find myself converting citation styles quite often, since lots of medical
journals don't support .bib
export. Therefore, I have a command makebib
defined in my .bashrc
-- you can have it too! Add the following lines to your
.bashrc
:
# convert .ris to .bib
function makebib() {
cat $1 | ris2xml | xml2bib
}
Then restart your shell, and test the output makebib pmcid-PMC5080291.ris
-- it should act the same as before, and if you desire,
you can redirect it to your bibtex file.
In this script, as in the compile_article
command before, the $1
is
whatever the first parameter you pass to your function is; $2
is the second
variable; and $@
is all of the variables you pass.
See what programs are running (task manager) with top
. If you want it to be
fancy and have colorful graphics, use htop
(install it if you need). See the
low-level hardware commands your system is running with dmesg
. View the
background daemon commands your system is running with journalctl
. Examine
your cpu with lspcu
. See your filesystem with lsblk
and df
. See your
wireless/wired connection hardware profiles with ip link
. Remember your
wireless name (mine is 'wlp4s0'). Use iftop -i wlp4s0
(replace 'wlp4s0' with
your wireless device name) to see the internet traffic you are uploading and
downloading. Use ps
to see currently running processes. Use pkill PROCESS_NAME
to force kill a given process (I sometimes need to use pkill chromium
when my internet bugs out).