Directory tracking in emacs terminal emulators

Emacs has a number of shells and terminal emulators. eshell and shell have some nice features but are not full featured terminal emulators: they can't run interactive programs such as top and less. Other emacs shells like term, ansi-term, and multi-term are full featured terminal emulators, but miss out on some advantages of eshell and shell. Most importantly, they don't do "directory tracking", where emacs uses your working directory in the shell as the default directory for opening files.

You can add directory tracking to term, ansi-term, and multi-term with a little bit of magic in your .bashrc or .zshrc. In my .zshrc, I have this snippet, partially based on this stackoverflow answer:

if [ -n "$INSIDE_EMACS" ]; then
    # function to set the dired and host for ansiterm
    set_eterm_dir() {
        print -P "\033AnSiTu %n"
        print -P "\033AnSiTh" "$(hostname -f)"
        print -P "\033AnSiTc %d"
    }

    # call prmptcmd whenever prompt is redrawn
    precmd_functions=($precmd_functions set_eterm_dir)
fi

Here, the function set_eterm_dir tells emacs what my current working directory is. It's added to the list precmd_functions so that it's called (and my emacs directory updated) whenever the prompt is drawn. set_eterm_dir works by printing out certain escaped strings to the terminal: \033 is an escape character code, while AnSiTu, AnSiTh, and AnSiTc tell the emacs terminal your username, hostname, and working directory. For simple cases only AnSiTc is needed, but if you are also going to enable remote directory tracking (below), you'll need to set AnSiTu and AnSiTh to reset your username and hostname after you're done ssh'ing around.

As mentioned above, you can even set directory tracking on remote machines! Then emacs knows to open files through TRAMP when you're ssh'd somewhere. To do this, you'll need to set your .bashrc or .zshrc on the remote machine. This is from my .bashrc on a remote server:

function set-eterm-dir {
    echo -e "\033AnSiTc" "$(pwd)"

    ## "farm" is the name of the server in my .ssh/config
    ## replace it with your server's name, or with $(hostname -f)
    ## you may also want to add 'echo -e "\033AnSiTu" "$LOGNAME"'
    echo -e "\033AnSiTh" "farm" 
}
if [ "$TERM" = "eterm-color" ]; then
    PROMPT_COMMAND=set-eterm-dir
fi

This snippet is based on a hint from the EmacsWiki but much simplified. PROMPT_COMMAND is called whenever the prompt is drawn in bash. We set it to the function set-eterm-dir that tells the emacs terminal emulator our directory and hostname. Notice I manually set the hostname to "farm". This is because I have named the Host as "farm" in my .ssh/config, and I want TRAMP to use the host settings in that file (e.g. automatic X11 forwarding).

Fix backspace ^? problems in OSX emacs terminal emulators

I sometimes like to use term, ansi-term, or multi-term to fire up shells in emacs. Recently I noticed a subtle problem with these shells in OSX: in certain programs (such as vim and less), the backspace key will insert a string ^? instead of deleting.

The reason for this is that stty erase is unset. You can check this by typing stty -a within these terminal emulators; in the output you will see that

...; erase = <undef>; ...

whereas we want erase = ^?. (^?, i.e. Control+Question mark, is the raw key mapped to backspace).

The terminal emulator should be setting this key from the eterm-color terminfo file, but for some reason I couldn't get this to work properly on OSX (other settings within the terminfo file appeared to be read, but for some reason the setting of backspace was being ignored). So to fix this, I instead wound up adding the following to my .zshrc (use .bashrc if you use bash):

if [[ -n $INSIDE_EMACS && $(uname) == 'Darwin' ]]; then
    stty ek
fi

This snippet checks if we are starting up a shell inside OSX emacs, and if so, manually sets backspace to use the system defaults. Instead of stty ek, you can also do stty erase ^\?.

A parallel einsum

I've written a small package, einsum2, for parallelizing some of the functionality in the einsum function of numpy.

einsum is a fantastic function for manipulating multidimensional arrays. In a nutshell, it's a generalization of matrix multiplication that allows for easy and readable vectorization for all sorts of operations. Unfortunately, it is entirely single-threaded, even though many einsum operations are embarrassingly parallel. By contrast, vanilla matrix or tensor operation (numpy.dot and numpy.tensordot) will be automatically parallel if you link numpy against a parallel implementation of BLAS (such as Intel MKL or OpenBLAS).

Unfortunately, expressing einsum in terms of dot or tensordot can lead to less readable and more verbose code, and in many cases is not even possible. One common example is "batched matrix multiplication", such as np.einsum("ijk,ikl->ijl", a, b), where we perform a matrix multiplication for each value of the index i. For example, i may be an index ranging over each datapoint in a dataset, and we are outputting a value for each point.

Batched matrix multiplication cannot be expressed as a dot or tensordot. It can be expressed in terms of another function, numpy.matmul, but this operation is still single-threaded (though this may eventually change – last year, Intel introduced batched GEMM operations).

With that in mind, I've written einsum2 to parallelize a subset of the functionality in einsum. It is also compatible with the awesome autograd package, and allows for automatic differentiation.

Under the hood, einsum2 is just a parallel implementation of batched matrix multiplication, and rearranges einsum expressions in terms of batched matrix multiplies. einsum2 can handle einsum expressions that can be rewritten as dot, tensordot, or matmul, but cannot handle einsum operations that involve repeated subscripts on the same array (these correspond to summing over diagonals in einsum). It also only works for two input arrays at a time.

I just wrote einsum2 today and it is still rather rough. Most notably, compiling is a bit awkward, especially on OSX. Any feedback or comments on how to improve would be appreciated!