Handy bash substitution

2013-02-24T04:06:00

Working on the command line is awesome. But there are times where all the typing gets unwieldy. No one likes to type a long file name a lot of times. Sure, using the up arrow and tab keys can make things faster sometimes, but I'll show you some tricks to make your life easier.

Let's start with a really common operation. You download a tarball, extract, cd and have fun. Usually, you'd do this:

$ wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.7.9.tar.bz2
$ tar -xjf linux-3.7.9.tar.bz2
$ cd linux-3.7.9
$ # bob's your uncle

Using the Tab key, you can save a lot of typing:

$ wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.7.9.tar.bz2
$ tar -xjf l<TAB>
$ cd l<TAB>
$ # bob's your uncle

However, this is highly dependent on the contents of your directory, names in your path and auto-complete configurations.

I'm not being picky here. While testing these commands, I had to press Tab twice. First, I had a directory called libs, so it stopped at li. Second, I had an episode of the Linux Action Show in a file called linuxactionshowep248.mp4, so it stopped at linux. And, after you decompress the file, it stops at linux-3.7.9.

Also, if your auto-completion isn't smart enough to filter names that aren't files or directories when you type Tab after cd and tar, you could have many more conflicts.

I don't mean to bash (ha!) on auto-completion, just point you to more efficient alternatives: bash (get it?) has a nice feature called substitutions. You may have come across it before if you ever tried to add an unescaped exclamation mark (henceforth mentioned using the niftier name "bang"):

$ echo "Bang!"
bash: !": event not found

Or got surprised by it:

$ echo "balrog:!you shall not pass:/nonexistent:/bin/false" >> /etc/passwd
echo "balrog:youtube-dl http://www.youtube.com/watch?v=pLgJ7pk0X-s shall not pass:/nonexistent:/bin/false" >> /tmp/test
$ sleep 3d; grep balrog /etc/passwd
balrog:youtube-dl http://www.youtube.com/watch?v=pLgJ7pk0X-s shall not pass:/nonexistent:/bin/false

That is because the bang is used to reference previous commands. The simplest is substitution of the last command:

$ echo billy
billy
$ !!
echo billy
billy

One thing I should mention: to avoid chaos and destruction (or at least inform you that chaos and destruction have happened) the command with all substitutions made is printed before its output. That is the fourth line in the example, showing that !! was substituted by echo billy.

This substitution is simple and needs no explanation, but it raises a question: what is it useful for? Have you ever typed a (possibly long) command, just to get the following message after running it?

$ cat /sys/module/fglrx/sections/.gnu.linkonce.t._ZN20OS_COMMON_INTERFACES22cailMicroEngineControlEPv14_MICRO_ENGINE_21_MICRO_ENGINE_ACTION_P28_MICRO_ENGINE_CONTROL_INPUT_P29_MICRO_ENGINE_CONTROL_OUTPUT_
cat: /sys/module/fglrx/sections/.gnu.linkonce.t._ZN20OS_COMMON_INTERFACES22cailMicroEngineControlEPv14_MICRO_ENGINE_21_MICRO_ENGINE_ACTION_P28_MICRO_ENGINE_CONTROL_INPUT_P29_MICRO_ENGINE_CONTROL_OUTPUT_:
Permission denied

You can solve this by typing <UP><HOME>sudo<SPACE> (if you type <UP><LEFT>{201}sudo<SPACE>, I will personally punch you in the face). But you can save a lot of typing just using

$ sudo !!
sudo cat /sys/module/fglrx/sections/.gnu.linkonce.t._ZN20OS_COMMON_INTERFACES22cailMicroEngineControlEPv14_MICRO_ENGINE_21_MICRO_ENGINE_ACTION_P28_MICRO_ENGINE_CONTROL_INPUT_P29_MICRO_ENGINE_CONTROL_OUTPUT_
[sudo] password for billy:

Here, !! got substituted by the last command, as you can see on the echo. Some other uses I found (which you may not understand if you're not on the path to enlightenment becoming a bash enthusiast) were:

$ cat my_files
file1
file2
file3
$ cat $(!!)
cat $(cat my_files)
contents_of_file1
contents_of_file2
contents_of_file3
$ perl -e 'bang your head on the keyboard'
# god only knows
$ echo "!!" > file_to_save_command
$ # This could be done by clever use of Ctrl-x Ctrl-e, but
$ # sometimes you're in a hurry. That's alright, as long as
$ # you do it carefully.

When ! is followed by a number n, it executes the nth command in the history (which you can check with the history command). More useful, though, is that, if n is negative, it will execute the last nth command:

$ echo some
some
$ echo thing
thing
$ !-2
echo some
some
$ !-3 other thing
echo some other thing
some other thing

Without thinking much, you can discover that !! is just an easier way to spell !-1.

Now, these are useful techniques, but we are just getting started! Usually, you don't need the whole command, but just one or a few arguments. Common scenario:

$ ls ef
son daughter dog
$ # oh my, ef is a directory
$ cd ef

or how about:

$ mv some_directory a_different_name_for_some_directory
$ cd a_different_name_for_some_directory

It is very common to address the same file on multiple, successive commands. You can, again, save some typing:

$ ls ef
son daughter dog
$ # oh my, ef is a directory
$ cd !$
cd ef
$ mv some_directory a_different_name_for_some_directory
$ cd !$
cd some_directory a_different_name_for_some_directory

As you can see, !$ is substituted by the last argument of the last command. If you know your regular expressions, you can make a link with the "end of line" symbol. But there's more. This is just a nice shortcut to the more general form !n:m. This means: from the nth, get the mth argument.

$ tar -f my_tar -cz file1 file2 file3
$ du -sh !:2
4.0K    my_tar
$ du -sh !-2:4-6
4.0K    file1
4.0K    file2
4.0K    file3

What goes before the colon can be any of the previous substitutions: !!, !n or !-n. Note that you can specify a range using :n-m, which will get substituted by the parameters n to m. If you need them in a different order, or some non-contiguous arguments, just use more than one substitution:

$ diff ours theirs
$ # gah
$ diff !:2 !:1
diff theirs ours

Alright, I guess this is enough information for a single post. Have fun playing with The Bash and see you soon for more advanced uses of history substitution.

P.S.: I couldn't leave without some closing notes.

1. I learned everything shown here from scattered information on the internet. I decided to compile them because I never found a document which explained them in detail AND without burying them on other bash-related commands.

   That said, a good reference (not so good for self-learning, but, if you read the text, you should be okay) is bash's man page. Search for "Event Designators" or \!\! (escaping is mandatory):

   $ man bash
   /Event Designators<CR>

2. While all this may seem hard to use practically, I recommend you just keep it in a dark corner of your mind. I promise you'll start to find places where you can apply it. They can be used to write commands quickly, especially several one-shot commands in a row. Just don't try forcing the use when it's not necessary. After all, they were created to save time.

3. If you ever got stabbed by not knowing how the bang works (like the examples I showed in the beginning), you can avoid getting stabbed again by escaping the bangs, either with a backslash or single quotes:

   $ echo Bang\!
   Bang!
   $ echo 'balrog:!you shall not pass:/nonexistent:/bin/false' >> /etc/passwd

4. More of a side note, but to get that humongous file name, I discovered a new (string of) command(s) to get the longest line from a text.

   $ find /sys/ | awk '{print length, $0;}' | sort -nr | head -n 1

   You can probably do the same thing (although it might be a little slower) using a while loop and another bash trick: while $variable (or ${variable}) will give you the contents of a variable, ${#variable} will give you the length of that content.

   $ find /sys/ | while read i; do echo "${#i} $i"; done | sort -nr | head -n 1

   Apparently, Jethro Tull's Too Old to Rock 'n' Roll, Too Old to Die generates pretty long file names (the title song is the second only to a really long cache file name on my file system).