Expansions

Expansions, substitution and splitting are performed on the command line after it has been split into tokens.

1. brace expansion     [preamble]{string[,…] } [postamble]
2. tilde expansion     ~/myfile ; ~tony is tony's home directory.
3. parameter and variable expansion    ${parameter} $variableName
4. command substitution     `command`
5. arithmetic expansion     $(( expression ))
6. word splitting     $IFS are word splitters unless null (set|grep IFS)
7. filename expansion     * ? ( [

On some systems process substitution is performed at the same time as parameter, variable, and arithmetic expansion and command substitution.

Brace expansion, word splitting, filename expansion and "$@" can change the number of words of the expansion.
Special Parameters and "${name[@]}" (see Arrays).

After all expansions, Quote Removal is performed.

1. Brace Expansion ^†

   A mechanism to generate strings with a common section.
   Not to be confused with array references for example ${arrayname[0]}.
   Patterns to be expanded are in the form:[preamble]{string[,…] } [postamble]
   preamble is prepended to each string then the postamble is appended to each resulting string, expanding left to right.

   Brace expansions may be nested.
   For example:
   

   a{3,2,1}e expands to a3e a2e a1e
   {0..12}            0 1 2 3 4 5 6 7 8 9 10 11 12
   {3..-2}            3 2 1 0 -1 -2
   {a..g}             a b c d e f g
   {g..a}             g f e d c b a

   Brace expansion is performed before other expansions, and characters special to other expansions are preserved.
   Expansion is strictly textual.

   This is used as shorthand when there is a common prefix of the strings to be generated : mkdir /usr/local/src/bash/{old,new,dist} expands to: mkdir /usr/local/src/bash/old mkdir /usr/local/src/bash/new mkdir /usr/local/src/bash/dist

   ---

   mkdir {phyics,chem,math,english}{_teacher,_student{1,2,3,4}} expands to mkdir phyics_teacher phyics_student1 phyics_student2 phyics_student3 phyics_student4       chem_teacher chem_student1 chem_student2 chem_student3 chem_student4       math_teacher math_student1 math_student2 math_student3 math_student4       english_teacher english_student1 english_student2 english_student3 english_student4

   ---

   chown root /usr/{ucb/{ex,edit},lib/{ex?.?*,how_ex}} expands to: chown root /usr/ucb/ex /usr/ucb/edit /usr/lib/ex?.?* /usr/lib/how_ex There is most likely a better example ed.

2. Tilde (directory) Expansion

   1. A tilde (~) alone is replaced with the value of $HOME.
   2. A word beging with ~ and the next characters specify a login-name the word is replaced by the home directory of that user.
      

       >  echo my home is ~ 
      my home is  /Volume/Users/home-student/me 
      > echo  His home is  ~mjones 
      His home is /Volume/Users/home-faculty/mjones 

   3. Variable assignments of the form VVVV=~:~uu
      Set search path to my bin directory, then djones/locker

      echo $PATH
      /home/jsmith:/ubin
      > export PATH=~bin:~/djones/locker
      > echo $PATH
      /home/jsmith/bin:/home/djones/locker

   4. Directory
      • ~+ specifies $PWD
      • ~- $OLDPWD
      • ~+[n] specifies n^th entry in directory stack (see dirs).

3. Parameter Expansion ${parameter}

4. Command Substitution

    $(command)   -or-  `command` is replaced with output (STDOUT) of command.
   Trailing newlines are deleted. Embedded newlines may be removed during word splitting.
   
   > date
   > Sun Sep 21 19:17:50 EDT 2008
   > NOW=`date`
   > echo $NOW
   > Sun Sep 21 19:17:50 EDT 2008

   When using the `command` form, backslash(\) escapes only \, $ and `.
   When using the $(command) form, all characters between the parentheses make up the command; none are treated specially.

   Command substitutions may be nested. To nest when using the backquoted form, escape the inner backquotes with backslashes.

   If the substitution occurs within quotes ("), word splitting and filename expansion are not performed.

5. Arithmetic Expansion

   Arithmetic expansion allows for the evaluation of an arithmetic expression and the substitution of the result.
   The format for arithmetic expansion is: $(( expression ))

   expression treated as if it were within double quotes, but a double quote inside the parentheses is not treated specially.
   All tokens in the expression undergo parameter expansion, command substitution, and quote removal.
   Arithmetic substitutions may be nested.

   The evaluation is performed according to the rules of shell arithmetic.
   If the expression is invalid, Bash displays a message indicating failure to the standard error and no substitution occurs.
   For example: syntax error in expression (error token is "~a")

6. Process Substitution

   Process substitution is performed simultaneously with parameter expansion and variable expansion, command substitution, and arithmetic expansion.

   Takes the form   <(list)   or   >(list)

   The process list is run with its input or output connected to a named pipe (FIFO) The name of this file is passed as an argument to the current command as the result of the expansion.
   >(list) form : writing to the file will provide input for list.
   <(list) form : the file passed as an argument should be read to obtain the output of list.

7. Word Splitting

   The results of parameter expansion, command substitution, and arithmetic expansion (that did not occur within double quotes) are scanned for word splitting.

   Each character of $IFS (Input Field Seperator) is a delimiter, and splits the results of the other expansions into words.
   

   • If IFS is null, no word splitting occurs.
   • If IFS is unset, or is the default ( space, tab, newline), then any sequence of IFS characters delimits words.
     
   • If IFS sequences of the whitespace characters space and tab are ignored at the beginning and end of the word, as long as the whitespace character is in the IFS
   • Any character in IFS that is not IFS whitespace, along with any adjacent IFS whitespace characters, delimits a field.
     A sequence of IFS whitespace characters is also treated as a delimiter.
     

   Explicit null arguments ("" or '') are retained.
   Unquoted implicit null arguments (resulting from the expansion of parameters that have no values) are removed.
   If a parameter with no value is expanded within double quotes, a null argument results and is retained.
   If no expansion occurs, no splitting is performed.

8. Filename Expansion (globbing)

   After word splitting, words containing *, ?, (, or [ are patterns     (-f supresses this).
   These are replaced with an alphabetically sorted list of matching file names. ( abc* does not mean anything that starts with abc )
   If no matching file names are found the word is left unchanged unless nullglob is disabled, then the word is removed.
   If nocaseglob is enabled, the match is case insenitive.
   The dot ( ., the hidden file prefix) must be matched explicitly, unless dotglob is set. A dot not as a prefix is not treated specially^†.
   The directory indicators . and .. are always ignored
   The slash ( directory prefix) must always be matched explicitly.

   Filenames that match one of the patterns in GLOBIGNORE are removed from the list of matches.
   Setting GLOBIGNORE implies dotglob, so filenames beginning with a . will match.
   To ignoring filenames beginning with a ., make .* one of the patterns in GLOBIGNORE.
   dotglob is disabled when GLOBIGNORE is unset.

   ls u*	lists files beginning with lower case u
   ls [uU]*	lists files beginning with lower case u OR Upper case U

   Pattern Matching

   Any character in a pattern matches itself, other than the special pattern characters.
   NULL may not occur in a pattern.
   The special pattern characters must be quoted or escaped if they are to be matched literally. (Example use \* to match an astrix.)

   Special pattern characters

   ?	Matches any single character.
   *	Matches any string, including the null string.
   [c1c2…cn]	Matches any one of the enclosed characters. Example: [Dd][Gg]erman
   [c1-c2]	a range ; any character that sorts between them, inclusive, is matched.   A - is matched if it as the first or last character in the range.   A ] is matched if it as the first character in the range. If the first character following the [ is a ! or a ^ then any character not enclosed is matched. Example: To EXCLUDE all non-alphanumeric characters. [!~@#$%^&*!()_+|\=-{}:"';?><,./`,] In the default C locale, [a-dx-z] is equivalent to [abcdxyz]. Some locales sort characters in dictionary order, in which case [a-dx-z] might be equivalent to [aBbCcDdxXyYz]†. To force the use of the C locale set $LC_COLLATEor $LC_ALL to C.
   [:class:]	character class, where class is one of: alpha digit alnum ascii cntrl graph print punct blank space lower upper word xdigit (The word character class matches letters, digits, and the character _.) A character class matches any character belonging to that class. example: [:alpha:] or [:space:].
   [=c=]	equivalence class are matches all characters with the same collation weight (as defined by the current locale) as the character c.
   [.symbol.]	matches the collating symbol symbol.

   If extglob is enabled using the shopt builtin, several extended pattern matching. operators are recognized.

   In the following description, a pattern-list is a list of one or more patterns separated by a |.
   Composite patterns may be formed using one or more of the following sub-patterns:

   ?(pattern-list)	Matches zero or one occurrence of the patterns.
   *(pattern-list)	Matches zero or more "
   +(pattern-list)	Matches one or more "
   @(pattern-list)	Matches exactly one "
   !(pattern-list)	Matches anything except one of the patterns.

9. Quote Removal

   Lastly, unquoted occurrences of \ , ' , and "   (that did not result from an expansion) are removed.

Redirections