Stream EDit

non-interactive edit

sed [-n] [[-e] scriptCommands] [-f scriptFile] [file]

Copies file to STDOUT, editing it according to a script of commands.

Examples:
sed s/blue/green/ infile > outfile
sed -e "s/old/older/" -e "s/new/old" needsMultipleChanges.txt
sed -f bunch-o-cmds.sed afile anotherfile > da-SEDed-file
sed "s/;/;\n/" > myprog.c Make multi-statment-lines be on seperate lines)

Warning: sed "s/xx/yy/" file > file results in an empty file! Use sed --in-place "s/xx/yy/" file
Note: sed "s/xx/yy/" file doesn't update file, rather outputs to STDOUT (perhaps the terminal )

-n no default output.
-excute command
-f scriptfile
-n
--quiet
--silent
suppress automatic printing of pattern space
-e script
--expression=script
add the script to the commands to be executed
-f script-file
--file=script-file
add the contents of script-file to the commands to be executed
--follow-symlinks follow symlinks when processing in place
-i[suffix]
--in-place[=suffix]
in place edit (makes backup if suffix is supplied)
Actually creates a temp file sedxxxxx then deletes original and renames sedxxxxx so there must be enough space for sedxxxxx
-l n
--line-length=n
specify the desired line-wrap length for the l command
--posix disable all GNU extensions.
-E
-r
--regexp-extended
use extended regular expressions in the script (for portability use POSIX -E).
-s
--separate
consider files as separate rather than as a single
continuous long stream.
--sandbox operate in sandbox mode.
-u
--unbuffered
load minimal amounts of data from the input files and flush the output buffers more often
-z
--null-data
separate lines by NUL characters
--help display this help and exit
--version output version information and exit

Normally, sed compiles the entire script then cycles through the input:

  • reading a line into a WORK area (unless there is something left after a D ),
  • deleting trailing
  • applying, in sequence, all commands whose address select the WORK area and
  • writing the resulting WORK area to standard output including a

    A script consists of editing commands, one per line:

    [address[,address] ]   function    [arguments]

    A function includes substitute, append, change, delete, and others
    Some of the functions use a HOLD area to save all or part of the WORK area for subsequent retrieval.

    arguments depend on the function

    Notes on regular expressions:

    Function Description
    2 s/regexp/replacement/f

    Substitute the replacement string for instances of the regexp in WORK.

    Character immediating following s will be used as the delimeter ( not necessarily /). Useful if regexp contains slashes.

      /bin/df | sed s:/Volumes/::
    See ed(1).

    flags are:

    • g Global. Substitute all nonoverlapping instances of regexp
    • n = 1 - 512. Substitute for just the nth occurrence of the regexp
      n.b. using 2 for flags changes the second occurance IN THE RANGE OF LINES,
      not the second occurance in every line.
      i.e. only one string will be changed unless you use g
    • p Print WORK if a replacement was made.
    • i ignore case (not supported in Mac OSX version)
    • w file Write the WORK area, appending to file if a replacement was made.

    Special strings:

    • Bracket Expression [ … ] list (ex: [13579], range (ex:[4-9]) and named (ex: [[:digit:]]
    • Repetition ¿\{1,4\} (in this example: at least 1 and as many as 4 ¿s)
    • Subexpression /chapter \(.*\)/ matches chapter numbers (or letters);
      subexpression \1 on the RightHandSide (RHS) as a backreferene.
    •  &  in the replacment signifies the matched regexp Example:
      s/<td/& align=right/
      changes
      <td to <td align=right>

    2 n       W STDOUT
       in W
    next. Output WORK ; read the next line of input to WORK.
    2 N   W + \n + in WAppend a new-line and the Next line of input to WORK.
    Resulting in an embedded new-line, that can be substituted by null, joining the current WORK and the next input line.
    Example: Join continuation of a span tag had been split onto next line
    /<span$/N; s/<span\n/<span/

    Example: Join 3 lines as in the result from head -n1 * into one line:

    >head -n1 mys*  |cut -c1-50
    ==> mysql <==
    if [ $# -lt 1 ] ; then 
    
    ==> mysql.server <==
    #!/bin/sh
    
    ==> mysql.server.vvvqq <==
    #!/bin/sh
    
    ==> mysqladmin.vvvqqq <==
    echo /usr/local/mysql/bin/mysqladmin $*
    >
    >
    > head -n1 *|cut -c1-50 |sed "N;s/\n//;N;s/\n//" 
    ==> mysql <==if [ $# -lt 1 ] ; then 
    ==> mysql.server <==#!/bin/sh
    ==> mysql.server.vvvqq <==#!/bin/sh
    2 G   W + H W WORK Gets [newline] and HOLD appended .
    2 h   W H HOLD gets contents of WORK .
    2 H   H+W H Append WORK to HOLD .
    2 g   H W contents of the HOLD get put to WORK
    2 x   W X H Exchange the contents of WORK and HOLD .
    2 P Print. Copy WORK to the standard output.
    (useful if -n option was used to supress normal output.)
    2 p print. Copy the initial segment of the WORK area through the first new-line to the standard output.
    1 a \
    text
    append text after outputting the selcted line.
    If the text is several lines, all (but the last) must end with \ .

    After the head tag, append a title tag, "nice document" and the ending title tag:

    /<head>/a \
    <title>\
    nice document\
    <title>
    1 i \
    text
    insert text before outputting the selected line. ( it seems that functions that insert should be before functions that reference the entire file.,
    '</body>'i\
    <style>\
    td {text-align:right}\
    </style>
    2 c \
    text
    change. Delete WORK. Output text. Start the next cycle.
    2 d delete WORK. Start the next cycle.
    2 D Delete the initial segment of WORK through the first new-line. Start the next cycle.
    1 r file read file
    file must be the last thing on the line and be preceeded by exactly one blank.
    1 w file write. Append WORK to file
    file must be the last thing on the line and be preceeded by exactly one blank.
    The first occurrence of w causes file to be cleared. Each time sed is used, file is overwritten.

    Each w file is created before processing begins.
    There can be at most 10 distinct w file arguments.

    1 = output the current line number of input.
    (good for debugging) (The current line number changes.)
    2 b [label] Branch to the command bearing the :label.
    If label is omitted, branch to the end of the script.
    2 t [label] test. Branch to the command bearing the :label if any substitutions have been made since the most recent reading of an input line or execution of a t.
    If label is omitted, branch to the end of the script.
    0 :label define a label for b and t commands to branch to.
    1 q quit.
    2 { Execute the following commands through a matching } only when WORK is selected.
    2 ! function Don't apply the function (or group, if function is {…) only to lines not selected by the address(es).
      # comment Special case: On the first line #n, the default output will be suppressed.
    2 y/string1/string2/

    Transform. Replace all occurrences of characters in string1 with the corresponding characters in string2.
    string1 and string2 must have the same number of characters.
      example:
    to change all uppercase to lowercase
    y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/

    2 l (ell) output WORK area in an unambiguous form.
    Non-printable characters are displayed in octal notation and
    long lines are folded.
    EF 82 B7 are NOT altered

    Consider the strings command instead.

    Example:
    change
    Dec 31 19:37
      to
    19:37 Dec 31

    command  results
    s/... .. ..:../& &/ duplicates the entire "date time" string
    Dec 31 19:37 Dec 31 19:37
    s/... ..// deletes the first Dec 31 19:37 Dec 31 19:37
    s/..:..//2deletes the 2nd 19:37 19:37 Dec 31
    (I expect you could use the X command)

    Take lines with message followed by target of message and put target at end of message:

    Host abt.info not found: 3(NXDOMAIN)
    abt
    Host ach.info not found: 3(NXDOMAIN)
    ach
    acj
    
    sed "/NX/N;/NX/s/\n//" 
    
    Host abt.info not found: 3(NXDOMAIN)abt
    Host ach.info not found: 3(NXDOMAIN)ach
    acj
    Change 123,456 to 123456
    substitute from 1 to 3 characters followed by a comma to the same 1 to 3 characters comma and a zz substitute ,zz to null
    s/.\{1,3\},/&zz/ ; s/,zz//
    The new-line escape sequence, \n, matches a new-line embedded in WORK.

    A period . does not match the ending new-line of WORK.
    Use the negation function ! to have commands applied only to non-selected WORK areas .
    Backslashes in text are treated like backslashes in the replacement string of an s command, and may be used to protect initial blanks and tabs against the stripping that is done on every script line.

    command list example:

    s/xxx/yyy/ { n b top }


    See also:tr (translate single characters), awk, ed, grep(includes regular expression discussion), environ

    9 May 10, copy pasted list from a PDF and ended up with x'EF 82 B7 ' . Only way I could get them out was to copy the file and delete everything except the x'EF 82 B7 ' then prefix it with s/ and suffix it with /zzz/g )
    Gotta be a better way! (see Replacing 3byte strings above see also sed-3.02, see also sedFAQ, gnu.org/sed


    Some information taken from SunOS 5.4 man page of 14 Sep 1992 As of 3/19/13 the current version of sed is 4.2.1.
    Most of this is based on the 3.02 version and still is applicable.

    Mac OS uses old version of sed which does not uspport -v and among other things, does not support \n as a new-line in string2 of the substitute command! (see sedFAQ4.6


    Error message like:
    sed: -e expression #1, char 1: …
    might be due to omitting the -f to specify a script file.
    sed file.sed targetFile or sed -e file.sed targetFile
    should be
    sed -f script.sed targetFile