Monday 17 June 2013

SED Examples and Tutorial

Meta-characters

Meta-characters Includes non alpha numeric characters, astrics, $ sign, question mark etc etc.

^ -- matches the charactes(s) at the beginning of a line.
eg:- # sed -ne '/^dog/p' animals.txt


n --- will suppress the output lines, only print matching lines.


p --- print matched lines/contents


$ -- matches the charactes(s) at the end of a line.
eg:- # sed -ne '/dog$/p' animals.txt

Match line which contains only 'dog', bgeinning and end with dog
# sed -ne '/^dog$/p' animals.txt
# sed -ne '/^dog$/' ----- reads from STDIN, give name each time and enter

Using a pipe

# cat animals.txt | sed -ne '/^dog$/Ip'
I --- will pring upper case characters as well
 dog
 DOG

Matches any character(typically except new line)


# sed -ne '/^d...$/IP' animals.txt (^d...$ --- search for word starting with d up to four character (three dots ...) eg:- deer


    Regular Expressions Quantifiers

* --- zero or more matches of the previous character
+ --- 1 or more matches of the previous character
? --- zero or 1 of the previous character

When you use characters like *,+,? etc it needs to escape. here + symbol needs to escaped. ' \ ' is the escape character and use it infront of the + symbol (\+)
 

# sed -ne '/^d.\+/IP' animals.txt

NOTE: escape quantifiers in Regular Expressions using the escape sequence or escape characters ie, \


     Character Classes
 

Allows to search for a range of characters
 [0-9]
 [a-z][A-Z]

# sed -ne '/^d[0-9]/Ip' animals.txt --- It will search with word starting with d and followed by a numeric.

# sed -ne '/^d.\+[0-9]/Ip' animals.txt --- Beginning with a line d and followed by one or more character followed by numeric
 

Note: Character classes match 1 and only 1 character

   Intro To SED
 

sed [options] 'instruction file' | pipe | STDIN

sed -e instruction
sed -f script_file
sed -ne

SED accepts instructions based on '/pattern_to_match/action'
 

# sed -ne '1p' animals.txt ---- 1p means print first line of the file.

# sed -ne '$p' animals.txt ---- prints last printable line of the file. 

 Not print blank space

# sed -ne '2,4p' animals.txt ---- prints lines 2-4 from files

# sed -ne '1!p' animals.txt ---- prints all except line number one

# sed -ne '1,4!p' animals.txt ---- prints all except lines one to four.

# sed -ne '/dog/p' --- any line contain word dog will print --- case sensitive

# sed -ne '/dog/Ip' --- any line contain word dog will print --- case insensitive

# sed -ne '/[0-9]/p' animals.txt --- search for lines with numeric characters

# sed -ne '/cat/,/deer/p' animals.txt ---- print all lines beginning with 

 cat and ending with deer

# sed -ne '/deer/,+2p' animals.txt --- deer followed by two additional 

 line after deer.

     Delete Lines using SED addresses

# sed -ne '/^$/d' animals.txt --- delete blank lines from a file.

Note:- drop '-n' option to see what is deleted when deleting

# sed -e '1~2d' animals.txt --- it delete every 2nd line beginning with 

 line 1  ie, 1,3,5.....

# sed -e '1d' animals.txt --- first line will delete from the file animals.txt

# sed -e '1,4d' animals.txt  ---- removes lines from 1-4 from animals.txt


# sed -i '/^*/d' filename ---- remove the line which starting with *

   Search and replace using sed
 

sed -e 's/find/replace/g' animals.txt -----> replace find with replace LHS(left hand side) supports literals and RegExes RHS supports literals and back references.

s ----> substitute

sed -e 's/abc/123/' -   -----> anter abc when cursor blink
sed -e 's/abc/123/I' -  ---> replaces by case insecitive

Note:- Replcements occur on the FIRST match, unless 'g' is appended to the s/find/repace/g sequence.

g --- globally

Tasks:-
     1. Remove all blank lines
     2. Substitute 'cat', regardless of case, with 'Tiger'

# sed -ne '/^$/d' -e 's/Cat/Tiger/Igp' animals.txt OR

# sed -ne '/^$/d; s/Cat/Tiger/Igp' animals.txt 
 Like this also we can substitute instead of giving -e option
  
In first line We pass multiple commands at a time by using -e option. Here first sed repace all blank lines in the file and second sed replace all Cat with Tiger.
     
Note:- Whenever using '-n' option you must specify the print modifier '-p' in the action section. If you are using -n use -p also to print the affected line or it wont print the affected lines.


  Update source File -- Backup Source File

# sed -i.bak -e '/^$/d; s/cat/tiger/Igp' animals.txt 

Here it performs as above but also replaces the source file and backs it 
 up with .bak extensions.
 

Note:- there is no space between -i and .bak

   Search and Replace

# sed -e '/Keyword/s/find/replace/g' file
 

sed -e '/tiger/s/dog/mutt/g' animals.txt --- It will search every line with tiger and in it dog will replace with mutt
 

sed -ne '/tiger/s/dog/mutt/gp' animals.txt --- substitute dog with mutt where line contains tiger
 

sed -ne '/tiger/s/dog/mutt/gIp' animals.txt --- case insensitive match

# sed -ne '/^tiger/s/dog/mutt/gIp' animals.txt --- it will replace only if the line begin with tiger

Dog tiger ---- this line it wont do anything
tiger dog ---- it will replace

# sed -ne '/^tiger/Is/dog/mutt/gIp' animals.txt  ----- replace lines start with tiger case insensitively in this the I is passed to the address section and s is to rest

#### Focus on RHS side of search and replace function ####
Note:- SED reserves a few characters to help with the substition based on the matched pattern from the LHS

& = the full vlue of LHS(patterned mathc) OR the values or info stored in pattern space

Taks:- interspears each line with the word 'Animal'
# sed -ne 's/.*/&/p' animals.txt ----- replaces the matched pattern with the matched pattern


.* ---- means all characters on a line, minus new line characters

# sed -ne 's/.*/Animal &/p' animals.txt
.* search for everything on a line and replace it with Animal and followed by orginal contentent on the line followed by print.


interspears Animal on each line

# sed -ne 's/.*/Animal: &/p' animals.txt
if this : character gives any error escape it using a back slash(\)

then it wont give any error.

Taks:- values ended with numeric

# sed -ne 's/.*[0-9]/&/p/' animals.txt --- returns animals with atleast one numeric at the end

man2007
dog4
horse

# sed -ne 's/.*[0-9]\{1\}/&/p/' animals.txt --- returns animals with only one numeric at the end


# sed -ne 's/.*[0-9]\{4\}/&/p/' animals.txt --- returns animals with four numeric values at the end
 

# sed -ne 's/.*[0-9]\{1,4\}/&/p/' animals.txt --- print numeric characters that end with one-four characters at the end of the name.

\{1\} ---- curley braces should be escaped.
Within curley braces indicate number of matches that could appley. One and only one numeric character that could match.

# sed -ne 's/[a-z][0-9]\{1\}$/&/p/' animals.txt --- all lower case characters as well
 

.* -- should also apply to numeric values

to ensure only one numeric comes at the end of the line you should include $.
 

     Grouping and BackReferences

Note:- Segments matches into backreferences using escaped  parenthesis: \{RegEx\}

# sed -ne 's/\{.*\}\{[0-9]\}/&/p' animals.txt ---- This creates two variables: \1 & \2
display all the items contains numeric values

# sed -ne 's/\{.*\}\{[0-9]\}$/\1/p' animals.txt ---- This creates two variables: \1 & \2 but references to \1
 

# sed -ne 's/\{.*\}\{[0-9]\}$/\2/p' animals.txt ---- This creates two variables: \1 & \2 but references to \2

# sed -ne 's/\{.*\}\{[0-9]\}$/\1\2/p' animals.txt
deer2009
dog1
cat2
DOG1
CAT2
dog12
cat123

# sed -ne 's/\{.*\}\{[0-9]\}$/\1 \2/p' animals.txt
deer200 9
dog 1
cat 2
DOG 1
CAT 2
dog 12
cat12 3

# sed -ne 's/\{.*\}\{[0-9]\}$/\1 : \2/p' animals.txt
deer200 : 9
dog : 1
cat : 2
DOG : 1
dog1 : 2
cat12 :3

    Apply changes to multiple files
 

SED supports globbing via wildcards: *, ? etc

ex:- create multiple files animals.txt,animals2.txt,animals3.txt

# sed -ne 's/\{.*\}\{[0-9]\}$/\1 \2/p' animals*txt ---  this creates two variables \1 and \2 but references \1 and \2
this text will take effect in all files starting with animals.

   SED Scripts
 

it supports scripting, ie the ability to dump 1 or more instructions into 1 file.

sed -f script_file_name followed by one or more text files to process

sed -f script_file_name text_file

Task: Multiple transformations on animals.text file
1. /^$/d -- remove blank lines
2. s/dog/frog/I ---- substitute dog with frog
3. s/.*/Animals: &/ ---- interspears animals: to output
4. s/animals/mammals/Ig ----- replaced animals with mammals
5. s/\{[a-z]*\}\{[0-9]*\}/\1/Ip --- strips traling numeric values from alphas. here if you use \2 instead of \1 you can see the traling values.

put this command in a file called animals.sed and make sure that your file animals.txt contains blank line between each words

# sed -f animals.sed animals.txt
 

# sed -f animals.sed animals.txt > animals4.txt ----> output will forward to this file.

RangeDescription
'4,10d' Lines starting from 4th till 10th are deleted
'10,4d' Only 10th line is deleted, because sed does not work in reverse direction.
'4,+5d' This will match line 4 in the file, delete that line, continue to delete the next five lines, and then cease its deletion and print the rest
'2,5!d' This will deleted everything except starting from 2nd till 5th line.
'1~3d'  This deletes the first line, steps over the next three lines, and then deletes the fourth line. Sed continues applying this pattern until the end of the file.
'2~2d'  This tells sed to delete the second line, step over the next line, delete the next line, and repeat until the end of the file is reached.
'4,10p' Lines starting from 4th till 10th are printed
'4,d'   This would generate syntax error.
',10d'  This would also generate syntax error.



















No comments:

Post a Comment