Sed and awk are the most powerful text processing tools on the command line. Sed excels at substitution, awk at extraction and analysis of structured data.
Sed — Stream Editor¶
sed ‘s/foo/bar/g’ soubor.txt sed -i ‘s/foo/bar/g’ soubor.txt sed ‘/^#/d’ config.conf sed ‘/^$/d’ soubor.txt sed -n ‘/BEGIN/,/END/p’ soubor.txt
Awk — Pattern Scanning¶
awk ‘{print $1, $3}’ soubor.txt awk ‘$3 > 100 {print $1, $3}’ data.txt awk -F’:’ ‘{print $1, $7}’ /etc/passwd awk ‘{sum += $2} END {print sum}’ data.txt awk ‘{count[$1]++} END {for (k in count) print k, count[k]}’ access.log
Practical Examples¶
awk ‘{print $1}’ access.log | sort | uniq -c | sort -rn | head awk ‘{print $9}’ access.log | sort | uniq -c | sort -rn awk -F’,’ ‘$3 > 1000 {print $0}’ sales.csv awk -F’:’ ‘$3 >= 1000 {print $1}’ /etc/passwd
When to Use Which¶
- Sed — substitution, deleting lines, in-place editing
- Awk — columns, calculations, aggregation
- Combination — sed for preprocessing, awk for analysis
Indispensable¶
Investing in sed and awk pays off many times over when working with logs and data.