Networking

Unix and Linux network configuration. Multiple network interfaces. Bridged NICs. High-availability network configurations.

Applications

Reviews of latest Unix and Linux software. Helpful tips for application support admins. Automating application support.

Data

Disk partitioning, filesystems, directories, and files. Volume management, logical volumes, HA filesystems. Backups and disaster recovery.

Monitoring

Distributed server monitoring. Server performance and capacity planning. Monitoring applications, network status and user activity.

Commands & Shells

Cool Unix shell commands and options. Command-line tools and application. Things every Unix sysadmin needs to know.

Home » Commands & Shells, Text Processing

Useful awk one-liners

Submitted by on November 19, 2005 – 10:38 pm 7 Comments

Useful awk code snippets for your sysadmin tasks.

pattern matching and processing 

awk 'pattern {action}' filename 

reads one line at a time from file, checks for pattern match, performs action if pattern matched pattern

NR is a special awk variable meaning the line number of the current record
can use a line number, to select a specific line, by comparing it to NR (for example: NR == 2)
can specify a range of line numbers (for example: NR == 2, NR == 4)
can specify a regular expression, to select all lines that match

$n are special awk variables, meaning the value of the nth field (field delimiter is space or tab)

$0 is the entire record
can use field values, by comparing to $n (for example: $3 == 65)
every line is selected if no pattern is specified

instructions

print - print line(s) that match the pattern, or print fields within matching lines
print is default if no action is specified
there are many, many instruction, including just about all C statements with similar syntax
other instructions will be covered in future courses

examples, using the file cars from page 654 of "A Practical Guide to Linux"

awk 'NR == 2, NR == 4' cars - print the 2nd through 4th lines (default action is to print entire line)
awk '/chevy/' cars - print only lines matching regular expression, same as grep 'chevy' cars
awk '{print $3, $1}' cars - print third and first field of all lines (default pattern matches all lines)
awk '/chevy/ {print $3, $1}' cars - print third and first fiield of lines matching regular expression
awk '$3 == 65' cars - print only lines with a third field value of 65
awk '$5 < = 3000' cars - print only lines with a fifth field value that is less than or equal to 3000
the file testfile can be used with the following examples:
awk '{print $1}' testfile - print first field of every record
awk '{print $3 $1}' testfile
awk '{print $3, $1}' testfile - inserts output field separator (variable OFS, default is space)
awk -F, '{print $2}' testfile - specifies that , is input field separator, default is space or tab
awk '$2 ~ /[0-9]/ {print $3, $1}' testfile - searches for reg-exp (a digit) only in the second field
awk '{printf "%-30s%20sn", $3, $2}' testfile - print 3rd field left-justified in a 30 character field, 2nd field right-justified in a 20 character field, then skip to a new line (required with printf)

awk '$3 <= 23' testfile - prints lines where 3rd field has a value <= 23
awk '$3 <='$var1' {print $3}' testfile - $var1 is a shell variable, not an awk variable, e.g. first execute: var1=23
awk '$3<='$2' {$3++} {print $0}' testfile - if field 3 <= argument 2 then increment field 3, e.g. first execute: set xxx 23

awk '$3> 1 && $3 < 23' testfile - prints lines where 3rd field is in range 1 to 23
awk '$3 < 2 || $3 > 4' testfile - prints lines where 3rd field is outside of range 2 to 4
awk '$3 < "4"' testfile - double quotes force string comparison
NF is an awk variable meaning # of fields in current record
awk '! (NF == 4)' testfile - lines without 4 fields
NR is an awk variable meaning # of current record
awk 'NR == 2,NR==7' testfile - range of records from record number 2 to 7
BEGIN is an awk pattern meaning "before first record processed"
awk 'BEGIN {OFS="~"} {print $1, $2}' testfile - print 1st and 2nd field of each record, separated by ~
END is an awk pattern meaning "after last record processed"
awk '{var+=$3} END {print var}' testfile - sum of 3rd fields in all records
awk '{var+=$3} END {print var/NR}' testfile - average of 3rd fields in all records - note that awk handles decimal arithmetic

awk '$5 > var {var=$5} END {print var}' testfile - maximum of 5th fields in all records
awk '$5 > var {var=$5} END {print var}' testfile - maximum of 5th fields in all records
sort -rk5 testfile | awk 'NR==1 {var=$5} var==$5 {print $0}' - print all records with maximum 5th field

Simple awk operations involving functions within the command line:

awk '/chevy/' cars

# Match lines (records) that contain the keyword chevy note that chevy is a regular expression...

awk '{print $3, $1}' cars

# Pattern not specified - therefore, all lines (records) for fields 3 and 1 are displayed
# Note that comma (,) between fields represents delimiter (ie. space)

awk '/chevy/ {print $3, $1}' cars

# Similar to above, but for chevy

awk '/^h/' cars

# Match cars that begin with h

awk '$1 ~ /^h/' cars                       ### useful ###

# Match with field #1 that begins with h

awk '$1 ~ /h/' cars

# Match with field #1 any epression containing the letter h

awk '$2 ~ /^[tm]/ {print $3, $2, "$" $5}' cars

# Match cars that begin with t or m and display field 3 (year), field 2 (model name) and then $ followed by field 4 (price)

--------------------------------------------------------------------------------------------------
Complex awk operations involving functions within the command line:

awk ‘/chevy/ {print $3, $1}’ cars
# prints 3rd & 1st fields of record containing chevy

awk ‘$1 ~ /^c/ {print $2, $3}’ cars
# print 2nd & 3rd fields of record with 1st field beginning with c

awk ‘NR==2 {print $1, $4}’ cars
# prints 1st & 4th fields of record for record #2

awk ‘NR==2, NR==8 {print $2, $3}’ cars
# prints 2nd & 3rd fields of record for records 2 through 8

awk ‘$3 >= 65 {print $3, $1}’ cars
# prints 3rd & 1st fields of record with 3rd field >= 65

awk ‘$5 >= “2000” && $5 < “9000” {print $2, $3}’ cars
# prints 2nd & 3rd fields of record within range of 2000 to under 9000
Print Friendly, PDF & Email

7 Comments »

  • David says:

    my legs are pretty awk like their really short and my calves are super short so idk but basically from my knee to about mid-thigh my legs like go inwards but then past mid-thigh to the beginning of my thigh they go out and ive been trying to get a thigh gap but lately ive been gaining muscle in that area causing the top half of my thighs to go out even more. How can i loose weight in that area and everywhere without gaining muscle and also what stretches or what in general can i do to grow taller and lengthen my legs?

  • Scott Bull says:

    I have my first date with my bf tmrw and i don’t want it to be really awk so i was wondering if any one had any good conversation starters.
    Its my first date with him cause we got together like yesterday

  • kiltakblog says:

    So me and my ex are friends nows but sshe never wants to chill. She always txts and calls me at nightt like as if we were still together. But when i ask her to chill or to go out she always gets all awk or finds an excuse to not go. I think shes affaird to like me again but im not sure. Its so hard for me to understand her. Woman are so complex. Smh. Any advise?

  • Austin says:

    Theres this girl who seems really cool and I wanna get to be friends with her. However, she doesn’t know who I am and I don’t know how to go up and say anything. I can’t just strike up a convo with a stranger in the hallway at school, that would be awk, I think more so for her. How can I get to know her without being creepy? I’m friends with some of her friends but I’ve never even said hi to her b4.

  • Pacman says:

    I’m looking for a good way to understand what the hell I’m doing. I was wondering if there were aany good books or just stick to online tutorials and try to get some basics down first. I messed with it a little bit in PHP and I see the power they have. I’m hooked. I want to master them. Especially for things like sed, awk, grep, and perl.

  • Picean says:

    how are you supposed to kiss a girl. how do you know she wants to be kissed. where are good place to kiss (a movie seems kinda awk or cheesy). do girls like a surprise kiss as a first kiss.

  • DuckieM10 says:

    all my friends have added what schools they are going to next year already and i haven’t yet. i just feel like its late and it would be awk if i added it now.

1 Pingbacks »

Leave a Reply

%d bloggers like this: