2 of 4

Working with TextLesson 2 of 4

Searching with grep and regular expressions

Find the lines that matter in configuration files and logs, using patterns instead of exact words.

  • grep
  • grep -E
  • grep -i
  • grep -v
  • grep -c
  • grep -r
  • grep -n
  • grep -A/-B

45 min5 exercises

After this lesson you can

  • Search files and directories with the options you need most: -i, -v, -n, -c, -r, -l, -w.
  • Write basic and extended regular expressions with anchors, classes, repetition and alternation.
  • Show a configuration file without its comments and blank lines.

The idea

grep prints the lines of its input that match a pattern. The pattern is a regular expression: ordinary characters match themselves, and a few special characters describe shapes. Learning a dozen of them is enough for nearly all administration work.

PatternMeaning
^ and $Start and end of a line
.Any one character
[abc], [0-9], [^:]One character from a set, a range, or not in a set
*Zero or more of the previous item
|, +, ?, {n,m}, ( )Alternation, one or more, optional, counted repeat, grouping. Plain grep needs a backslash for these; grep -E takes them as written

The options you will use most:

OptionEffect
-iIgnore case
-vInvert: lines that do not match
-nShow line numbers
-cPrint only a count of matching lines
-rSearch directories recursively
-lPrint only the names of files that match
-wMatch whole words only
-oPrint only the matching part
-A n, -B n, -C nShow n lines after, before, or around each match

Always put the pattern in single quotes so the shell does not expand *, $ or | before grep sees them.

Try it

grep -n '^root' /etc/passwd
grep -c nologin /etc/passwd
1:root:x:0:0:Super User:/root:/bin/bash
16
grep -Ev '^\s*(#|$)' /etc/ssh/sshd_config
Include /etc/ssh/sshd_config.d/*.conf
AuthorizedKeysFile	.ssh/authorized_keys
Subsystem	sftp	/usr/libexec/openssh/sftp-server

The same file is mostly commented advice. This pattern says: not (zero or more spaces, then # or the end of line). It is the quickest way to see what a service is really configured to do.

grep -E '^[^:]*:[^:]*:[0-9]{4,}:' /etc/passwd
alex:x:1000:1000:alex:/home/alex:/bin/bash
bob:x:1001:1001::/home/bob:/bin/bash
nobody:x:65534:65534:Kernel Overflow User:/:/sbin/nologin

[^:]*: reads “any run of non-colon characters, then a colon”, which skips one field.

grep -C 1 PermitRootLogin /etc/ssh/sshd_config
grep -rl PermitRootLogin /etc 2>/dev/null
# Authentication:

#LoginGraceTime 2m
#PermitRootLogin prohibit-password
#StrictModes yes
/etc/ssh/sshd_config

Going further

egrep is grep -E. For patterns you build from a variable, quote the variable: grep -- "$pattern" file (the -- ends option parsing in case the pattern begins with -). grep -q prints nothing and only sets the exit status, which is how scripts test whether a line exists: grep -q '^alex:' /etc/passwd && echo found.

Practice

0 of 5 done

Do these on your own system. They get harder. Check each result, then tick it off.

  1. Warm-up

    Find a user

    Print the line of /etc/passwd for the user root, with its line number. Then print how many lines mention nologin.

    Check your result

    The first command prints 1:root:x:0:0:.... The count is a number such as 16.

    Hint

    -n adds line numbers and -c counts matching lines.

    Solution
    grep -n '^root' /etc/passwd
grep -c nologin /etc/passwd
  2. Practice

    Show only the live settings

    Print /etc/ssh/sshd_config without comment lines (starting with #, possibly after spaces) and without blank lines.

    Check your result

    Every printed line is a real directive. grep -c '^\s*#' on the same file counts the comments you removed.

    Hint

    Invert the match with -v, and use -E to combine two patterns with |. A blank line matches ^$.

    Solution
    grep -Ev '^\s*(#|$)' /etc/ssh/sshd_config
  3. Practice

    Write precise patterns

    From /etc/passwd, print only (a) accounts whose user ID is exactly 1000 or higher, using a pattern on the third field, and (b) lines that end in bash. Use only grep.

    Check your result

    (a) prints lines such as alex:x:1000:1000:.... (b) prints the lines for root, alex and others whose shell is /bin/bash.

    Hint

    For (a): ^[^:]:[^:]:[0-9]{4,}: with -E. For (b): bash$.

    Solution
    grep -E '^[^:]*:[^:]*:[0-9]{4,}:' /etc/passwd
grep 'bash$' /etc/passwd
  4. Practice

    Search a tree and show context

    Find which files under /etc contain the word PermitRootLogin, listing only file names and hiding errors. Then show that line in its file together with the two lines before and after it.

    Check your result

    The first command prints file names such as /etc/ssh/sshd_config. The second prints five lines with the match in the middle.

    Hint

    -r recurses, -l prints names only, -C 2 adds context. Send stderr to /dev/null.

    Solution
    grep -rl PermitRootLogin /etc 2>/dev/null
grep -C 2 PermitRootLogin /etc/ssh/sshd_config
  5. Challenge

    Pull every IPv4 address out of a file

    Create ~/lab/net.txt with a few lines of mixed text that include IPv4 addresses (for example host a is 10.0.0.5, gateway 10.0.0.1). Print only the addresses themselves, one per line, sorted and without duplicates.

    Check your result

    Each output line is one dotted address and none repeats.

    Hint

    -o prints only the matching part. A rough pattern is [0-9]{1,3}(\.[0-9]{1,3}){3} with -E.

    Solution
    cat > ~/lab/net.txt <<'EOT'
host a is 10.0.0.5, gateway 10.0.0.1
host b is 10.0.0.5 again
dns 192.168.1.254 is up
EOT
grep -Eo '[0-9]{1,3}(\.[0-9]{1,3}){3}' ~/lab/net.txt | sort -u