Shell ScriptingLesson 4 of 4

Processing command output

Capture what a command prints, parse it with the tools you already know, and build small useful scripts from the pieces.

$( )
| while read
awk
cut
grep -c
sort | uniq -c
mapfile
tee
wc -l

50 min5 exercises

After this lesson you can

Capture output into a variable with command substitution and use it in tests and messages.
Parse command output into fields with cut, awk and read, and filter it before looping.
Combine tests, loops and parsing into a complete script with validation, output and a correct exit status.

The idea

Most real scripts are small pipelines of commands you already know, glued together by a few lines of shell. The glue is command substitution: $(command) runs the command and replaces itself with what it printed, trailing newline removed.

today=$(date +%F)
lines=$(wc -l < /etc/passwd)
echo "$today: $lines accounts"

A good script follows a shape: check the inputs, collect the data, decide or loop, report, return a meaningful status. Everything in this lesson fits that shape.

Three ways to take output apart

Need	Use	Example
One column of fixed-separator data	`cut`	`cut -d: -f1 /etc/passwd`
Conditions or columns of variable layout	`awk`	`awk -F: '$3 >= 1000 {print $1}' /etc/passwd`
Process lines in the shell itself	`while read`	`while IFS=: read -r u _ uid _; do ...`

And to summarise: sort | uniq -c | sort -rn counts how often each distinct line occurs and puts the most frequent first. It is the answer to a surprising number of “which X appears most” questions.

Try it

day=$(date +%A)
kernel=$(uname -r | cut -d- -f1)
count=$(grep -c bash /etc/passwd)
echo "Today is $day, kernel $kernel, $count bash users"

Today is Tuesday, kernel 6.12.0, 3 bash users

Parse with awk

awk -F: '$7 !~ /nologin|false/ {print $1}' /etc/passwd

root
alex
dana

-F: sets the field separator, $7 is the seventh field, !~ means “does not match the pattern”. The part in braces runs for every line that passes the condition.

Count by key

grep 'Failed password' sample.log | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn

      3 10.0.0.5
      1 10.0.0.9

NF is the number of fields on the line, so $(NF-3) is the fourth from the end, which is the address in an sshd failure line whatever the user name was. Counting from the end is more robust than counting from the start when earlier fields vary.

A report written and shown at the same time

{
    echo "Host: $(hostname)"
    echo "Date: $(date +%F)"
    echo "Root usage: $(df --output=pcent / | tail -1 | tr -d ' ')"
} | tee report.txt

Host: rhel10
Date: 2026-11-03
Root usage: 18%

Braces group several commands so their combined output goes through one pipe.

Putting it together

#!/bin/bash
if [ $# -ne 1 ]; then
    echo "Usage: $0 SHELL" >&2
    exit 2
fi
found=0
while IFS=: read -r user _ _ _ _ _ shell; do
    if [ "${shell##*/}" = "$1" ]; then
        echo "$user"
        found=1
    fi
done < /etc/passwd
if [ "$found" -eq 0 ]; then
    echo "none"
    exit 1
fi

findusers.sh bash
findusers.sh zsh; echo "status=$?"
findusers.sh; echo "status=$?"

root
alex
none
status=1
Usage: /usr/local/bin/findusers.sh SHELL
status=2

It validates its input, loops over data without a subshell (redirecting the file in at done <), uses a flag to remember whether anything matched, and ends with a status the caller can test. ${shell##*/} strips everything up to the last slash, so /sbin/nologin becomes nologin.

Going further

Quote "$var" and "$(command)" everywhere unless you want word splitting. mapfile -t lines < <(command) loads output into an array, one element per line. set -o pipefail makes a pipeline fail if any stage fails, not just the last. shellcheck finds common mistakes automatically, though it is not installed in the exam. If a script grows past about fifty lines or needs data structures, that is the point to consider Python, but for administration glue, Bash with these building blocks is the right tool and is what EX200 expects.

Practice

0 of 5 done

Do these on your own system. They get harder. Check each result, then tick it off.

Warm-upDone
Put output in a variable
Write a script that stores the output of date +%A, uname -r and the number of lines in /etc/passwd in three variables and prints one sentence using all of them.
Check your result
Output like Today is Tuesday, kernel 6.12.0-55.el10.x86_64, 41 accounts.
Hint
day=$(date +%A). For the count use wc -l < /etc/passwd, which prints only the number.
Solution
```
cat > ~/lab/facts.sh <<'EOT'
#!/bin/bash
day=$(date +%A)
kernel=$(uname -r)
accounts=$(wc -l < /etc/passwd)
echo "Today is $day, kernel $kernel, $accounts accounts"
EOT
chmod +x ~/lab/facts.sh
~/lab/facts.sh
```
PracticeDone
List the accounts that can log in
Write ~/lab/loginusers.sh that prints the names of accounts whose shell is not /sbin/nologin or /usr/sbin/nologin or /bin/false, followed by a count in the form N accounts can log in. Use awk or a while read loop on /etc/passwd.
Check your result
root and your normal users are listed, system accounts are not, and the last line gives the count.
Hint
The shell is field 7. awk -F: '$7 !~ /nologin|false/ {print $1}' /etc/passwd. Count with wc -l or increment a counter.
Solution
```
cat > ~/lab/loginusers.sh <<'EOT'
#!/bin/bash
list=$(awk -F: '$7 !~ /nologin|false/ {print $1}' /etc/passwd)
echo "$list"
echo "$(echo "$list" | wc -l) accounts can log in"
EOT
chmod +x ~/lab/loginusers.sh
~/lab/loginusers.sh
```
PracticeDone
Summarise a log
Using journalctl -u sshd --no-pager -o cat (or /var/log/secure), count how many Failed password lines there are per source IP address, most frequent first. If your lab has none, create a sample file with a few lines such as Failed password for root from 10.0.0.5 port 22 ssh2 and run on that. Put the logic in a script that takes the log file as its argument.
Check your result
Output is a list such as 3 10.0.0.5 and 1 10.0.0.9, sorted by count descending.
Hint
grep 'Failed password' FILE | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn. Count fields from the end with NF.
Solution
```
cat > ~/lab/sample.log <<'EOT'
Failed password for root from 10.0.0.5 port 22 ssh2
Failed password for alex from 10.0.0.9 port 22 ssh2
Failed password for root from 10.0.0.5 port 22 ssh2
Accepted password for bob from 10.0.0.7 port 22 ssh2
Failed password for root from 10.0.0.5 port 22 ssh2
EOT
cat > ~/lab/failures.sh <<'EOT'
#!/bin/bash
[ -r "$1" ] || { echo "Usage: $0 LOGFILE" >&2; exit 2; }
grep 'Failed password' "$1" | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn
EOT
chmod +x ~/lab/failures.sh
~/lab/failures.sh ~/lab/sample.log
```
PracticeDone
Make a one-page report
Write ~/lab/report.sh that writes a report to the file named by its first argument (default report.txt) containing the host name, the date, the root file system usage percentage and the five processes using most memory (ps -eo pid,comm,%mem --sort=-%mem | head -6). Print the same text to the screen at the same time.
Check your result
The file and the screen both show the host, date, usage and five processes (plus header). Running it twice replaces the file, not appends.
Hint
tee FILE writes and prints. Group the commands in { ...; } | tee "$out" so the whole block feeds one tee.
Solution
```
cat > ~/lab/report.sh <<'EOT'
#!/bin/bash
out="${1:-report.txt}"
{
    echo "Host: $(hostname)"
    echo "Date: $(date)"
    echo "Root usage: $(df --output=pcent / | tail -1 | tr -d ' ')"
    echo "Top memory:"
    ps -eo pid,comm,%mem --sort=-%mem | head -6
} | tee "$out"
EOT
chmod +x ~/lab/report.sh
~/lab/report.sh /tmp/report.txt
```
ChallengeDone
An exam-style task: script with a loop, a test and an argument
Write /usr/local/bin/findusers.sh. It takes a shell name as its only argument (for example bash). It lists every account whose login shell ends with that name, one per line, and prints none and exits 1 if there are no matches. With no argument it prints a usage message to standard error and exits 2. Make it executable by all users. Test with bash, nologin and zsh.
Check your result
findusers.sh bash lists root and normal users. findusers.sh nologin lists system accounts. findusers.sh zsh prints none and echo $? shows 1. Without an argument the exit status is 2.
Hint
awk -F: -v s="$1" '$7 ~ (s "$") {print $1}' /etc/passwd or loop with while IFS=: read -r u x x x x x sh, comparing ${sh##*/} to the argument.
Solution
```
sudo tee /usr/local/bin/findusers.sh <<'EOT'
#!/bin/bash
if [ $# -ne 1 ]; then
    echo "Usage: $0 SHELL" >&2
    exit 2
fi
found=0
while IFS=: read -r user _ _ _ _ _ shell; do
    if [ "${shell##*/}" = "$1" ]; then
        echo "$user"
        found=1
    fi
done < /etc/passwd
if [ "$found" -eq 0 ]; then
    echo "none"
    exit 1
fi
EOT
sudo chmod 755 /usr/local/bin/findusers.sh
findusers.sh bash
findusers.sh zsh; echo $?
findusers.sh; echo $?
```

The idea

Three ways to take output apart

Try it

Parse with awk

Count by key

A report written and shown at the same time

Putting it together

Going further

Put output in a variable

List the accounts that can log in

Summarise a log

Make a one-page report

An exam-style task: script with a loop, a test and an argument