Search This Blog


Piped Redirects and Variables

When you try to perform a while loop with input from another command it is possible you will see unexpected results.

Stolen directly from

# Works only in ksh88/ksh93, or bash 4.2 with lastpipe enabled
# In other shells, this will print 0

printf '%s\n' foo bar |
while read -r line
linecount=$((linecount + 1))

echo "total number of lines: $linecount"

The reason for this potentially surprising behaviour, as described above, is that each SubShell introduces a new variable context and environment. The while loop above is executed in a new subshell with its own copy of the variable linecount created with the initial value of '0' taken from the parent shell. This copy then is used for counting. When the while loop is finished, the subshell copy is discarded, and the original variable linecount of the parent (whose value hasn't changed) is used in the echo command.

Different shells exhibit different behaviors in this situation:

BourneShell creates a subshell when the input or output of anything (loops, case etc..) but a simple command is redirected, either by using a pipeline or by a redirection operator ('<', '>').

BASH creates a new process only if the loop is part of a pipeline.

KornShell creates it only if the loop is part of a pipeline, but not if the loop is the last part of it. The read example above actually works in ksh88 and ksh93! (but not mksh)

POSIX specifies the bash behaviour, but as an extension allows any or all of the parts of the pipeline to run without a subshell (thus permitting the KornShell behaviour, as well).

There are a few ways to get around this:

If Input is a file

while read -r line; do linecount=$((linecount + 1)); done < file echo $linecount

Bash Only Process Substitution

# Bash
while read -r line
done < <(grep PATH /etc/profile) echo "total number of lines: $linecount"

Store Output of while loop in variable

unset -v foo
echo nothing | {
while read line; do
echo $foo
echo foo2: $foo

# Example output:
# foo2: bar (ok)

Additional Methods and Explanations

No comments:

Post a Comment