A subprocess.Popen Gotcha

In answering a recent question, I noticed a bit of a gotcha while using
subprocess.Popen. It's important to note the use of the 'args' parameter if
setting the shell argument to True.

When the shell parameter is set to True, the subprocess module essentially
does the following:

exec_list = ['/bin/sh',  '-c' ] +  args
os.execvp(exec_list[0], exec_list)

The 'args' parameter to the Popen initializer can be either a sequence or
a simple string. In most cases, args ought to be a string value here,
such that a shell is executed and in turn fires off the command with
its arguments:

subprocess.Popen('/bin/ls /var/log', shell=True)

Will result in:

/bin/sh -c '/bin/ls /var/log/'

However, it is still perfectly valid to use a exec-style sequence for
'args.' The resulting command executed will be different, however.

subprocess.Popen(('/bin/ls', '/var/log'), shell=True)

Will result in:


/bin/sh -c '/bin/ls' '/var/log'


The end result is quite different. In the first example, '/var/log' is
passed as an argument to '/bin/ls.' In the second example, both '/bin/ls'
and '/var/log' are passed as arguments to '/bin/sh.'

Since there is no immediate error generated, the caller simply winds up with
unexpected results. The same problem doesn't exist in reverse. If a string
with multiple arguments is passed when the default shell=False is used, an
immediate error is raised:

>>> subprocess.Popen('/bin/ls /var/log', shell=False)
Traceback (most recent call last):
 File "", line 1, in 
 File "/usr/local/lib/python2.6/subprocess.py", line 595, in __init__
   errread, errwrite)
 File "/usr/local/lib/python2.6/subprocess.py", line 1106, in _execute_child
   raise child_exception
OSError: [Errno 2] No such file or directory

My assumption is that this is because we're actually trying to execute
'/bin/ls /var/log' as one filename (including a space).

Something to keep in my "what the..?" pile.