1 # subprocess - Subprocesses with accessible I/O streams

     2 #

     3 # For more information about this module, see PEP 324.

     4 #

     5 # This module should remain compatible with Python 2.2, see PEP 291.

     6 #

     7 # Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se>

     8 #

     9 # Licensed to PSF under a Contributor Agreement.

    10 # See http://www.python.org/2.4/license for licensing details.

    11 

    12 r"""subprocess - Subprocesses with accessible I/O streams

    13 

    14 This module allows you to spawn processes, connect to their

    15 input/output/error pipes, and obtain their return codes.  This module

    16 intends to replace several other, older modules and functions, like:

    17 

    18 os.system

    19 os.spawn*

    20 os.popen*

    21 popen2.*

    22 commands.*

    23 

    24 Information about how the subprocess module can be used to replace these

    25 modules and functions can be found below.

    26 

    27 

    28 

    29 Using the subprocess module

    30 ===========================

    31 This module defines one class called Popen:

    32 

    33 class Popen(args, bufsize=0, executable=None,

    34             stdin=None, stdout=None, stderr=None,

    35             preexec_fn=None, close_fds=False, shell=False,

    36             cwd=None, env=None, universal_newlines=False,

    37             startupinfo=None, creationflags=0):

    38 

    39 

    40 Arguments are:

    41 

    42 args should be a string, or a sequence of program arguments.  The

    43 program to execute is normally the first item in the args sequence or

    44 string, but can be explicitly set by using the executable argument.

    45 

    46 On UNIX, with shell=False (default): In this case, the Popen class

    47 uses os.execvp() to execute the child program.  args should normally

    48 be a sequence.  A string will be treated as a sequence with the string

    49 as the only item (the program to execute).

    50 

    51 On UNIX, with shell=True: If args is a string, it specifies the

    52 command string to execute through the shell.  If args is a sequence,

    53 the first item specifies the command string, and any additional items

    54 will be treated as additional shell arguments.

    55 

    56 On Windows: the Popen class uses CreateProcess() to execute the child

    57 program, which operates on strings.  If args is a sequence, it will be

    58 converted to a string using the list2cmdline method.  Please note that

    59 not all MS Windows applications interpret the command line the same

    60 way: The list2cmdline is designed for applications using the same

    61 rules as the MS C runtime.

    62 

    63 bufsize, if given, has the same meaning as the corresponding argument

    64 to the built-in open() function: 0 means unbuffered, 1 means line

    65 buffered, any other positive value means use a buffer of

    66 (approximately) that size.  A negative bufsize means to use the system

    67 default, which usually means fully buffered.  The default value for

    68 bufsize is 0 (unbuffered).

    69 

    70 stdin, stdout and stderr specify the executed programs' standard

    71 input, standard output and standard error file handles, respectively.

    72 Valid values are PIPE, an existing file descriptor (a positive

    73 integer), an existing file object, and None.  PIPE indicates that a

    74 new pipe to the child should be created.  With None, no redirection

    75 will occur; the child's file handles will be inherited from the

    76 parent.  Additionally, stderr can be STDOUT, which indicates that the

    77 stderr data from the applications should be captured into the same

    78 file handle as for stdout.

    79 

    80 If preexec_fn is set to a callable object, this object will be called

    81 in the child process just before the child is executed.

    82 

    83 If close_fds is true, all file descriptors except 0, 1 and 2 will be

    84 closed before the child process is executed.

    85 

    86 if shell is true, the specified command will be executed through the

    87 shell.

    88 

    89 If cwd is not None, the current directory will be changed to cwd

    90 before the child is executed.

    91 

    92 If env is not None, it defines the environment variables for the new

    93 process.

    94 

    95 If universal_newlines is true, the file objects stdout and stderr are

    96 opened as a text files, but lines may be terminated by any of '\n',

    97 the Unix end-of-line convention, '\r', the Macintosh convention or

    98 '\r\n', the Windows convention.  All of these external representations

    99 are seen as '\n' by the Python program.  Note: This feature is only

   100 available if Python is built with universal newline support (the

   101 default).  Also, the newlines attribute of the file objects stdout,

   102 stdin and stderr are not updated by the communicate() method.

   103 

   104 The startupinfo and creationflags, if given, will be passed to the

   105 underlying CreateProcess() function.  They can specify things such as

   106 appearance of the main window and priority for the new process.

   107 (Windows only)

   108 

   109 

   110 This module also defines two shortcut functions:

   111 

   112 call(*popenargs, **kwargs):

   113     Run command with arguments.  Wait for command to complete, then

   114     return the returncode attribute.

   115 

   116     The arguments are the same as for the Popen constructor.  Example:

   117 

   118     retcode = call(["ls", "-l"])

   119 

   120 check_call(*popenargs, **kwargs):

   121     Run command with arguments.  Wait for command to complete.  If the

   122     exit code was zero then return, otherwise raise

   123     CalledProcessError.  The CalledProcessError object will have the

   124     return code in the returncode attribute.

   125 

   126     The arguments are the same as for the Popen constructor.  Example:

   127 

   128     check_call(["ls", "-l"])

   129 

   130 Exceptions

   131 ----------

   132 Exceptions raised in the child process, before the new program has

   133 started to execute, will be re-raised in the parent.  Additionally,

   134 the exception object will have one extra attribute called

   135 'child_traceback', which is a string containing traceback information

   136 from the childs point of view.

   137 

   138 The most common exception raised is OSError.  This occurs, for

   139 example, when trying to execute a non-existent file.  Applications

   140 should prepare for OSErrors.

   141 

   142 A ValueError will be raised if Popen is called with invalid arguments.

   143 

   144 check_call() will raise CalledProcessError, if the called process

   145 returns a non-zero return code.

   146 

   147 

   148 Security

   149 --------

   150 Unlike some other popen functions, this implementation will never call

   151 /bin/sh implicitly.  This means that all characters, including shell

   152 metacharacters, can safely be passed to child processes.

   153 

   154 

   155 Popen objects

   156 =============

   157 Instances of the Popen class have the following methods:

   158 

   159 poll()

   160     Check if child process has terminated.  Returns returncode

   161     attribute.

   162 

   163 wait()

   164     Wait for child process to terminate.  Returns returncode attribute.

   165 

   166 communicate(input=None)

   167     Interact with process: Send data to stdin.  Read data from stdout

   168     and stderr, until end-of-file is reached.  Wait for process to

   169     terminate.  The optional input argument should be a string to be

   170     sent to the child process, or None, if no data should be sent to

   171     the child.

   172 

   173     communicate() returns a tuple (stdout, stderr).

   174 

   175     Note: The data read is buffered in memory, so do not use this

   176     method if the data size is large or unlimited.

   177 

   178 The following attributes are also available:

   179 

   180 stdin

   181     If the stdin argument is PIPE, this attribute is a file object

   182     that provides input to the child process.  Otherwise, it is None.

   183 

   184 stdout

   185     If the stdout argument is PIPE, this attribute is a file object

   186     that provides output from the child process.  Otherwise, it is

   187     None.

   188 

   189 stderr

   190     If the stderr argument is PIPE, this attribute is file object that

   191     provides error output from the child process.  Otherwise, it is

   192     None.

   193 

   194 pid

   195     The process ID of the child process.

   196 

   197 returncode

   198     The child return code.  A None value indicates that the process

   199     hasn't terminated yet.  A negative value -N indicates that the

   200     child was terminated by signal N (UNIX only).

   201 

   202 

   203 Replacing older functions with the subprocess module

   204 ====================================================

   205 In this section, "a ==> b" means that b can be used as a replacement

   206 for a.

   207 

   208 Note: All functions in this section fail (more or less) silently if

   209 the executed program cannot be found; this module raises an OSError

   210 exception.

   211 

   212 In the following examples, we assume that the subprocess module is

   213 imported with "from subprocess import *".

   214 

   215 

   216 Replacing /bin/sh shell backquote

   217 ---------------------------------

   218 output=`mycmd myarg`

   219 ==>

   220 output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]

   221 

   222 

   223 Replacing shell pipe line

   224 -------------------------

   225 output=`dmesg | grep hda`

   226 ==>

   227 p1 = Popen(["dmesg"], stdout=PIPE)

   228 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)

   229 output = p2.communicate()[0]

   230 

   231 

   232 Replacing os.system()

   233 ---------------------

   234 sts = os.system("mycmd" + " myarg")

   235 ==>

   236 p = Popen("mycmd" + " myarg", shell=True)

   237 pid, sts = os.waitpid(p.pid, 0)

   238 

   239 Note:

   240 

   241 * Calling the program through the shell is usually not required.

   242 

   243 * It's easier to look at the returncode attribute than the

   244   exitstatus.

   245 

   246 A more real-world example would look like this:

   247 

   248 try:

   249     retcode = call("mycmd" + " myarg", shell=True)

   250     if retcode < 0:

   251         print >>sys.stderr, "Child was terminated by signal", -retcode

   252     else:

   253         print >>sys.stderr, "Child returned", retcode

   254 except OSError, e:

   255     print >>sys.stderr, "Execution failed:", e

   256 

   257 

   258 Replacing os.spawn*

   259 -------------------

   260 P_NOWAIT example:

   261 

   262 pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")

   263 ==>

   264 pid = Popen(["/bin/mycmd", "myarg"]).pid

   265 

   266 

   267 P_WAIT example:

   268 

   269 retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")

   270 ==>

   271 retcode = call(["/bin/mycmd", "myarg"])

   272 

   273 

   274 Vector example:

   275 

   276 os.spawnvp(os.P_NOWAIT, path, args)

   277 ==>

   278 Popen([path] + args[1:])

   279 

   280 

   281 Environment example:

   282 

   283 os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)

   284 ==>

   285 Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})

   286 

   287 

   288 Replacing os.popen*

   289 -------------------

   290 pipe = os.popen(cmd, mode='r', bufsize)

   291 ==>

   292 pipe = Popen(cmd, shell=True, bufsize=bufsize, stdout=PIPE).stdout

   293 

   294 pipe = os.popen(cmd, mode='w', bufsize)

   295 ==>

   296 pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin

   297 

   298 

   299 (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize)

   300 ==>

   301 p = Popen(cmd, shell=True, bufsize=bufsize,

   302           stdin=PIPE, stdout=PIPE, close_fds=True)

   303 (child_stdin, child_stdout) = (p.stdin, p.stdout)

   304 

   305 

   306 (child_stdin,

   307  child_stdout,

   308  child_stderr) = os.popen3(cmd, mode, bufsize)

   309 ==>

   310 p = Popen(cmd, shell=True, bufsize=bufsize,

   311           stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)

   312 (child_stdin,

   313  child_stdout,

   314  child_stderr) = (p.stdin, p.stdout, p.stderr)

   315 

   316 

   317 (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize)

   318 ==>

   319 p = Popen(cmd, shell=True, bufsize=bufsize,

   320           stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)

   321 (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)

   322 

   323 

   324 Replacing popen2.*

   325 ------------------

   326 Note: If the cmd argument to popen2 functions is a string, the command

   327 is executed through /bin/sh.  If it is a list, the command is directly

   328 executed.

   329 

   330 (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)

   331 ==>

   332 p = Popen(["somestring"], shell=True, bufsize=bufsize

   333           stdin=PIPE, stdout=PIPE, close_fds=True)

   334 (child_stdout, child_stdin) = (p.stdout, p.stdin)

   335 

   336 

   337 (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode)

   338 ==>

   339 p = Popen(["mycmd", "myarg"], bufsize=bufsize,

   340           stdin=PIPE, stdout=PIPE, close_fds=True)

   341 (child_stdout, child_stdin) = (p.stdout, p.stdin)

   342 

   343 The popen2.Popen3 and popen2.Popen4 basically works as subprocess.Popen,

   344 except that:

   345 

   346 * subprocess.Popen raises an exception if the execution fails

   347 * the capturestderr argument is replaced with the stderr argument.

   348 * stdin=PIPE and stdout=PIPE must be specified.

   349 * popen2 closes all filedescriptors by default, but you have to specify

   350   close_fds=True with subprocess.Popen.

   351 """

   352 

   353 import sys

   354 mswindows = (sys.platform == "win32")

   355 

   356 import os

   357 import types

   358 import traceback

   359 import gc

   360 import signal

   361 

   362 # Exception classes used by this module.

   363 class CalledProcessError(Exception):

   364     """This exception is raised when a process run by check_call() returns

   365     a non-zero exit status.  The exit status will be stored in the

   366     returncode attribute."""

   367     def __init__(self, returncode, cmd):

   368         self.returncode = returncode

   369         self.cmd = cmd

   370     def __str__(self):

   371         return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode)

   372 

   373 

   374 if mswindows:

   375     import threading

   376     import msvcrt

   377     if 0: # <-- change this to use pywin32 instead of the _subprocess driver

   378         import pywintypes

   379         from win32api import GetStdHandle, STD_INPUT_HANDLE, \

   380                              STD_OUTPUT_HANDLE, STD_ERROR_HANDLE

   381         from win32api import GetCurrentProcess, DuplicateHandle, \

   382                              GetModuleFileName, GetVersion

   383         from win32con import DUPLICATE_SAME_ACCESS, SW_HIDE

   384         from win32pipe import CreatePipe

   385         from win32process import CreateProcess, STARTUPINFO, \

   386                                  GetExitCodeProcess, STARTF_USESTDHANDLES, \

   387                                  STARTF_USESHOWWINDOW, CREATE_NEW_CONSOLE

   388         from win32process import TerminateProcess

   389         from win32event import WaitForSingleObject, INFINITE, WAIT_OBJECT_0

   390     else:

   391         from _subprocess import *

   392         class STARTUPINFO:

   393             dwFlags = 0

   394             hStdInput = None

   395             hStdOutput = None

   396             hStdError = None

   397             wShowWindow = 0

   398         class pywintypes:

   399             error = IOError

   400 else:

   401     import select

   402     import errno

   403     import fcntl

   404     import pickle

   405 

   406 __all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "CalledProcessError"]

   407 

   408 try:

   409     MAXFD = os.sysconf("SC_OPEN_MAX")

   410 except:

   411     MAXFD = 256

   412 

   413 # True/False does not exist on 2.2.0

   414 #try:

   415 #    False

   416 #except NameError:

   417 #    False = 0

   418 #    True = 1

   419 

   420 _active = []

   421 

   422 def _cleanup():

   423     for inst in _active[:]:

   424         if inst._internal_poll(_deadstate=sys.maxint) >= 0:

   425             try:

   426                 _active.remove(inst)

   427             except ValueError:

   428                 # This can happen if two threads create a new Popen instance.

   429                 # It's harmless that it was already removed, so ignore.

   430                 pass

   431 

   432 PIPE = -1

   433 STDOUT = -2

   434 

   435 

   436 def call(*popenargs, **kwargs):

   437     """Run command with arguments.  Wait for command to complete, then

   438     return the returncode attribute.

   439 

   440     The arguments are the same as for the Popen constructor.  Example:

   441 

   442     retcode = call(["ls", "-l"])

   443     """

   444     return Popen(*popenargs, **kwargs).wait()

   445 

   446 

   447 def check_call(*popenargs, **kwargs):

   448     """Run command with arguments.  Wait for command to complete.  If

   449     the exit code was zero then return, otherwise raise

   450     CalledProcessError.  The CalledProcessError object will have the

   451     return code in the returncode attribute.

   452 

   453     The arguments are the same as for the Popen constructor.  Example:

   454 

   455     check_call(["ls", "-l"])

   456     """

   457     retcode = call(*popenargs, **kwargs)

   458     cmd = kwargs.get("args")

   459     if cmd is None:

   460         cmd = popenargs[0]

   461     if retcode:

   462         raise CalledProcessError(retcode, cmd)

   463     return retcode

   464 

   465 

   466 def list2cmdline(seq):

   467     """

   468     Translate a sequence of arguments into a command line

   469     string, using the same rules as the MS C runtime:

   470 

   471     1) Arguments are delimited by white space, which is either a

   472        space or a tab.

   473 

   474     2) A string surrounded by double quotation marks is

   475        interpreted as a single argument, regardless of white space

   476        or pipe characters contained within.  A quoted string can be

   477        embedded in an argument.

   478 

   479     3) A double quotation mark preceded by a backslash is

   480        interpreted as a literal double quotation mark.

   481 

   482     4) Backslashes are interpreted literally, unless they

   483        immediately precede a double quotation mark.

   484 

   485     5) If backslashes immediately precede a double quotation mark,

   486        every pair of backslashes is interpreted as a literal

   487        backslash.  If the number of backslashes is odd, the last

   488        backslash escapes the next double quotation mark as

   489        described in rule 3.

   490     """

   491 

   492     # See

   493     # http://msdn.microsoft.com/library/en-us/vccelng/htm/progs_12.asp

   494     result = []

   495     needquote = False

   496     for arg in seq:

   497         bs_buf = []

   498 

   499         # Add a space to separate this argument from the others

   500         if result:

   501             result.append(' ')

   502 

   503         needquote = (" " in arg) or ("\t" in arg) or ("|" in arg) or not arg

   504         if needquote:

   505             result.append('"')

   506 

   507         for c in arg:

   508             if c == '\\':

   509                 # Don't know if we need to double yet.

   510                 bs_buf.append(c)

   511             elif c == '"':

   512                 # Double backslashes.

   513                 result.append('\\' * len(bs_buf)*2)

   514                 bs_buf = []

   515                 result.append('\\"')

   516             else:

   517                 # Normal char

   518                 if bs_buf:

   519                     result.extend(bs_buf)

   520                     bs_buf = []

   521                 result.append(c)

   522 

   523         # Add remaining backslashes, if any.

   524         if bs_buf:

   525             result.extend(bs_buf)

   526 

   527         if needquote:

   528             result.extend(bs_buf)

   529             result.append('"')

   530 

   531     return ''.join(result)

   532 

   533 

   534 class Popen(object):

   535     def __init__(self, args, bufsize=0, executable=None,

   536                  stdin=None, stdout=None, stderr=None,

   537                  preexec_fn=None, close_fds=False, shell=False,

   538                  cwd=None, env=None, universal_newlines=False,

   539                  startupinfo=None, creationflags=0):

   540         """Create new Popen instance."""

   541         _cleanup()

   542 

   543         self._child_created = False

   544         if not isinstance(bufsize, (int, long)):

   545             raise TypeError("bufsize must be an integer")

   546 

   547         if mswindows:

   548             if preexec_fn is not None:

   549                 raise ValueError("preexec_fn is not supported on Windows "

   550                                  "platforms")

   551             if close_fds and (stdin is not None or stdout is not None or

   552                               stderr is not None):

   553                 raise ValueError("close_fds is not supported on Windows "

   554                                  "platforms if you redirect stdin/stdout/stderr")

   555         else:

   556             # POSIX

   557             if startupinfo is not None:

   558                 raise ValueError("startupinfo is only supported on Windows "

   559                                  "platforms")

   560             if creationflags != 0:

   561                 raise ValueError("creationflags is only supported on Windows "

   562                                  "platforms")

   563 

   564         self.stdin = None

   565         self.stdout = None

   566         self.stderr = None

   567         self.pid = None

   568         self.returncode = None

   569         self.universal_newlines = universal_newlines

   570 

   571         # Input and output objects. The general principle is like

   572         # this:

   573         #

   574         # Parent                   Child

   575         # ------                   -----

   576         # p2cwrite   ---stdin--->  p2cread

   577         # c2pread    <--stdout---  c2pwrite

   578         # errread    <--stderr---  errwrite

   579         #

   580         # On POSIX, the child objects are file descriptors.  On

   581         # Windows, these are Windows file handles.  The parent objects

   582         # are file descriptors on both platforms.  The parent objects

   583         # are None when not using PIPEs. The child objects are None

   584         # when not redirecting.

   585 

   586         (p2cread, p2cwrite,

   587          c2pread, c2pwrite,

   588          errread, errwrite) = self._get_handles(stdin, stdout, stderr)

   589 

   590         self._execute_child(args, executable, preexec_fn, close_fds,

   591                             cwd, env, universal_newlines,

   592                             startupinfo, creationflags, shell,

   593                             p2cread, p2cwrite,

   594                             c2pread, c2pwrite,

   595                             errread, errwrite)

   596 

   597         # On Windows, you cannot just redirect one or two handles: You

   598         # either have to redirect all three or none. If the subprocess

   599         # user has only redirected one or two handles, we are

   600         # automatically creating PIPEs for the rest. We should close

   601         # these after the process is started. See bug #1124861.

   602         if mswindows:

   603             if stdin is None and p2cwrite is not None:

   604                 os.close(p2cwrite)

   605                 p2cwrite = None

   606             if stdout is None and c2pread is not None:

   607                 os.close(c2pread)

   608                 c2pread = None

   609             if stderr is None and errread is not None:

   610                 os.close(errread)

   611                 errread = None

   612 

   613         if p2cwrite is not None:

   614             self.stdin = os.fdopen(p2cwrite, 'wb', bufsize)

   615         if c2pread is not None:

   616             if universal_newlines:

   617                 self.stdout = os.fdopen(c2pread, 'rU', bufsize)

   618             else:

   619                 self.stdout = os.fdopen(c2pread, 'rb', bufsize)

   620         if errread is not None:

   621             if universal_newlines:

   622                 self.stderr = os.fdopen(errread, 'rU', bufsize)

   623             else:

   624                 self.stderr = os.fdopen(errread, 'rb', bufsize)

   625 

   626 

   627     def _translate_newlines(self, data):

   628         data = data.replace("\r\n", "\n")

   629         data = data.replace("\r", "\n")

   630         return data

   631 

   632 

   633     def __del__(self, sys=sys):

   634         if not self._child_created:

   635             # We didn't get to successfully create a child process.

   636             return

   637         # In case the child hasn't been waited on, check if it's done.

   638         self._internal_poll(_deadstate=sys.maxint)

   639         if self.returncode is None and _active is not None:

   640             # Child is still running, keep us alive until we can wait on it.

   641             _active.append(self)

   642 

   643 

   644     def communicate(self, input=None):

   645         """Interact with process: Send data to stdin.  Read data from

   646         stdout and stderr, until end-of-file is reached.  Wait for

   647         process to terminate.  The optional input argument should be a

   648         string to be sent to the child process, or None, if no data

   649         should be sent to the child.

   650 

   651         communicate() returns a tuple (stdout, stderr)."""

   652 

   653         # Optimization: If we are only using one pipe, or no pipe at

   654         # all, using select() or threads is unnecessary.

   655         if [self.stdin, self.stdout, self.stderr].count(None) >= 2:

   656             stdout = None

   657             stderr = None

   658             if self.stdin:

   659                 if input:

   660                     self.stdin.write(input)

   661                 self.stdin.close()

   662             elif self.stdout:

   663                 stdout = self.stdout.read()

   664                 self.stdout.close()

   665             elif self.stderr:

   666                 stderr = self.stderr.read()

   667                 self.stderr.close()

   668             self.wait()

   669             return (stdout, stderr)

   670 

   671         return self._communicate(input)

   672 

   673 

   674     def poll(self):

   675         return self._internal_poll()

   676 

   677 

   678     if mswindows:

   679         #

   680         # Windows methods

   681         #

   682         def _get_handles(self, stdin, stdout, stderr):

   683             """Construct and return tupel with IO objects:

   684             p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite

   685             """

   686             if stdin is None and stdout is None and stderr is None:

   687                 return (None, None, None, None, None, None)

   688 

   689             p2cread, p2cwrite = None, None

   690             c2pread, c2pwrite = None, None

   691             errread, errwrite = None, None

   692 

   693             if stdin is None:

   694                 p2cread = GetStdHandle(STD_INPUT_HANDLE)

   695             if p2cread is not None:

   696                 pass

   697             elif stdin is None or stdin == PIPE:

   698                 p2cread, p2cwrite = CreatePipe(None, 0)

   699                 # Detach and turn into fd

   700                 p2cwrite = p2cwrite.Detach()

   701                 p2cwrite = msvcrt.open_osfhandle(p2cwrite, 0)

   702             elif isinstance(stdin, int):

   703                 p2cread = msvcrt.get_osfhandle(stdin)

   704             else:

   705                 # Assuming file-like object

   706                 p2cread = msvcrt.get_osfhandle(stdin.fileno())

   707             p2cread = self._make_inheritable(p2cread)

   708 

   709             if stdout is None:

   710                 c2pwrite = GetStdHandle(STD_OUTPUT_HANDLE)

   711             if c2pwrite is not None:

   712                 pass

   713             elif stdout is None or stdout == PIPE:

   714                 c2pread, c2pwrite = CreatePipe(None, 0)

   715                 # Detach and turn into fd

   716                 c2pread = c2pread.Detach()

   717                 c2pread = msvcrt.open_osfhandle(c2pread, 0)

   718             elif isinstance(stdout, int):

   719                 c2pwrite = msvcrt.get_osfhandle(stdout)

   720             else:

   721                 # Assuming file-like object

   722                 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())

   723             c2pwrite = self._make_inheritable(c2pwrite)

   724 

   725             if stderr is None:

   726                 errwrite = GetStdHandle(STD_ERROR_HANDLE)

   727             if errwrite is not None:

   728                 pass

   729             elif stderr is None or stderr == PIPE:

   730                 errread, errwrite = CreatePipe(None, 0)

   731                 # Detach and turn into fd

   732                 errread = errread.Detach()

   733                 errread = msvcrt.open_osfhandle(errread, 0)

   734             elif stderr == STDOUT:

   735                 errwrite = c2pwrite

   736             elif isinstance(stderr, int):

   737                 errwrite = msvcrt.get_osfhandle(stderr)

   738             else:

   739                 # Assuming file-like object

   740                 errwrite = msvcrt.get_osfhandle(stderr.fileno())

   741             errwrite = self._make_inheritable(errwrite)

   742 

   743             return (p2cread, p2cwrite,

   744                     c2pread, c2pwrite,

   745                     errread, errwrite)

   746 

   747 

   748         def _make_inheritable(self, handle):

   749             """Return a duplicate of handle, which is inheritable"""

   750             return DuplicateHandle(GetCurrentProcess(), handle,

   751                                    GetCurrentProcess(), 0, 1,

   752                                    DUPLICATE_SAME_ACCESS)

   753 

   754 

   755         def _find_w9xpopen(self):

   756             """Find and return absolut path to w9xpopen.exe"""

   757             w9xpopen = os.path.join(os.path.dirname(GetModuleFileName(0)),

   758                                     "w9xpopen.exe")

   759             if not os.path.exists(w9xpopen):

   760                 # Eeek - file-not-found - possibly an embedding

   761                 # situation - see if we can locate it in sys.exec_prefix

   762                 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),

   763                                         "w9xpopen.exe")

   764                 if not os.path.exists(w9xpopen):

   765                     raise RuntimeError("Cannot locate w9xpopen.exe, which is "

   766                                        "needed for Popen to work with your "

   767                                        "shell or platform.")

   768             return w9xpopen

   769 

   770 

   771         def _execute_child(self, args, executable, preexec_fn, close_fds,

   772                            cwd, env, universal_newlines,

   773                            startupinfo, creationflags, shell,

   774                            p2cread, p2cwrite,

   775                            c2pread, c2pwrite,

   776                            errread, errwrite):

   777             """Execute program (MS Windows version)"""

   778 

   779             if not isinstance(args, types.StringTypes):

   780                 args = list2cmdline(args)

   781 

   782             # Process startup details

   783             if startupinfo is None:

   784                 startupinfo = STARTUPINFO()

   785             if None not in (p2cread, c2pwrite, errwrite):

   786                 startupinfo.dwFlags |= STARTF_USESTDHANDLES

   787                 startupinfo.hStdInput = p2cread

   788                 startupinfo.hStdOutput = c2pwrite

   789                 startupinfo.hStdError = errwrite

   790 

   791             if shell:

   792                 startupinfo.dwFlags |= STARTF_USESHOWWINDOW

   793                 startupinfo.wShowWindow = SW_HIDE

   794                 comspec = os.environ.get("COMSPEC", "cmd.exe")

   795                 args = comspec + " /c " + args

   796                 if (GetVersion() >= 0x80000000L or

   797                         os.path.basename(comspec).lower() == "command.com"):

   798                     # Win9x, or using command.com on NT. We need to

   799                     # use the w9xpopen intermediate program. For more

   800                     # information, see KB Q150956

   801                     # (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp)

   802                     w9xpopen = self._find_w9xpopen()

   803                     args = '"%s" %s' % (w9xpopen, args)

   804                     # Not passing CREATE_NEW_CONSOLE has been known to

   805                     # cause random failures on win9x.  Specifically a

   806                     # dialog: "Your program accessed mem currently in

   807                     # use at xxx" and a hopeful warning about the

   808                     # stability of your system.  Cost is Ctrl+C wont

   809                     # kill children.

   810                     creationflags |= CREATE_NEW_CONSOLE

   811 

   812             # Start the process

   813             try:

   814                 hp, ht, pid, tid = CreateProcess(executable, args,

   815                                          # no special security

   816                                          None, None,

   817                                          int(not close_fds),

   818                                          creationflags,

   819                                          env,

   820                                          cwd,

   821                                          startupinfo)

   822             except pywintypes.error, e:

   823                 # Translate pywintypes.error to WindowsError, which is

   824                 # a subclass of OSError.  FIXME: We should really

   825                 # translate errno using _sys_errlist (or simliar), but

   826                 # how can this be done from Python?

   827                 raise WindowsError(*e.args)

   828 

   829             # Retain the process handle, but close the thread handle

   830             self._child_created = True

   831             self._handle = hp

   832             self.pid = pid

   833             ht.Close()

   834 

   835             # Child is launched. Close the parent's copy of those pipe

   836             # handles that only the child should have open.  You need

   837             # to make sure that no handles to the write end of the

   838             # output pipe are maintained in this process or else the

   839             # pipe will not close when the child process exits and the

   840             # ReadFile will hang.

   841             if p2cread is not None:

   842                 p2cread.Close()

   843             if c2pwrite is not None:

   844                 c2pwrite.Close()

   845             if errwrite is not None:

   846                 errwrite.Close()

   847 

   848 

   849         def _internal_poll(self, _deadstate=None):

   850             """Check if child process has terminated.  Returns returncode

   851             attribute."""

   852             if self.returncode is None:

   853                 if WaitForSingleObject(self._handle, 0) == WAIT_OBJECT_0:

   854                     self.returncode = GetExitCodeProcess(self._handle)

   855             return self.returncode

   856 

   857 

   858         def wait(self):

   859             """Wait for child process to terminate.  Returns returncode

   860             attribute."""

   861             if self.returncode is None:

   862                 obj = WaitForSingleObject(self._handle, INFINITE)

   863                 self.returncode = GetExitCodeProcess(self._handle)

   864             return self.returncode

   865 

   866 

   867         def _readerthread(self, fh, buffer):

   868             buffer.append(fh.read())

   869 

   870 

   871         def _communicate(self, input):

   872             stdout = None # Return

   873             stderr = None # Return

   874 

   875             if self.stdout:

   876                 stdout = []

   877                 stdout_thread = threading.Thread(target=self._readerthread,

   878                                                  args=(self.stdout, stdout))

   879                 stdout_thread.setDaemon(True)

   880                 stdout_thread.start()

   881             if self.stderr:

   882                 stderr = []

   883                 stderr_thread = threading.Thread(target=self._readerthread,

   884                                                  args=(self.stderr, stderr))

   885                 stderr_thread.setDaemon(True)

   886                 stderr_thread.start()

   887 

   888             if self.stdin:

   889                 if input is not None:

   890                     self.stdin.write(input)

   891                 self.stdin.close()

   892 

   893             if self.stdout:

   894                 stdout_thread.join()

   895             if self.stderr:

   896                 stderr_thread.join()

   897 

   898             # All data exchanged.  Translate lists into strings.

   899             if stdout is not None:

   900                 stdout = stdout[0]

   901             if stderr is not None:

   902                 stderr = stderr[0]

   903 

   904             # Translate newlines, if requested.  We cannot let the file

   905             # object do the translation: It is based on stdio, which is

   906             # impossible to combine with select (unless forcing no

   907             # buffering).

   908             if self.universal_newlines and hasattr(file, 'newlines'):

   909                 if stdout:

   910                     stdout = self._translate_newlines(stdout)

   911                 if stderr:

   912                     stderr = self._translate_newlines(stderr)

   913 

   914             self.wait()

   915             return (stdout, stderr)

   916 

   917         def send_signal(self, sig):

   918             """Send a signal to the process

   919             """

   920             if sig == signal.SIGTERM:

   921                 self.terminate()

   922             else:

   923                 raise ValueError("Only SIGTERM is supported on Windows")

   924 

   925         def terminate(self):

   926             """Terminates the process

   927             """

   928             TerminateProcess(self._handle, 1)

   929 

   930         kill = terminate

   931 

   932     else:

   933         #

   934         # POSIX methods

   935         #

   936         def _get_handles(self, stdin, stdout, stderr):

   937             """Construct and return tupel with IO objects:

   938             p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite

   939             """

   940             p2cread, p2cwrite = None, None

   941             c2pread, c2pwrite = None, None

   942             errread, errwrite = None, None

   943 

   944             if stdin is None:

   945                 pass

   946             elif stdin == PIPE:

   947                 p2cread, p2cwrite = os.pipe()

   948             elif isinstance(stdin, int):

   949                 p2cread = stdin

   950             else:

   951                 # Assuming file-like object

   952                 p2cread = stdin.fileno()

   953 

   954             if stdout is None:

   955                 pass

   956             elif stdout == PIPE:

   957                 c2pread, c2pwrite = os.pipe()

   958             elif isinstance(stdout, int):

   959                 c2pwrite = stdout

   960             else:

   961                 # Assuming file-like object

   962                 c2pwrite = stdout.fileno()

   963 

   964             if stderr is None:

   965                 pass

   966             elif stderr == PIPE:

   967                 errread, errwrite = os.pipe()

   968             elif stderr == STDOUT:

   969                 errwrite = c2pwrite

   970             elif isinstance(stderr, int):

   971                 errwrite = stderr

   972             else:

   973                 # Assuming file-like object

   974                 errwrite = stderr.fileno()

   975 

   976             return (p2cread, p2cwrite,

   977                     c2pread, c2pwrite,

   978                     errread, errwrite)

   979 

   980 

   981         def _set_cloexec_flag(self, fd):

   982             try:

   983                 cloexec_flag = fcntl.FD_CLOEXEC

   984             except AttributeError:

   985                 cloexec_flag = 1

   986 

   987             old = fcntl.fcntl(fd, fcntl.F_GETFD)

   988             fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag)

   989 

   990 

   991         def _close_fds(self, but):

   992             os.closerange(3, but)

   993             os.closerange(but + 1, MAXFD)

   994 

   995 

   996         def _execute_child(self, args, executable, preexec_fn, close_fds,

   997                            cwd, env, universal_newlines,

   998                            startupinfo, creationflags, shell,

   999                            p2cread, p2cwrite,

  1000                            c2pread, c2pwrite,

  1001                            errread, errwrite):

  1002             """Execute program (POSIX version)"""

  1003 

  1004             if isinstance(args, types.StringTypes):

  1005                 args = [args]

  1006             else:

  1007                 args = list(args)

  1008 

  1009             if shell:

  1010                 args = ["/bin/sh", "-c"] + args

  1011 

  1012             if executable is None:

  1013                 executable = args[0]

  1014 

  1015             # For transferring possible exec failure from child to parent

  1016             # The first char specifies the exception type: 0 means

  1017             # OSError, 1 means some other error.

  1018             errpipe_read, errpipe_write = os.pipe()

  1019             self._set_cloexec_flag(errpipe_write)

  1020 

  1021             gc_was_enabled = gc.isenabled()

  1022             # Disable gc to avoid bug where gc -> file_dealloc ->

  1023             # write to stderr -> hang.  http://bugs.python.org/issue1336

  1024             gc.disable()

  1025             try:

  1026                 self.pid = os.fork()

  1027             except:

  1028                 if gc_was_enabled:

  1029                     gc.enable()

  1030                 raise

  1031             self._child_created = True

  1032             if self.pid == 0:

  1033                 # Child

  1034                 try:

  1035                     # Close parent's pipe ends

  1036                     if p2cwrite is not None:

  1037                         os.close(p2cwrite)

  1038                     if c2pread is not None:

  1039                         os.close(c2pread)

  1040                     if errread is not None:

  1041                         os.close(errread)

  1042                     os.close(errpipe_read)

  1043 

  1044                     # Dup fds for child

  1045                     if p2cread is not None:

  1046                         os.dup2(p2cread, 0)

  1047                     if c2pwrite is not None:

  1048                         os.dup2(c2pwrite, 1)

  1049                     if errwrite is not None:

  1050                         os.dup2(errwrite, 2)

  1051 

  1052                     # Close pipe fds.  Make sure we don't close the same

  1053                     # fd more than once, or standard fds.

  1054                     if p2cread is not None and p2cread not in (0,):

  1055                         os.close(p2cread)

  1056                     if c2pwrite is not None and c2pwrite not in (p2cread, 1):

  1057                         os.close(c2pwrite)

  1058                     if errwrite is not None and errwrite not in (p2cread, c2pwrite, 2):

  1059                         os.close(errwrite)

  1060 

  1061                     # Close all other fds, if asked for

  1062                     if close_fds:

  1063                         self._close_fds(but=errpipe_write)

  1064 

  1065                     if cwd is not None:

  1066                         os.chdir(cwd)

  1067 

  1068                     if preexec_fn:

  1069                         preexec_fn()

  1070 

  1071                     if env is None:

  1072                         os.execvp(executable, args)

  1073                     else:

  1074                         os.execvpe(executable, args, env)

  1075 

  1076                 except:

  1077                     exc_type, exc_value, tb = sys.exc_info()

  1078                     # Save the traceback and attach it to the exception object

  1079                     exc_lines = traceback.format_exception(exc_type,

  1080                                                            exc_value,

  1081                                                            tb)

  1082                     exc_value.child_traceback = ''.join(exc_lines)

  1083                     os.write(errpipe_write, pickle.dumps(exc_value))

  1084 

  1085                 # This exitcode won't be reported to applications, so it

  1086                 # really doesn't matter what we return.

  1087                 os._exit(255)

  1088 

  1089             # Parent

  1090             if gc_was_enabled:

  1091                 gc.enable()

  1092             os.close(errpipe_write)

  1093             if p2cread is not None and p2cwrite is not None:

  1094                 os.close(p2cread)

  1095             if c2pwrite is not None and c2pread is not None:

  1096                 os.close(c2pwrite)

  1097             if errwrite is not None and errread is not None:

  1098                 os.close(errwrite)

  1099 

  1100             # Wait for exec to fail or succeed; possibly raising exception

  1101             data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB

  1102             os.close(errpipe_read)

  1103             if data != "":

  1104                 os.waitpid(self.pid, 0)

  1105                 child_exception = pickle.loads(data)

  1106                 raise child_exception

  1107 

  1108 

  1109         def _handle_exitstatus(self, sts):

  1110             if os.WIFSIGNALED(sts):

  1111                 self.returncode = -os.WTERMSIG(sts)

  1112             elif os.WIFEXITED(sts):

  1113                 self.returncode = os.WEXITSTATUS(sts)

  1114             else:

  1115                 # Should never happen

  1116                 raise RuntimeError("Unknown child exit status!")

  1117 

  1118 

  1119         def _internal_poll(self, _deadstate=None):

  1120             """Check if child process has terminated.  Returns returncode

  1121             attribute."""

  1122             if self.returncode is None:

  1123                 try:

  1124                     pid, sts = os.waitpid(self.pid, os.WNOHANG)

  1125                     if pid == self.pid:

  1126                         self._handle_exitstatus(sts)

  1127                 except os.error:

  1128                     if _deadstate is not None:

  1129                         self.returncode = _deadstate

  1130             return self.returncode

  1131 

  1132 

  1133         def wait(self):

  1134             """Wait for child process to terminate.  Returns returncode

  1135             attribute."""