Catching Python Subprocess Exception

I have a python script where i am calling another script using subprocess as below. sp = subprocess.Popen('script.py --arg1 --arg2', cwd=GIVEN_PATH, shell=True, stdout=sub

Solution 1:

No, you can't get the sys.exc_info() from a subprocess -- by the time you're able to see the return code, objects in memory (including stack traces) of the child process are already gone.

What you can do is parse the text your subprocess writes to stderr, by providing a separate pipe for the stderr output and reading that. Assuming that your program never writes to stderr otherwise, the only text that will show up in that stream will be the error message and stack trace.

You'll want to be cautious with this approach, though, because if the process writes more text to stderr than you can buffer, you'll deadlock. The best solution is to have a separate thread to read from both stdout and stderr in parallel.

Solution 2:

If you want to get sys.exc_info() then you could import the module and run functions from it instead of using subprocess module. You could use multiprocessing module if you need to run in a different process.

Another possibility is to use execfile() to run the module as a script in the same process as the parent script. If you want to pass command-line arguments to the script then you could manipulate sys.argv global list before calling execfile().

Or you could combine subprocess + execfile methods: write a wrapper script that uses execfile to run the child script and serializes all exceptions (e.g., using pickle) and run the wrapper script using subprocess in your parent script. Capture stderr separately, to be able to unpickle child's exception info.