What Is Subprocess In Python? [Explained With Examples]

The term “Python subprocess” refers to the use of the Python subprocess module to work with external processes from within a Python script. It allows you to spawn and interact with external processes, such as running system commands, launching other programs, and managing their input/output/error streams. The subprocess module provides a convenient way to execute and communicate with these external processes, making it a valuable tool for system scripting and automation tasks in Python.

Key functions and classes in the subprocess module include:

1. subprocess.run(): This function allows you to run an external command and wait for it to complete. You can capture the command’s output and check its return code.
2. subprocess.Popen(): This class creates a subprocess object, which represents a running external process. You can use it to interact with the process, communicate with its standard input/output/error streams, and manage its execution.
3. subprocess.PIPE: This constant is used to create pipes for the standard input, output, or error streams of a subprocess. It’s often used to capture or redirect the data flowing between your Python script and the external process.
4. subprocess.check_call() and subprocess.check_output(): These functions are wrappers around subprocess.run() that raise exceptions if the command returns a non-zero exit code or encounters an error.

Here’s a simple example of using subprocess.run() to run an external command and capture its output:

import subprocess

result = subprocess.run(["ls", "-l"], stdout=subprocess.PIPE, text=True)
print(result.stdout)
Code language: Python (python)

In this example, the subprocess.run() function runs the “ls -l” command and captures its standard output. The stdout=subprocess.PIPE argument specifies that the standard output should be captured and made available in the result.stdout attribute.

The subprocess module is commonly used in Python for tasks like running system commands, automating system administration tasks, launching and interacting with other programs, and more. It provides a powerful and flexible way to integrate Python with the external world of processes and command-line utilities.

When to use Python Subprocess

You should consider using the Python subprocess module when you need to interact with external processes, run system commands, or automate tasks that involve executing command-line utilities or other programs from within your Python script. Here are some common scenarios in which you might want to use subprocess:

1. Running System Commands: When you need to execute system-level commands, such as file operations (e.g., copying, moving, deleting files), managing processes, or querying system information, subprocess allows you to do so from your Python script.
2. Shell Scripting: If you have a series of shell commands or a shell script that you want to execute as part of a larger Python script, you can use subprocess to run those commands and capture their output.
3. Launching External Programs: You can use subprocess to launch and control external programs or applications from your Python code. This is useful when you want to automate interactions with other software.
4. Interacting with Command-Line Tools: Many command-line tools and utilities provide powerful functionality, and you can use subprocess to integrate them into your Python workflows. For example, you can use tools like grep, awk, or sed within your Python script to process and manipulate data.
5. Parallel Processing: If you need to run multiple processes concurrently or asynchronously, you can use subprocess to manage and coordinate their execution.
6. Capturing Output: subprocess allows you to capture and process the output (stdout and stderr) of external processes, which can be useful for logging, error handling, or further processing.
7. Cross-Platform Compatibility: subprocess is designed to work on different operating systems (Windows, macOS, Linux, etc.), making it a versatile choice for cross-platform scripting.
8. Automating System Administration: You can use subprocess to automate system administration tasks, such as configuring network settings, managing users and groups, or scheduling tasks.
9. Testing and Benchmarking: In some cases, you may want to run external programs as part of your testing or benchmarking procedures to assess performance or verify functionality.

It’s important to use subprocess with caution and follow best practices to ensure security and reliability. Be mindful of potential security risks when running user-provided or untrusted commands, and always validate and sanitize inputs. Additionally, consider error handling and exception handling to gracefully manage failures when working with external processes.

Python Subprocess Examples

Here are some practical examples of using the Python subprocess module for various tasks:

1. Running a Simple Command and Capturing Output:

import subprocess

result = subprocess.run(["ls", "-l"], stdout=subprocess.PIPE, text=True)
print(result.stdout)
Code language: Python (python)

This code runs the “ls -l” command, captures its standard output, and prints it.

2. Running a Command with Arguments:

import subprocess

filename = "example.txt"
subprocess.run(["touch", filename])
Code language: Python (python)

This code uses subprocess.run() to create a new file named “example.txt” by running the “touch” command with an argument.

3. Running a Command with Error Handling:

import subprocess

try:
    subprocess.run(["unknown_command"], check=True)
except subprocess.CalledProcessError as e:
    print(f"Command failed with return code {e.returncode}")
Code language: Python (python)

Here, subprocess.run() attempts to run an unknown command and raises an exception if it fails.

4. Piping Data to a Command:

import subprocess

data = "Hello, World!"
result = subprocess.run(["grep", "World"], input=data, stdout=subprocess.PIPE, text=True)
print(result.stdout)
Code language: Python (python)

This code uses subprocess.run() to pipe the “data” string to the “grep” command, which searches for the word “World” in the input data.

5. Running Shell Commands and Shell Features:

import subprocess

command = "echo 'Hello, Shell!' && ls -l"
result = subprocess.run(command, shell=True, stdout=subprocess.PIPE, text=True)
print(result.stdout)
Code language: Python (python)

Here, the code runs a shell command that consists of multiple commands separated by “&&” using shell=True. It captures and prints the output.

6. Running a Process Asynchronously:

import subprocess

proc = subprocess.Popen(["ping", "example.com"])
# Continue with other tasks while the ping command runs
proc.wait()
Code language: Python (python)

This code uses subprocess.Popen() to run the “ping” command asynchronously and continues with other tasks while it’s running.

7. Redirecting Input and Output to Files:

import subprocess

with open("output.txt", "w") as output_file:
    subprocess.run(["ls", "-l"], stdout=output_file)
Code language: Python (python)

This code redirects the standard output of the “ls -l” command to an “output.txt” file.

8. Handling Standard Error:

import subprocess

try:
    subprocess.run(["ls", "/nonexistent"], check=True, stderr=subprocess.PIPE, text=True)
except subprocess.CalledProcessError as e:
    print(f"Command failed with return code {e.returncode}")
    print(f"Error output: {e.stderr}")
Code language: Python (python)

This example captures both standard output and standard error and prints the error message when the “ls” command fails.

These examples demonstrate various use cases of the subprocess module in Python for running external commands, managing processes, and handling input and output streams. Depending on your specific task, you can adapt these examples to suit your needs.

Read More;

• Python Profiling kcachegrind
• Python cProfile Label
• Python Profile GUI
• Python Profiling in Pycharm With Example
• How To Round Up In Python?
• How To Convert List To String In Python
• Python Error: “AttributeError: __enter__”
• subprocess-exited-with-error in Python
• Python Volume Profile With Example
• Python Profile Subprocess 
• subprocess.Popen to multiprocessing 
• Python Profile Plot