Python Scripting Reference

Python Scripting Reference

Welcome, fellow coder! Whether you're automating tasks, processing data, or building tools, Python scripting is your trusty companion. This guide covers essential concepts and practical examples to make your scripting journey smooth and efficient.

Working with Files and Directories

Let's start with one of the most common scripting tasks: file operations. Python's built-in functions make reading and writing files straightforward.

Here's how you can read a text file:

with open('data.txt', 'r') as file:
    content = file.read()
    print(content)

Writing to a file is just as simple:

with open('output.txt', 'w') as file:
    file.write('Hello, World!')

For handling directories, the os module is your best friend:

import os

# Create directory
os.makedirs('my_folder', exist_ok=True)

# List files
files = os.listdir('.')
print(files)

# Check if path exists
if os.path.exists('my_file.txt'):
    print('File exists!')

Command Line Arguments

Scripts often need to accept input from the command line. The sys module provides access to command line arguments through sys.argv.

import sys

print(f"Script name: {sys.argv[0]}")
print(f"Arguments: {sys.argv[1:]}")

For more sophisticated argument parsing, use the argparse module:

import argparse

parser = argparse.ArgumentParser(description='Process some integers.')
parser.add_argument('integers', metavar='N', type=int, nargs='+',
                    help='an integer for the accumulator')
parser.add_argument('--sum', dest='accumulate', action='store_const',
                    const=sum, default=max,
                    help='sum the integers (default: find the max)')

args = parser.parse_args()
print(args.accumulate(args.integers))

Error Handling

Robust scripts handle errors gracefully. Python's try-except blocks are essential for this.

try:
    with open('nonexistent.txt', 'r') as file:
        content = file.read()
except FileNotFoundError:
    print("File not found! Creating a new one.")
    with open('nonexistent.txt', 'w') as file:
        file.write('New content')
except PermissionError:
    print("Permission denied!")
except Exception as e:
    print(f"Unexpected error: {e}")

Working with Dates and Times

The datetime module is indispensable for time-related operations:

from datetime import datetime, timedelta

# Current time
now = datetime.now()
print(f"Current time: {now}")

# Formatting dates
formatted = now.strftime("%Y-%m-%d %H:%M:%S")
print(f"Formatted: {formatted}")

# Time arithmetic
tomorrow = now + timedelta(days=1)
print(f"Tomorrow: {tomorrow}")

Common File Operations

Operation Function Example
Read file open() with open('file.txt', 'r') as f:
Write file open() with open('file.txt', 'w') as f:
Check existence os.path.exists() os.path.exists('file.txt')
Delete file os.remove() os.remove('file.txt')
List directory os.listdir() os.listdir('.')

Regular Expressions

The re module provides powerful pattern matching capabilities:

import re

text = "Contact us at support@example.com or sales@company.org"

# Find all email addresses
emails = re.findall(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b', text)
print(f"Found emails: {emails}")

# Replace patterns
new_text = re.sub(r'@example\.com', '@company.com', text)
print(f"Updated text: {new_text}")

Working with JSON Data

JSON handling is incredibly simple with Python's built-in module:

import json

# Writing JSON to file
data = {
    "name": "John Doe",
    "age": 30,
    "city": "New York"
}

with open('data.json', 'w') as file:
    json.dump(data, file, indent=4)

# Reading JSON from file
with open('data.json', 'r') as file:
    loaded_data = json.load(file)
    print(loaded_data['name'])

Environment Variables

Accessing environment variables is crucial for configuration:

import os

# Get environment variable
database_url = os.getenv('DATABASE_URL', 'default_url')
print(f"Database URL: {database_url}")

# Set environment variable (for current process)
os.environ['CUSTOM_VAR'] = 'custom_value'

Subprocess Management

The subprocess module allows you to run system commands:

import subprocess

# Run a command and capture output
result = subprocess.run(['ls', '-l'], capture_output=True, text=True)
print(result.stdout)

# Check return code
if result.returncode == 0:
    print("Command executed successfully!")
else:
    print(f"Command failed: {result.stderr}")

Essential Python Modules for Scripting

  • os - Operating system interfaces
  • sys - System-specific parameters
  • argparse - Command-line argument parsing
  • json - JSON encoding and decoding
  • re - Regular expressions
  • datetime - Date and time operations
  • subprocess - Subprocess management
  • shutil - High-level file operations
  • pathlib - Object-oriented path handling

Path Handling with pathlib

The modern way to handle file paths:

from pathlib import Path

# Create Path object
current_dir = Path('.')
file_path = current_dir / 'data' / 'file.txt'

# Check if file exists
if file_path.exists():
    print(f"File size: {file_path.stat().st_size} bytes")

# Read file content
content = file_path.read_text()

Working with CSV Files

The csv module makes handling CSV files straightforward:

import csv

# Reading CSV
with open('data.csv', 'r') as file:
    reader = csv.DictReader(file)
    for row in reader:
        print(f"Name: {row['name']}, Age: {row['age']}")

# Writing CSV
with open('output.csv', 'w', newline='') as file:
    writer = csv.writer(file)
    writer.writerow(['Name', 'Age', 'City'])
    writer.writerow(['John Doe', '30', 'New York'])

Logging for Scripts

Proper logging helps debug and monitor your scripts:

import logging

# Configure logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s',
    filename='script.log'
)

logger = logging.getLogger(__name__)

# Log messages
logger.debug('Detailed information')
logger.info('General information')
logger.warning('Something might be wrong')
logger.error('An error occurred')
logger.critical('Critical error!')

File Compression

Python can handle various compression formats:

import gzip
import zipfile

# Gzip compression
with gzip.open('file.txt.gz', 'wt') as f:
    f.write('Compressed content')

# ZIP files
with zipfile.ZipFile('archive.zip', 'w') as zipf:
    zipf.write('file1.txt')
    zipf.write('file2.txt')

Common String Operations

String manipulation is fundamental in scripting:

text = "  Hello, World!  "

# Basic operations
trimmed = text.strip()
lowercase = text.lower()
replaced = text.replace('World', 'Python')

# Splitting and joining
words = text.split()
joined = '-'.join(words)

# Formatting
name = "Alice"
age = 25
message = f"{name} is {age} years old"

Performance Tips

When writing scripts, consider these performance aspects:

  • Use list comprehensions instead of loops when possible
  • Prefer generators for large datasets
  • Use local variables inside loops for better performance
  • Consider caching expensive operations
  • Use appropriate data structures for your needs

Template for a Complete Script

Here's a template that incorporates many best practices:

#!/usr/bin/env python3
"""
Script description goes here.
"""

import argparse
import logging
import sys
from pathlib import Path

def main():
    parser = argparse.ArgumentParser(description='Process some data.')
    parser.add_argument('input', help='Input file path')
    parser.add_argument('-o', '--output', help='Output file path')

    args = parser.parse_args()

    # Setup logging
    logging.basicConfig(level=logging.INFO)

    try:
        input_path = Path(args.input)
        if not input_path.exists():
            raise FileNotFoundError(f"Input file {args.input} not found")

        # Process data here
        logging.info(f"Processing {args.input}")

        if args.output:
            output_path = Path(args.output)
            # Write output
            logging.info(f"Output written to {args.output}")

    except Exception as e:
        logging.error(f"Error: {e}")
        return 1

    return 0

if __name__ == "__main__":
    sys.exit(main())

Remember, the best way to learn scripting is by doing. Start with small tasks and gradually tackle more complex problems. Practice regularly and don't hesitate to consult Python's excellent documentation when you need help. Happy scripting!