Read & Write Text Files in Python

Working with text files is one of the most common tasks you'll encounter as a Python developer. Whether you're processing data, saving user input, or reading configuration files, knowing how to handle text files efficiently is an essential skill. Today, we'll explore everything you need to know about reading from and writing to text files using Python.

Opening Files

The foundation of file handling in Python is the built-in open() function. This function creates a file object that you can use to interact with the file's contents.

The basic syntax is straightforward:

file = open('filename.txt', 'mode')

The mode parameter specifies how you want to interact with the file. Here are the most common modes: - 'r' - Read mode (default) - 'w' - Write mode (overwrites existing file) - 'a' - Append mode (adds to existing file) - 'x' - Exclusive creation (fails if file exists) - 't' - Text mode (default) - 'b' - Binary mode

You can combine modes like 'wt' for writing text or 'rb' for reading binary.

Important: Always close your files using the close() method when you're done working with them to free up system resources. Here's a basic example:

# Open a file for writing
file = open('example.txt', 'w')
file.write('Hello, World!')
file.close()

Reading Files

Python offers several methods for reading file contents, each suited for different scenarios.

The simplest approach is using read(), which reads the entire file content as a single string:

file = open('example.txt', 'r')
content = file.read()
print(content)
file.close()

For larger files, you might want to read line by line using readline():

file = open('example.txt', 'r')
first_line = file.readline()
second_line = file.readline()
file.close()

To read all lines into a list, use readlines():

file = open('example.txt', 'r')
lines = file.readlines()
file.close()

You can also iterate through the file object directly, which is memory-efficient for large files:

file = open('example.txt', 'r')
for line in file:
    print(line.strip())  # strip() removes newline characters
file.close()

Writing Files

Writing to files is just as straightforward as reading. The main methods are write() and writelines().

The write() method writes a string to the file:

file = open('output.txt', 'w')
file.write('This is line 1\n')
file.write('This is line 2\n')
file.close()

The writelines() method writes a list of strings to the file:

lines = ['First line\n', 'Second line\n', 'Third line\n']
file = open('output.txt', 'w')
file.writelines(lines)
file.close()

Remember that write modes overwrite existing files. If you want to add content without deleting what's already there, use append mode ('a'):

file = open('existing_file.txt', 'a')
file.write('This appends to the end\n')
file.close()

The With Statement

While manually closing files works, Python provides a more elegant solution: the with statement. This context manager automatically closes the file for you, even if an error occurs.

Here's how it works:

with open('example.txt', 'r') as file:
    content = file.read()
    # File automatically closed after this block

You can use the with statement for both reading and writing:

# Writing with context manager
with open('output.txt', 'w') as file:
    file.write('Hello from context manager!\n')

# Reading with context manager
with open('output.txt', 'r') as file:
    content = file.read()
    print(content)

This approach is considered best practice because it ensures files are properly closed and reduces the risk of resource leaks.

Handling File Paths

When working with files, you'll often need to handle different path formats. Python's os.path module provides useful functions for this:

import os

# Join paths safely
full_path = os.path.join('folder', 'subfolder', 'file.txt')

# Check if file exists
if os.path.exists('myfile.txt'):
    print('File exists!')

# Get absolute path
abs_path = os.path.abspath('relative_file.txt')

For more advanced path manipulation, consider using the pathlib module, which offers an object-oriented approach:

from pathlib import Path

# Create a Path object
file_path = Path('data') / 'input.txt'

# Check existence
if file_path.exists():
    print(f'File size: {file_path.stat().st_size} bytes')

# Read content
content = file_path.read_text()

Error Handling

File operations can fail for various reasons: files might not exist, you might lack permissions, or the disk could be full. Proper error handling makes your code more robust.

Use try-except blocks to handle potential errors:

try:
    with open('nonexistent.txt', 'r') as file:
        content = file.read()
except FileNotFoundError:
    print("The file doesn't exist!")
except PermissionError:
    print("You don't have permission to read this file!")
except Exception as e:
    print(f"An unexpected error occurred: {e}")

You can also check if a file exists before attempting to open it:

import os

filename = 'data.txt'
if os.path.exists(filename):
    with open(filename, 'r') as file:
        content = file.read()
else:
    print(f"{filename} does not exist")

Common file-related exceptions include: - FileNotFoundError - PermissionError - IsADirectoryError - IOError

Working with Different Encodings

Text files can use different character encodings, which specify how characters are represented as bytes. The most common encoding is UTF-8, but you might encounter others like ASCII, Latin-1, or Windows-1252.

You can specify the encoding when opening a file:

# Read a file with specific encoding
with open('file.txt', 'r', encoding='utf-8') as file:
    content = file.read()

# Write with specific encoding
with open('output.txt', 'w', encoding='latin-1') as file:
    file.write('Some text')

If you encounter encoding errors, you can handle them with the errors parameter:

# Replace problematic characters instead of raising error
with open('file.txt', 'r', encoding='utf-8', errors='replace') as file:
    content = file.read()

# Ignore problematic characters
with open('file.txt', 'r', encoding='utf-8', errors='ignore') as file:
    content = file.read()

Practical Examples

Let's look at some real-world scenarios where file handling is essential.

Processing a CSV-like file:

with open('data.csv', 'r') as file:
    for line in file:
        # Split by comma and strip whitespace
        values = [value.strip() for value in line.split(',')]
        print(values)

Creating a configuration file:

config = {
    'theme': 'dark',
    'language': 'en',
    'notifications': True
}

with open('config.txt', 'w') as file:
    for key, value in config.items():
        file.write(f'{key}={value}\n')

Reading that configuration file later:

config = {}
with open('config.txt', 'r') as file:
    for line in file:
        if '=' in line:
            key, value = line.strip().split('=', 1)
            # Convert string 'True' to boolean True
            if value.lower() == 'true':
                value = True
            elif value.lower() == 'false':
                value = False
            config[key] = value

Logging application events:

import datetime

def log_message(message):
    timestamp = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
    with open('app.log', 'a') as log_file:
        log_file.write(f'{timestamp} - {message}\n')

# Usage
log_message('Application started')
log_message('User logged in')

File Operations Beyond Reading/Writing

Sometimes you need to perform other file operations like renaming, deleting, or checking file properties.

import os
import shutil

# Rename a file
os.rename('old_name.txt', 'new_name.txt')

# Delete a file
os.remove('file_to_delete.txt')

# Copy a file
shutil.copy('source.txt', 'destination.txt')

# Get file size
size = os.path.getsize('myfile.txt')
print(f'File size: {size} bytes')

# Get modification time
mtime = os.path.getmtime('myfile.txt')
print(f'Last modified: {mtime}')

Best Practices

Following these guidelines will make your file handling code more reliable and maintainable:

• Always use the with statement for automatic resource cleanup
• Specify encoding explicitly rather than relying on system defaults
• Handle exceptions appropriately for robust error handling
• Use absolute paths when possible to avoid confusion
• Close files promptly when not using context managers
• Test with different file sizes to ensure performance

Remember that file operations can be slow compared to in-memory operations, so minimize unnecessary disk access in performance-critical applications.

By mastering these text file handling techniques, you'll be well-equipped to tackle a wide range of programming tasks that involve persistent data storage. The concepts we've covered today form the foundation for working with many types of data files, from simple text documents to complex data formats.