Python REST API Cheatsheet

Whether you're building a web service or integrating with one, understanding how to work with REST APIs in Python is a fundamental skill. This cheatsheet covers the essentials—from making simple requests to handling authentication and responses. Let’s dive in.

Making HTTP Requests

Python offers several ways to interact with REST APIs, but the most common method is using the requests library. It's simple, powerful, and widely adopted. If you haven't installed it yet, you can do so with pip:

pip install requests

Once installed, you can start making requests. Here’s a quick example of a GET request:

import requests

response = requests.get('https://jsonplaceholder.typicode.com/posts')
print(response.status_code)
print(response.json())

This code fetches a list of posts from a test API and prints both the HTTP status code and the JSON response. The status_code tells you if the request was successful (e.g., 200 for OK), while response.json() parses the response body as JSON.

You can also make other types of requests, like POST, PUT, and DELETE. Here’s how to send data with a POST request:

data = {'title': 'New Post', 'body': 'This is the content', 'userId': 1}
response = requests.post('https://jsonplaceholder.typicode.com/posts', json=data)
print(response.json())

Notice the use of the json parameter—it automatically sets the Content-Type header to application/json and serializes the dictionary for you.

When working with APIs, you’ll often need to include additional information in your requests, such as query parameters, headers, or authentication tokens. Let’s explore those next.

Handling Query Parameters and Headers

Many APIs require you to send query parameters to filter or paginate results. With requests, you can pass these as a dictionary using the params argument:

params = {'userId': 1}
response = requests.get('https://jsonplaceholder.typicode.com/posts', params=params)

This appends ?userId=1 to the URL. It’s a clean way to build dynamic URLs without manual string concatenation.

Headers are equally important. They allow you to send metadata about your request, such as content type, authentication tokens, or custom flags. Here’s how to include headers:

headers = {'Authorization': 'Bearer your_token_here'}
response = requests.get('https://api.example.com/data', headers=headers)

Common headers you might use include: - Authorization for tokens - Content-Type to specify data format - User-Agent to identify your client

Always check the API documentation to see which headers are required or recommended.

If you need to send form data instead of JSON, use the data parameter instead of json:

form_data = {'key': 'value'}
response = requests.post('https://httpbin.org/post', data=form_data)

This is useful for APIs that expect traditional form submissions.

Authentication Methods

Securing API endpoints is common, and you’ll encounter several authentication methods. The simplest is HTTP Basic Auth, which requires a username and password:

from requests.auth import HTTPBasicAuth

response = requests.get('https://api.example.com/protected', auth=HTTPBasicAuth('user', 'pass'))

You can also pass a tuple directly:

response = requests.get('https://api.example.com/protected', auth=('user', 'pass'))

For token-based authentication, you typically include the token in the Authorization header. Here’s an example using a Bearer token:

headers = {'Authorization': 'Bearer YOUR_ACCESS_TOKEN'}
response = requests.get('https://api.example.com/data', headers=headers)

Some APIs use API keys passed as query parameters or headers. For example:

params = {'api_key': 'YOUR_API_KEY'}
# or
headers = {'X-API-Key': 'YOUR_API_KEY'}

OAuth is more complex and often requires a dedicated library like requests-oauthlib, but for simple cases, you can manually handle the token.

Always keep your credentials secure—avoid hardcoding them in your scripts. Use environment variables or configuration files instead.

Handling Responses

After making a request, you need to handle the response appropriately. The response object contains all the information returned by the server.

First, check the status code to ensure the request was successful:

if response.status_code == 200:
    data = response.json()
else:
    print(f"Error: {response.status_code}")

For convenience, you can use response.raise_for_status(), which raises an exception for non-200 status codes:

response = requests.get('https://api.example.com/data')
response.raise_for_status()  # Raises an error for bad status codes
data = response.json()

The response body can be accessed in different formats: - response.text for plain text - response.json() for JSON data (raises error if not JSON) - response.content for binary data

If you’re dealing with large responses, you might want to stream the content:

response = requests.get('https://example.com/large-file', stream=True)
for chunk in response.iter_content(chunk_size=8192):
    process(chunk)

This avoids loading the entire response into memory at once.

Headers from the response can also be useful:

print(response.headers['Content-Type'])

They might include information like rate limits, content type, or caching directives.

Error Handling and Timeouts

Network requests can fail for many reasons—timeouts, connection errors, or invalid responses. It’s important to handle these gracefully.

Use a try-except block to catch exceptions:

try:
    response = requests.get('https://api.example.com/data', timeout=5)
    response.raise_for_status()
except requests.exceptions.Timeout:
    print("The request timed out.")
except requests.exceptions.RequestException as e:
    print(f"An error occurred: {e}")

Setting a timeout is crucial to prevent your application from hanging indefinitely. The value is in seconds.

You might also encounter HTTP errors like 404 Not Found or 500 Internal Server Error. raise_for_status() helps here, but you can also handle specific status codes:

if response.status_code == 404:
    print("Resource not found.")
elif response.status_code == 500:
    print("Server error.")

For retrying failed requests, you can use the requests library with a loop or a more advanced method like the tenacity library.

Always implement robust error handling to make your code resilient and user-friendly.

Working with JSON Data

Most modern APIs use JSON for data exchange. Python’s json module is built-in, but requests makes it even easier with response.json().

When sending data, you can pass a dictionary to the json parameter, and it will be serialized automatically:

data = {"name": "John", "age": 30}
response = requests.post('https://api.example.com/users', json=data)

If you need to work with JSON strings directly, you can use json.dumps() and json.loads():

import json

json_string = '{"name": "John", "age": 30}'
response = requests.post('https://api.example.com/users', data=json_string, headers={'Content-Type': 'application/json'})

But using the json parameter is simpler and less error-prone.

When receiving JSON responses, response.json() returns a Python dictionary or list, which you can then manipulate:

data = response.json()
print(data['name'])

For nested data, you can traverse the structure as you would with any dictionary.

If the JSON response is large or you want to process it iteratively, you can use response.iter_lines() or response.json() might still be efficient enough for most cases.

Session Objects for Efficiency

If you’re making multiple requests to the same API, using a Session object can improve performance by reusing the underlying TCP connection. It also persists cookies and headers across requests.

Here’s how to use it:

with requests.Session() as session:
    session.headers.update({'Authorization': 'Bearer token'})
    response1 = session.get('https://api.example.com/data1')
    response2 = session.get('https://api.example.com/data2')

Any headers you set on the session will be sent with every request, which is handy for authentication.

Sessions are also useful for maintaining state, such as cookies for logged-in users:

session = requests.Session()
session.post('https://api.example.com/login', json={'username': 'user', 'password': 'pass'})
response = session.get('https://api.example.com/profile')  # Now authenticated

Remember to close the session when you’re done, or use a context manager as shown above.

Sessions can significantly speed up your code when making many requests, so use them where appropriate.

Advanced Topics: Pagination and Rate Limiting

Many APIs paginate their results to avoid overwhelming the client or server. Pagination means the data is split across multiple pages, and you need to make several requests to fetch everything.

Common pagination methods include: - Page numbers (e.g., ?page=2) - Cursors or tokens (e.g., ?next_token=abc) - Link headers (with next and prev URLs)

Here’s an example using page numbers:

page = 1
all_data = []
while True:
    response = requests.get(f'https://api.example.com/items?page={page}')
    data = response.json()
    if not data:
        break
    all_data.extend(data)
    page += 1

For APIs that use Link headers, you can check response.headers['Link'] for the next URL.

Rate limiting is another common practice. APIs often restrict how many requests you can make in a given time period. The rate limit info is usually in the response headers:

remaining = response.headers.get('X-RateLimit-Remaining')
if remaining and int(remaining) == 0:
    print("Rate limit exceeded. Waiting...")
    time.sleep(60)

Always respect rate limits to avoid being blocked. Implement retry logic with backoff if necessary.

Testing and Mocking APIs

When writing tests for code that uses APIs, you don’t want to hit the real server every time. Instead, you can mock the responses using libraries like responses or unittest.mock.

Install responses with:

pip install responses

Here’s how to mock a GET request:

import responses
import requests

@responses.activate
def test_get_data():
    responses.add(
        responses.GET,
        'https://api.example.com/data',
        json={'key': 'value'},
        status=200
    )
    response = requests.get('https://api.example.com/data')
    assert response.json() == {'key': 'value'}

This allows you to simulate API responses without network calls.

You can also mock errors:

responses.add(
    responses.GET,
    'https://api.example.com/data',
    status=404
)

Testing with mocks makes your tests faster and more reliable, and it avoids dependencies on external services.

Useful Libraries and Tools

While requests is the go-to library for many, there are other tools you might find helpful:

• http.client: Standard library module for low-level HTTP requests.
• urllib.request: Another standard library option, but more verbose.
• aiohttp: For asynchronous requests, useful in async applications.
• httpx: A modern alternative with async support and HTTP/2.

For command-line testing, tools like curl or HTTPie are great. For exploring APIs interactively, Postman or Insomnia are popular GUI options.

Here’s a quick example using httpx for async requests:

import httpx
import asyncio

async def fetch_data():
    async with httpx.AsyncClient() as client:
        response = await client.get('https://api.example.com/data')
        return response.json()

data = asyncio.run(fetch_data())

Choose the tool that best fits your needs and project structure.

Summary of Best Practices

To wrap up, here are some key best practices to keep in mind when working with REST APIs in Python:

• Always use timeouts to prevent hanging requests.
• Handle errors and exceptions gracefully.
• Use sessions for multiple requests to the same host.
• Respect rate limits and implement retry logic if needed.
• Secure your authentication tokens and avoid hardcoding them.
• Test your code with mocked API responses.

By following these guidelines, you’ll write more robust, efficient, and maintainable code. Happy coding!