API WB Ozon: handling 429 and 50x errors

Handling 429 and 50x Errors in Wildberries and Ozon APIs

When scraping data from Wildberries and Ozon APIs, developers often run into 429 (too many requests) and 50x (server errors). These crop up due to heavy load on endpoints. Tech support's go-to fix is usually retries. We'll cover three solid strategies: basic retry handling, optimizing request volumes, and monitoring with partial reloads.

Basic Retry Handling

Start by treating 429 and 50x errors as expected hiccups. Build retry logic with exponential backoff and parse headers like X-Ratelimit-Retry. This cuts down on bans and boosts resilience.

Here's a sample function for POST requests with retries:

def get_oz(url, heads, params, max_retries=5, timeout=30):
    result = {}
    for retries in range(1, max_retries + 1):  
        
        try:
            response = requests.post(url, headers=heads, json=params, timeout=timeout)
        except requests.RequestException as e:
            print(f'!!! Network error on attempt {retries}: {e}!!!')
            time.sleep(2 * retries)
            continue  
                
        # 200 means success    
        if response.status_code == 200:
            return response.json()
            
        # 429 - Too many requests, smart pause and retry
        elif response.status_code == 429:    
            pause_time = 2 ** retries
            
            # X-Ratelimit-Retry - Wildberries trick, Ozon might add it someday
            if 'X-Ratelimit-Retry' in response.headers:
                pause_time = int(response.headers['X-Ratelimit-Retry']) + 5
            print(f'get_oz: ! -Too many requests- on attempt {retries}. Pausing {pause_time} sec')
            time.sleep(pause_time)
        
        # 5xx errors - pause and retry                    
        elif 500 <= response.status_code < 600:
            pause_time = 2 ** retries
            print(f'get_oz: ! -Error {response.status_code}- on attempt {retries}. Pausing {pause_time} sec')
            time.sleep(pause_time)
        
        # Any other error - bail out!                
        else:
            print(f'get_oz: !!! API data fetch failed! code: {response.status_code}')
            return None


    print(f'get_oz: !!! All {max_retries} attempts failed')
  
    return None

• 5 retries handle most typical loads.
• Handle pagination loops (offset/page) at a higher level.
• On total failure, back off for an hour or more.

This slots right into any ETL pipeline for APIs.

Optimizing Data Volume

Big requests (like a week's transactions) trigger errors on later pagination pages. Break them into small, independent batches: by day, category, or filters. Save each batch to your DB immediately — losing one doesn't wipe your progress.

Benefits:

• Fewer pages per request (1–3 vs. 20).
• Independent retries per batch.
• Partial saves on failures.

Example: Loading Ozon transactions by day:

date_from = today() - timedelta(days=7)
date_to = today() - timedelta(days=1)
current_date = date_from
delta = timedelta(days=1)

# Loop through days from date_from to date_to
while current_date <= date_to:  
    transactions_oneday = pd.DataFrame([])           
    url = "https://api-seller.ozon.ru/v3/finance/transaction/list"
    ipage = 1
    all_pages_data = []
    while True:
        data = {
            "filter": {
                "date": {
                    "from": str(current_date) + "T00:00:00.000Z",
                    "to": str(current_date) + "T23:59:59.999Z"
                }
            },
            "page": ipage,
            "page_size": 1000
            }
        dd = []
        apioz_response = get_oz(url, headers, data, 5)
        if len(apioz_response) > 0 :
            dd = apioz_response['result']['operations']
            all_pages_data.extend(dd)
        else:
            print(f'!!! Empty API response')    

        # If page has <1000 rows, it's the last - break    
        if len(dd) < 1000: break

        ipage +=1
            
    transactions_oneday = pd.DataFrame(all_pages_data)        
    # Export to DB right away what we got from API
    export_to_db('oz_transactions', transactions_oneday)

    # Next day
    current_date += delta    

Nested loops might look messy, but reliability skyrockets.

Monitoring and Partial Reloads

For full automation, add a safety net: background monitoring. After the main run (say, 6–7 AM), check logs/DB hourly (8–12 AM).

• Review last task status.
• Check data integrity (date gaps).
• Trigger targeted retries just for missing chunks.

This minimizes manual fixes, especially for daily updates.

Key Takeaways

• Use exponential backoff with retry headers for 429.
• Split large requests into daily/filter batches with instant saves.
• Add hourly monitoring post-collection for gap-filling.
• 5 retries cover 95% of failures; total flops get an hour+ delay.
• Integrate into ETL without tight pagination ties.

— Editorial Team