Welcome to the next lesson in our backend engineering course. In our previous sessions, we focused on how to retrieve data successfully and efficiently, culminating in our discussion on scalable cursor-based pagination. Now, we turn our attention to an equally critical aspect of API design: what happens when things go wrong?
An API's quality is often judged not by its "happy path" but by how it behaves under failure. In backend interviews, being able to design robust and predictable error handling is a sign of a mature engineer. Today, you will learn to design a consistent API error response format with structured error codes and messages. We will move beyond simple error strings to a standardized, machine-readable format that makes your API predictable, secure, and easy for clients to integrate with.
The Chaos of Inconsistent Errors
First, let's establish why a simple error message is not enough. Imagine you're building a client application that consumes three different API endpoints.
- Endpoint A fails and returns:
400 Bad Requestwith the body{"message": "User validation failed"}. - Endpoint B fails and returns:
400 Bad Requestwith the body{"error_details": "Invalid input for field 'email'"}. - Endpoint C fails and returns:
200 OKwith the body{"success": false, "reason": "Item out of stock"}.
This inconsistency creates a nightmare for the client developer. They cannot write a single piece of logic to handle errors; instead, they have to parse different fields (message, error_details, reason) and even check the response body for a success flag on 200 OK responses, which is a major anti-pattern.
The video below from the CodeOpinion channel does an excellent job of illustrating this exact problem.
BEYOND Status Codes! Better REST HTTP API Error Responses
This video starts by showing two common but inconsistent API error responses, highlighting why a standardized approach is needed.
Please watch the first part of the video, from the beginning until the introduction to the problem. Notice how the two example error responses have completely different structures, making them difficult to handle programmatically.
As you can see, this inconsistency makes it impossible for client applications to reliably handle errors, display meaningful messages to users, or implement automated logic like retries.
A Layered Approach to Error Design
A well-designed error response provides information in layers, from general to specific.
Layer 1: HTTP Status Codes
The foundation of any REST API error response is the correct use of HTTP status codes. These codes provide the first, high-level signal about the outcome of a request. As a backend developer, you must use them correctly.
- 4xx Client Errors: The request itself is flawed (e.g., bad syntax, missing authentication, requesting a non-existent resource). The client is expected to fix the request before trying again.
- 5xx Server Errors: The server failed to fulfill a valid request due to an internal problem (e.g., a database connection failure, an unhandled exception). The client can often retry the same request later.
The article "How to Design Error Responses in REST APIs" provides a solid reference for common status codes.
How to Design Error Responses in REST APIs - OneUptime
This article from OneUptime gives a practical overview of error response design, starting with the fundamentals.
Focus on the section HTTP Status Codes. Review the tables for Client Errors (4xx) and Server Errors (5xx), paying special attention to the difference between codes like 400, 401, 403, and 422. Also, note the "Common Mistakes" section, which highlights the anti-pattern of using 200 OK for application errors.
Using the correct status code is non-negotiable. It allows network components like caches, load balancers, and monitoring tools to understand the request outcome without inspecting the response body.
Layer 2: A Standardized Error Body (RFC 7807)
While status codes are essential, they are too general. A 400 Bad Request doesn't tell the client what was bad about the request. To solve this, the industry has converged on a standard called Problem Details for HTTP APIs (RFC 7807), recently updated as RFC 9457. This standard defines a consistent, machine-readable JSON object for communicating error details.
A Problem Details object has a few key fields:
type: A URI that uniquely identifies the type of problem. This is the most important field for programmatic handling. It acts as a stable, machine-readable error code.title: A short, human-readable summary of the problem type.status: The HTTP status code repeated in the body for convenience.detail: A human-readable explanation specific to this occurrence of the problem.instance: A URI that identifies the specific occurrence of the problem (e.g., the API endpoint that was called).
Here is an example of a well-formed error response following this standard.
The following video segment provides a clear walkthrough of these fields and their purpose.
BEYOND Status Codes! Better REST HTTP API Error Responses
The presenter explains the motivation behind Problem Details (RFC 7807) and breaks down each of its standard fields.
Please watch from the introduction of Problem Details. Pay close attention to the explanation of the type field as a stable, machine-readable identifier and the concept of "extensions" for adding custom data.
By adopting this standard, you provide a predictable structure that all API clients can rely on, vastly simplifying error handling.
Layer 3: Fine-Grained Application Codes
While the type URI in RFC 7807 is powerful, some APIs also provide a simpler string-based application code. These codes offer a fine-grained, human-friendly way to categorize errors within your application's domain. A great example of this is Stripe, which is famous for its excellent developer experience.
These codes, like account_closed or invalid_card_number, can be added as an extension to the Problem Details object.
{
"type": "https://api.example.com/errors/card-declined",
"title": "Card Declined",
"status": 422,
"detail": "Your card was declined by the payment processor.",
"instance": "/v1/charges/ch_12345",
"code": "card_declined" // Application-specific code
}
The article from OneUptime gives some excellent principles for designing these codes.
How to Design Error Responses in REST APIs - OneUptime
This section of the article differentiates HTTP status codes from fine-grained application error codes and provides design principles for them.
Read the section Error Codes vs Status Codes. Focus on the "Design Principles" and the example taxonomy, which shows how to group codes by domain (e.g., auth., payment.).
Implementation Patterns and Best Practices
Knowing the format is one thing; implementing it robustly is another. This is where your skills in Go and backend architecture come into play.
Custom Exception Hierarchies
The best way to manage errors in your application code is to create a custom exception/error hierarchy. In Go, you would define custom error types that implement the error interface. In Python, you'd create custom exception classes inheriting from Exception.
The pattern is to have a base APIError and then more specific types that derive from it:
APIError(base type)NotFoundErrorValidationErrorAuthenticationErrorPermissionDeniedError
This allows your service logic to throw specific, meaningful errors. The ArjanCodes video, while using Python and FastAPI, demonstrates this universal concept perfectly.
This Is How You Do PROPER Exception Handling With FastAPI
This video shows how to create a custom exception hierarchy and then use a centralized handler to convert those exceptions into structured JSON responses. The principles are directly applicable to a Go backend using middleware.
Watch the segments on creating a custom exception hierarchy and then integrating it with FastAPI using a custom handler. The key idea is to catch your specific error types and map them to the correct HTTP status code and response body.
Centralized Error Handling Middleware
You would then create a single, centralized error-handling middleware. This middleware sits in your request-response chain, catches any of the custom APIError types thrown by your application logic, and formats them into the standard RFC 7807 JSON response before sending it to the client. This keeps your business logic clean (just return err) and ensures all API error responses are consistent.
Handling Validation Errors
A very common interview topic is handling validation errors. A bad API returns the first validation error it finds. A good API returns all validation errors at once. This is typically done by adding an errors array as an extension to the Problem Details object.
How to Design Error Responses in REST APIs - OneUptime
This section focuses on the best practices for handling validation errors, a very common scenario.
Read the section Validation Error Responses. Pay close attention to the "Best Practices" list and the example response, which includes an errors array detailing each specific field error.
Security: Logging vs. Exposing
Finally, a crucial point for production systems and interviews is security. Never expose internal implementation details in an error response. This includes:
- Stack traces
- Database error messages or query details
- Internal file paths or server names
- Sensitive configuration values
The correct approach is to log the full, detailed error internally for debugging and return a sanitized, user-friendly error to the client. The bridge between these two is a trace_id.
How to Design Error Responses in REST APIs - OneUptime
This final reading covers the critical distinction between what you log and what you expose, and the security implications of error responses.
Please read the sections Logging vs Exposing Errors and Security Considerations. The key takeaway is the role of the trace_id in connecting a public-facing error to detailed internal logs without leaking sensitive information.
Conclusion
Designing error responses is not an afterthought; it's a core feature of a well-built API. By mastering this, you demonstrate an ability to think from the consumer's perspective and build robust, secure, and professional backend services.
Key Takeaways:
- Start with Status Codes: Always use the correct 4xx/5xx HTTP status code.
- Adopt a Standard: Use the RFC 7807 Problem Details format for a consistent, machine-readable error body.
- Be Specific: Use the
typeURI for programmatic handling and optionally add a fine-grainedcodefor application-specific errors. - Be Helpful: Provide clear
titleanddetailmessages. For validation, return all errors at once. - Be Secure: Never leak internal details. Log detailed errors internally and provide a
trace_idin the public response to correlate them. - Centralize Logic: Use a custom error hierarchy in your code and a centralized middleware to handle formatting and response generation.
In our journey through API design, we have now covered how to return both successful data collections and structured errors. In the next lesson, we will move beyond the traditional request-response model and explore patterns for real-time communication by comparing WebSockets, Server-Sent Events, and long polling.