Building Reusable API Clients for DevOps Platforms

1. The Foundation: The Client Struct and Constructor

The first step in creating a reusable package is to move the code out of main and into its own directory. Let's imagine we're creating a github package.

The core of any API client is a struct that holds essential, long-lived information: the API's base URL, an HTTP client to make requests, and any necessary credentials like an API token.

As the video shows, the basic structure looks like this:

// Inside a new directory, e.g., "github/client.go"
package github

import "net/http"

type Client struct {
    BaseURL    string
    apiKey     string // Unexported, as it should be handled internally
    HTTPClient *http.Client
}

func NewClient(apiKey string) *Client {
    return &Client{
        BaseURL:    "https://api.github.com",
        apiKey:     apiKey,
        HTTPClient: &http.Client{},
    }
}

This simple pattern encapsulates the client's dependencies. Any function that needs to make an API call will be a method on a pointer to this Client struct (*Client), giving it access to the base URL and the HTTP client.

2. A Flexible API: The Functional Options Pattern

Our NewClient function is a good start, but it's rigid. What if a user wants to:

• Point the client to a GitHub Enterprise instance with a different URL?
• Provide a custom http.Client (like the retryablehttp client from our last lesson)?
• Set a custom user agent?

We could add more arguments to NewClient, but this quickly becomes unwieldy. A more idiomatic and scalable solution in Go is the functional options pattern. This pattern allows you to pass a variable number of configuration functions to the constructor.

Let's apply this pattern to our GitHub client.

// In github/client.go

package github

import (
    "net/http"
    "time"
)

// Option is a function that configures a Client.
type Option func(*Client)

// WithBaseURL allows overriding the default API base URL.
func WithBaseURL(baseURL string) Option {
    return func(c *Client) {
        c.BaseURL = baseURL
    }
}

// WithHTTPClient allows providing a custom HTTP client.
func WithHTTPClient(httpClient *http.Client) Option {
    return func(c *Client) {
        c.HTTPClient = httpClient
    }
}

type Client struct {
    BaseURL    string
    apiKey     string
    HTTPClient *http.Client
}

// NewClient creates a new GitHub API client.
func NewClient(apiKey string, opts ...Option) *Client {
    // Default client
    client := &Client{
        BaseURL: "https://api.github.com",
        apiKey:  apiKey,
        HTTPClient: &http.Client{
            Timeout: 10 * time.Second,
        },
    }

    // Apply all provided options
    for _, opt := range opts {
        opt(client)
    }

    return client
}

Now, a user can create a client like this:

// Simple client with defaults
client1 := github.NewClient("my-api-key")

// Client for GitHub Enterprise
client2 := github.NewClient("my-api-key", github.WithBaseURL("https://github.my-company.com/api/v3"))

// Client using our retryable client from the last lesson
retryClient := retryablehttp.NewClient()
// ... configure retryClient ...
stdClient := retryClient.StandardClient()
client3 := github.NewClient("my-api-key", github.WithHTTPClient(stdClient))

This approach is clean, extensible, and very common in professional Go libraries.

3. Organizing Endpoints with Services

A large API like GitHub's has hundreds of endpoints. Putting all the corresponding methods directly on the Client struct would create a massive, disorganized file. A better approach is to group related endpoints into services. For example, all user-related endpoints go into a UsersService, repository endpoints into a RepositoriesService, and so on.

Each service is a struct that holds a pointer back to the main client, using it to make the actual HTTP calls.

Here is how we would apply this to our Client:

// In github/client.go

type Client struct {
    BaseURL    string
    apiKey     string
    HTTPClient *http.Client

    // Services for different API resources
    Users *UsersService
}

func NewClient(apiKey string, opts ...Option) *Client {
    // ... (previous NewClient code) ...

    // Wire up the services
    client.Users = &UsersService{client: client}

    return client
}

// In a new file, github/users.go
package github

import (
    "context"
    "fmt"
)

// UsersService handles communication with the user related
// methods of the GitHub API.
type UsersService struct {
    client *Client
}

type User struct {
    Login     string `json:"login"`
    Name      string `json:"name"`
    PublicRepos int    `json:"public_repos"`
}

// Get fetches a user. Passing an empty string for username will fetch the
// authenticated user.
func (s *UsersService) Get(ctx context.Context, username string) (*User, error) {
    var path string
    if username != "" {
        path = fmt.Sprintf("/users/%s", username)
    } else {
        path = "/user"
    }
    // We will implement sendRequest shortly
    // req, err := s.client.newRequest(http.MethodGet, path, nil)
    // ...
    return nil, nil // Placeholder
}

The user of our package can now access user-related methods cleanly: client.Users.Get(...).

4. Abstracting Boilerplate with a Helper Function

Every API call involves similar steps: create a request, set headers (like Authorization and Content-Type), send it, check the status code, and decode the response. Repeating this logic in every service method is tedious. We can abstract this into a private helper method on the main Client.

Let's add a similar helper, do, to our Client. This method will become the workhorse of our entire library.

// In github/client.go

func (c *Client) do(ctx context.Context, method, path string, body, result interface{}) (*http.Response, error) {
    url := c.BaseURL + path
    
    var reqBody io.Reader
    if body != nil {
        b, err := json.Marshal(body)
        if err != nil {
            return nil, fmt.Errorf("failed to marshal request body: %w", err)
        }
        reqBody = bytes.NewBuffer(b)
    }

    req, err := http.NewRequestWithContext(ctx, method, url, reqBody)
    if err != nil {
        return nil, fmt.Errorf("failed to create request: %w", err)
    }

    req.Header.Set("Authorization", "Bearer "+c.apiKey)
    req.Header.Set("Accept", "application/vnd.github.v3+json")
    if body != nil {
        req.Header.Set("Content-Type", "application/json")
    }
    
    resp, err := c.HTTPClient.Do(req)
    if err != nil {
        return nil, fmt.Errorf("failed to send request: %w", err)
    }
    defer resp.Body.Close()

    if resp.StatusCode < 200 || resp.StatusCode >= 300 {
        // You would typically decode a specific GitHub error response struct here
        return resp, fmt.Errorf("request failed with status: %s", resp.Status)
    }

    if result != nil {
        if err := json.NewDecoder(resp.Body).Decode(result); err != nil {
            return resp, fmt.Errorf("failed to decode response: %w", err)
        }
    }

    return resp, nil
}

Now, our UsersService.Get method becomes much simpler:

// In github/users.go
func (s *UsersService) Get(ctx context.Context, username string) (*User, error) {
    path := "/user"
    if username != "" {
        path = fmt.Sprintf("/users/%s", username)
    }

    var user User
    _, err := s.client.do(ctx, http.MethodGet, path, nil, &user)
    if err != nil {
        return nil, err
    }

    return &user, nil
}

5. Designing for Testability

A reusable library must be testable. We cannot have our unit tests making real network calls to GitHub. The key to testability is to depend on interfaces, not concrete types.

Notice that our Client holds an *http.Client. To test our logic, we need to be able to replace this with a mock that simulates server responses. The video from the beginning of the lesson explains this perfectly.

The core idea is to define a small interface for just the part of http.Client we use: the Do method.

// In github/client.go

// httpClient is an interface for a client that can make HTTP requests.
// It's satisfied by *http.Client.
type httpClient interface {
    Do(req *http.Request) (*http.Response, error)
}

type Client struct {
    // ...
    HTTPClient httpClient // Use the interface type
}

By using this interface, our tests can provide a fake httpClient that returns a canned response, allowing us to test our JSON decoding, error handling, and business logic without ever hitting the network.

Conclusion

Congratulations! You've learned the fundamental patterns for building a professional, reusable API client in Go. This is a significant step up from writing simple scripts and is a core skill for creating robust DevOps tooling.

Key Takeaways:

• Package Structure: Isolate your client in its own package (e.g., github/).
• Client Struct: Use a central Client struct to hold the base URL, HTTP client, and API token.
• Functional Options: Use the Option function pattern for a flexible and extensible NewClient constructor.
• Service-Oriented Design: Group related API endpoints into services (e.g., UsersService, RepositoriesService) that delegate to the main client.
• DRY with Helpers: Abstract common request/response logic into a private helper method (e.g., do or sendRequest).
• Design for Testability: Depend on interfaces (like a httpClient interface) instead of concrete types to enable easy mocking in tests.

Next Up

Now that we have a reusable client package, the next logical step is to build a tool that uses it. In our next lesson, we will begin building a command-line interface (CLI) and learn how to parse command-line flags and arguments using Go's built-in flag package. This will be the user-facing part of the automation tools you aim to build.