Making HTTP Requests with `http.Client`

1. The Foundation: A Robust HTTP Client

Go's net/http package provides everything you need to work with HTTP. A central type is the http.Client, which is responsible for sending requests and managing state like cookies and persistent connections.

While you can make simple requests with package-level functions like http.Get(), it is a strong best practice to create your own http.Client instance. Why? The default client used by these helper functions has no timeout. This means a request to an unresponsive server could cause your program to hang indefinitely, which is a significant risk for any automation script.

Creating a custom client is straightforward and allows you to configure crucial behavior, most importantly, a request timeout.

The key takeaway is this piece of code:

client := &http.Client{
    Timeout: 10 * time.Second,
}

This simple configuration ensures that no single HTTP request will block your application for more than 10 seconds.

2. Fetching Data with GET Requests

A GET request is used to retrieve data from a server. This is what your browser does when you visit a webpage, and it's what your code will do to fetch the status of a service or pull configuration from an API endpoint.

The process involves a few clear steps:

1. Create the Request: Construct an http.Request object specifying the method ("GET"), the URL, and no body. Using http.NewRequest gives you more flexibility than the simpler http.Get helper.
2. Execute the Request: Use your custom client's Do() method to send the request.
3. Check for Errors: The Do() method returns a response and an error. Always check the error first (e.g., for network issues).
4. Close the Response Body: The response body is an io.ReadCloser that must be closed to release the underlying network connection. The defer statement is perfect for this: defer resp.Body.Close().
5. Check the Status Code: A successful network round-trip doesn't mean the request was successful. You must check the HTTP status code (e.g., resp.StatusCode != http.StatusOK).
6. Process the Body: If the status is OK, you can read the response body, for example, by decoding it from JSON into a Go struct.

Here's a summary of the code structure for a GET request:

package main

import (
	"fmt"
	"io"
	"net/http"
	"os"
	"time"
)

func main() {
	// 1. Create a custom client with a timeout
	client := &http.Client{
		Timeout: 10 * time.Second,
	}

	// 2. Create the request
	req, err := http.NewRequest("GET", "https://httpbin.org/get", nil)
	if err != nil {
		fmt.Printf("Error creating request: %s\n", err)
		os.Exit(1)
	}

	// 3. Execute the request
	resp, err := client.Do(req)
	if err != nil {
		fmt.Printf("Error making http request: %s\n", err)
		os.Exit(1)
	}
	// 4. IMPORTANT: Defer closing the body to prevent resource leaks
	defer resp.Body.Close()

	// 5. Check the status code
	if resp.StatusCode != http.StatusOK {
		fmt.Printf("Error: received status code %d\n", resp.StatusCode)
		os.Exit(1)
	}
	
	fmt.Println("Client: got response!")
	fmt.Printf("Client: status code: %d\n", resp.StatusCode)

	// 6. Read and print the body
	bodyBytes, err := io.ReadAll(resp.Body)
	if err != nil {
		fmt.Printf("Error reading response body: %s\n", err)
		os.Exit(1)
	}
	
	fmt.Printf("Client: response body: %s\n", string(bodyBytes))
}

3. Sending Data with POST Requests

A POST request is used to send data to a server, typically to create or update a resource. For example, you might POST a JSON payload to a GitLab API endpoint to trigger a new CI/CD pipeline.

The process is very similar to a GET request, with two key additions:

1. Provide a Request Body: The data you send must be an io.Reader. For JSON, this usually means marshaling a Go struct into a []byte slice and then wrapping it with a bytes.NewReader or bytes.NewBuffer.
2. Set the Content-Type Header: You must tell the server what kind of data you're sending. For JSON, this header's value is "application/json".

Combining these concepts, the structure for a POST request with a JSON body looks like this:

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"io"
	"net/http"
	"os"
	"time"
)

// A struct to model our JSON data
type JobPayload struct {
	Name   string `json:"name"`
	Branch string `json:"branch"`
}

func main() {
	client := &http.Client{
		Timeout: 10 * time.Second,
	}

	// 1. Create the data and marshal it to JSON
	payload := JobPayload{Name: "deploy-production", Branch: "main"}
	payloadBytes, err := json.Marshal(payload)
	if err != nil {
		// handle error
	}
	// Create an io.Reader from the JSON byte slice
	bodyReader := bytes.NewReader(payloadBytes)

	// 2. Create the request, now with a method and body
	req, err := http.NewRequest("POST", "https://httpbin.org/post", bodyReader)
	if err != nil {
		// handle error
	}

	// 3. Set the crucial Content-Type header
	req.Header.Set("Content-Type", "application/json")

	// Execute the request and process the response (same as GET)
	resp, err := client.Do(req)
	if err != nil {
		// handle error
	}
	defer resp.Body.Close()

	fmt.Printf("Status: %d\n", resp.StatusCode)
	
	respBody, _ := io.ReadAll(resp.Body)
	fmt.Printf("Response Body: %s\n", string(respBody))
}

{
  "service": "api-gateway",
  "status": "success"
}

package main

import (
    "bytes"
    "encoding/json"
    "net/http"
    "time"
)

type StatusReport struct {
    Service string `json:"service"`
    Status  string `json:"status"`
}

// reportStatus sends a deployment status to a monitoring service.
func reportStatus(serviceName, status string) (*http.Response, error) {
    client := &http.Client{
        Timeout: 5 * time.Second,
    }

    // --- YOUR CODE GOES HERE ---
    // 1. Create a StatusReport instance with the provided arguments.
    // 2. Marshal it into JSON bytes.
    // 3. Create an http.Request with the "POST" method, URL, and JSON body.
    // 4. Set the "Content-Type" header to "application/json".
    // 5. Execute the request with client.Do() and return its response and error.
    // --- END OF YOUR CODE ---
}

// reportStatus sends a deployment status to a monitoring service.
func reportStatus(serviceName, status string) (*http.Response, error) {
	client := &http.Client{
		Timeout: 5 * time.Second,
	}

	// 1. Create a StatusReport instance.
	report := StatusReport{
		Service: serviceName,
		Status:  status,
	}

	// 2. Marshal it into JSON bytes.
	jsonData, err := json.Marshal(report)
	if err != nil {
		return nil, err
	}

	// 3. Create an http.Request with the "POST" method, URL, and JSON body.
	req, err := http.NewRequest("POST", "https://example.com/api/report", bytes.NewBuffer(jsonData))
	if err != nil {
		return nil, err
	}

	// 4. Set the "Content-Type" header.
	req.Header.Set("Content-Type", "application/json")

	// 5. Execute the request and return the result.
	return client.Do(req)
}

Conclusion

In this lesson, you've learned the essential mechanics of interacting with HTTP services in Go. This skill is a cornerstone for building DevOps automation, allowing your tools to communicate with the vast ecosystem of web-based APIs.

Key Takeaways:

• Always create a custom http.Client to set a Timeout and avoid hanging requests.
• Use http.NewRequest and a client's Do() method to have full control over the request.
• For GET requests, the process is: create request, execute, check errors and status, defer body closure, and read the body.
• For POST requests, you must also provide an io.Reader as the request body and set the Content-Type header appropriately.
• Always remember defer resp.Body.Close() to prevent resource leaks.

Next Up

We've seen how to get a response body, but often that body is complex JSON. In our next lesson, we will dive deeper into parsing JSON data from API responses into Go structs. You'll learn how to handle nested objects, arrays, and fields with different names, enabling you to work with real-world API data effectively.