Deserializing Whiteboard Documents from Binary Format

The BinaryReader and the Perils of Input

Just as we created a BinaryWriter to abstract away the details of writing, we will now create a BinaryReader to manage reading from an std::istream. Its primary tool will be istream::read(char* data, std::streamsize count), the counterpart to ostream::write.

However, while writing is generally a safe operation (you control the data), reading is inherently dangerous. The file you are opening could be from anywhere. It might be corrupted, or worse, maliciously crafted to exploit your reading logic. Deserialization vulnerabilities are a serious class of software bugs.

Based on the article and best practices, our reader must adhere to these core principles:

• Never trust input: Treat every byte, especially values that represent sizes or counts, as untrusted and potentially hostile.
• Check before reading: Always verify that there are enough bytes remaining in the buffer or stream before attempting to read. This prevents reading past the end of the buffer, a common cause of crashes and vulnerabilities.
• Validate sizes and counts: When you read a value that dictates the size of a subsequent data block (like the number of points in a stroke), validate it against a reasonable upper bound. This prevents an attacker from specifying a huge size, causing your application to attempt an enormous memory allocation, leading to a Denial-of-Service (DoS) attack.
• Handle failure gracefully: If any check fails, the deserialization process must stop immediately, and the object being constructed should be left in a clean, default state.

Implementing a Safe BinaryReader

With these principles in mind, let's design our BinaryReader. It will wrap an istream and provide safe methods for reading primitive types.

A crucial part of safe reading is knowing the bounds of the data. For a file stream, we can determine the total file size. By tracking our current position, we can ensure we never read past the end.

#include <fstream>
#include <cstdint>
#include <type_traits>
#include <optional>

class BinaryReader {
public:
    explicit BinaryReader(std::istream& stream) : m_stream(stream) {
        // Determine the stream size to enable bounds checking
        m_stream.seekg(0, std::ios::end);
        m_stream_end = m_stream.tellg();
        m_stream.seekg(0, std::ios::beg);
    }

    // Safely reads a trivially copyable type.
    // Returns an empty optional if the read would go out of bounds or fails.
    template<typename T>
    std::optional<T> read() {
        static_assert(std::is_trivially_copyable_v<T>, "Value must be trivially copyable.");
        
        if (would_read_out_of_bounds(sizeof(T))) {
            return std::nullopt;
        }

        T value;
        m_stream.read(reinterpret_cast<char*>(&value), sizeof(T));
        if (!m_stream) {
            return std::nullopt; // Read operation failed
        }
        
        return value;
    }

    // Safely reads a block of raw bytes into a vector.
    std::optional<std::vector<char>> read_bytes(size_t size) {
        if (would_read_out_of_bounds(size)) {
            return std::nullopt;
        }

        std::vector<char> buffer(size);
        m_stream.read(buffer.data(), size);
        if (!m_stream) {
            return std::nullopt;
        }

        return buffer;
    }

    void seek(std::streampos pos) {
        m_stream.seekg(pos);
    }

    std::streampos tell() {
        return m_stream.tellg();
    }

private:
    bool would_read_out_of_bounds(size_t read_size) {
        return m_stream.tellg() < 0 || (m_stream.tellg() + read_size > m_stream_end);
    }

    std::istream& m_stream;
    std::streampos m_stream_end;
};

This BinaryReader class is our defensive tool. The read<T> method returns a std::optional<T>, which elegantly forces the calling code to handle the possibility of failure. This is much safer than returning a default value or relying on exception handling for simple parsing errors.

The template helper functions in the "Secure Programming Practices" article demonstrate a similar principle using raw pointers and size checks. Our class adapts this to the std::istream interface.

Reconstructing the Whiteboard State

Now we can use our BinaryReader to implement the top-level loadDocument function. The logic will mirror the saveDocument function but with added validation at every step.

// Returns a fully reconstructed Whiteboard object, or std::nullopt on failure.
std::optional<Whiteboard> loadDocument(const std::string& path) {
    std::ifstream file_stream(path, std::ios::in | std::ios::binary);
    if (!file_stream) {
        return std::nullopt; // Failed to open file
    }
    
    BinaryReader reader(file_stream);
    Whiteboard doc; // Start with a clean, default document

    // 1. Read and Validate File Header
    auto magic_bytes = reader.read_bytes(8);
    if (!magic_bytes || std::string(magic_bytes->begin(), magic_bytes->end()) != "WBDOC_V1") {
        return std::nullopt; // Invalid magic number
    }

    auto version = reader.read<uint32_t>();
    if (!version || *version != 1) {
        return std::nullopt; // Unsupported version
    }

    // 2. Read Chunks in a loop until EOF
    while (true) {
        auto chunk_id_opt = reader.read<uint32_t>();
        if (!chunk_id_opt) break; // Clean end-of-file

        auto chunk_len_opt = reader.read<uint32_t>();
        if (!chunk_len_opt) return std::nullopt; // Corrupt file, header cut short

        // Here we could add more validation on chunk_len, e.g., against a max size
        
        std::streampos chunk_end_pos = reader.tell() + static_cast<std::streampos>(*chunk_len_opt);

        switch (*chunk_id_opt) {
            case 'META':
                if (!read_meta_chunk(reader, doc.camera)) return std::nullopt;
                break;
            case 'PNTS':
                if (!read_paints_chunk(reader, doc.paint_palette)) return std::nullopt;
                break;
            case 'STRK':
                if (!read_strokes_chunk(reader, doc.strokes)) return std::nullopt;
                break;
            default:
                // Unknown chunk, skip it
                reader.seek(chunk_end_pos);
                break;
        }

        // Defensively seek to the end of the chunk to handle malformed chunk data
        reader.seek(chunk_end_pos);
    }

    return doc;
}

The logic for reading each chunk is a direct reversal of the writing process. Let's look at the implementation for the STRK chunk, which is the most complex.

// A helper to read the strokes and populate the vector. Returns false on failure.
bool read_strokes_chunk(BinaryReader& reader, std::vector<Stroke>& strokes) {
    auto num_strokes_opt = reader.read<uint32_t>();
    if (!num_strokes_opt) return false;

    // Validate against a reasonable limit to prevent DoS
    const uint32_t MAX_STROKES = 1000000;
    if (*num_strokes_opt > MAX_STROKES) return false;

    strokes.reserve(*num_strokes_opt);

    for (uint32_t i = 0; i < *num_strokes_opt; ++i) {
        auto paint_id_opt = reader.read<uint32_t>();
        if (!paint_id_opt) return false;

        auto num_points_opt = reader.read<uint32_t>();
        if (!num_points_opt) return false;
        
        // Another DoS protection
        const uint32_t MAX_POINTS_PER_STROKE = 250000;
        if (*num_points_opt > MAX_POINTS_PER_STROKE) return false;

        Stroke s;
        s.paint_id = *paint_id_opt;
        s.positions_x.resize(*num_points_opt);
        s.positions_y.resize(*num_points_opt);
        s.pressures.resize(*num_points_opt);

        // Read points from AoS on-disk format into our SoA in-memory format
        for (uint32_t j = 0; j < *num_points_opt; ++j) {
            auto pos_x_opt = reader.read<float>();
            auto pos_y_opt = reader.read<float>();
            auto pressure_opt = reader.read<float>();

            if (!pos_x_opt || !pos_y_opt || !pressure_opt) return false;

            s.positions_x[j] = *pos_x_opt;
            s.positions_y[j] = *pos_y_opt;
            s.pressures[j] = *pressure_opt;
        }
        strokes.push_back(std::move(s));
    }
    return true;
}

This function is a perfect example of defensive deserialization. We read the counts (num_strokes, num_points), validate them against sane limits, and then proceed with reading the data, checking for failure at every step.

Performance: Beyond Simple Parsing

Our BinaryReader is correct and safe, but for a high-performance application, it's worth exploring more advanced concepts. The approach we've taken involves parsing: we read bytes from the stream and copy them into newly constructed C++ objects. This is flexible but involves overhead.

An alternative is zero-copy deserialization. The idea is to design a binary format that can be accessed in-place without a separate parsing step. The data on disk or in a memory buffer is already in a usable format. This avoids allocations and memory copies, making reads extremely fast.

Libraries like Cap'n Proto are built around this philosophy.

While we won't refactor our project to use Cap'n Proto, understanding its design provides a glimpse into the next level of performance optimization. The core takeaway is that by carefully designing the data layout, you can eliminate the entire parsing step.

Handling Complex Object Graphs

Our document structure is a simple tree. But what if we had objects that were shared between different parts of our document? For example, what if multiple strokes could share a single, complex gradient paint object? In C++, you would model this with std::shared_ptr.

Serializing this correctly is non-trivial. A naive deserializer would create a separate copy of the shared object for each shared_ptr it encounters, breaking the shared relationship and bloating memory.

A correct implementation requires a mechanism to track object identities. During serialization, the first time a shared object is encountered, it's written with a unique ID. Subsequent references just write the ID. During deserialization, the reader maintains a map of IDs to already-created objects. When it sees an ID, it looks up the existing object instead of creating a new one. This reconstructs the graph correctly.

Conclusion

In this lesson, you've completed the data persistence loop by building a robust and secure deserializer. You now have a BinaryReader that can safely parse your .wbd files and reconstruct your application's state.

Key Takeaways:

• Deserialization is Dangerous: Always treat input data as untrusted. Validate sizes, check for buffer overruns, and handle failures gracefully.
• The BinaryReader Pattern: A dedicated reader class that wraps the input stream and provides safe, high-level methods is a clean and robust way to manage parsing logic.
• Parsing is the Inverse of Writing: The logic for reading a custom format directly mirrors the logic used to write it, but with the critical addition of constant validation.
• Zero-Copy is the Performance Horizon: For ultimate read performance, libraries like Cap'n Proto use in-place access to data via mmap, avoiding the overhead of parsing and copying altogether.

With saving and loading now complete, your whiteboard application has a solid foundation for managing user documents. In the next lesson, we will move to the final module and implement a key feature: exporting your work to vector-based SVG and generating LaTeX documents, allowing users to share their creations in standard, high-quality formats.