Welcome back. In the last lesson, you moved from theory to practice by using EXPLAIN ANALYZE to investigate the real-world execution of PostgreSQL queries. You saw firsthand how to compare the planner's estimates with actual performance, uncovering the story behind the numbers. That process of forming a hypothesis ("I think this will be an Index Scan"), running an experiment (EXPLAIN ANALYZE ...), and analyzing the results is the very essence of engineering.
This lesson is about formalizing that process. We will address a key practice from the Professional Engineering track: Start an implementation journal to track learnings, assumptions, and questions after each study session. This is not a detour from your technical work; it is a tool to accelerate it and deepen your understanding. An engineering journal, or logbook, is the primary instrument for turning experience into expertise. It is the written record that will enable you to defend your decisions and understand the "why" behind your own code months or even years from now.
1. The Philosophy of the Engineering Journal
A common mistake is to treat an engineering journal as mere note-taking—a simple log of commands run or facts collected. Its true purpose is far deeper. It is a tool for structured thinking and reflection, a practice that separates the novice from the master. In science, researchers use lab notebooks not just to record data, but to document their thought process, their failed hypotheses, and their moments of insight. As an engineer building complex systems, your process is no different.
The article "Using a logbook to improve your programming" by James Routley provides an excellent introduction to this concept, rooted in traditional engineering disciplines.
Using a logbook to improve your programming
Read this article to understand the core methodology and benefits of keeping a logbook for programming.
Read the sections "Logbooks" and "Logbooks for programming". Focus on the four-step process described and the three key advantages it offers for software development.
The key insight is that a journal forces you to move beyond simply doing the work to understanding it. It transforms passive learning into active learning by requiring you to articulate the problem, your approach, and most importantly, your reflections on the outcome.
2. The Anatomy of a High-Value Entry
What distinguishes a useful journal entry from a useless one? Detail, structure, and reflection. A vague entry like "Worked on query performance" is worthless. A high-value entry provides enough context that you (or a colleague) could replicate the work and understand the reasoning behind it.
The PDF "Lab Notebooks for Computational Mathematics" provides a stark contrast between a bad entry and a better one, setting a standard for the level of detail required for professional work.
[PDF] LAB NOTEBOOKS FOR COMPUTATIONAL MATHEMATICS ...
This document from the high-performance computing world offers a rigorous perspective on what makes a scientific lab notebook effective.
Read the sections titled "A MINIMAL DEFINITION" and "EXAMPLE ENTRIES". Pay close attention to the specific details included in the "better example": the hypothesis, the exact commands, commit hashes, and the conclusion drawn from the results. This is the level of detail to which you should aspire.
Based on these principles, a strong journal entry for your work should contain the following sections:
- Date & Context: What is the date? What topic or project file are you working on?
- Goal: In one sentence, what are you trying to achieve or understand in this session?
- Hypothesis / Plan: What do you expect to happen? What is your plan of attack? For example: "I hypothesize that adding an index on
column_xwill change the query plan from a Seq Scan to an Index Scan and reduce latency by at least 50%. My plan is to first runEXPLAIN ANALYZEon the original query, then create the index, and run it again to compare." - Observations / Process: This is the raw data of your work. Record the exact commands you run, the code you write, key outputs, and any error messages. This is where you would paste the
EXPLAIN ANALYZEoutput from the previous lesson. - Reflection / Analysis: This is the most critical part. Here, you interpret your observations. A powerful model for structuring this reflection is the DIEP model (Describe, Interpret, Evaluate, Plan), as explained in this short video on reflective writing.
This video from the University of Melbourne's Academic Skills unit provides a structured method for reflection that moves beyond simple description.
Watch the following segments: What is reflection?: Understand the purpose of reflection. Avoiding description: Learn the primary pitfall to avoid. The DIEP model: This introduces the Describe, Interpret, Evaluate, Plan framework. Color-coding technique: A practical method for ensuring your reflections have depth.
Your "Reflection" section should answer questions like:
- (Interpret) What do these results mean? Was my hypothesis correct? Why or why not? What surprised me?
- (Evaluate) How does this connect to what I learned previously? Was this approach effective? What were its limitations?
- (Plan) What are the key takeaways? What questions do I have now? What will I do differently next time?
3. Putting it into Practice
Now it is time to start your journal.
Step 1: Setup
In your project's root directory, create a new folder named engineering_journal. Inside this folder, you will create a new Markdown file for each study session, named with the date (e.g., YYYY-MM-DD.md). This simple, text-based format is powerful, searchable, and easily managed with Git.
Step 2: Your First Entry
Create a new file for today's date. Your first task is to write a journal entry about... starting your journal. Use the structure defined above.
Example Template (YYYY-MM-DD.md):
# Engineering Journal: YYYY-MM-DD
**Context:** Module 1, Lesson 7: Engineering Journals
**Goal:** To set up my engineering journal and understand the principles behind keeping one.
**Plan:**
1. Read the provided resources on engineering logbooks and reflective writing.
2. Create the `engineering_journal` directory in my project repo.
3. Create this first entry using the prescribed template to solidify my understanding of the format.
**Observations:**
* The Routley article emphasizes a 4-step process: Problem, Method, Process, Reflection.
* The CMU PDF highlighted the importance of specifics, showing a "bad" vs "good" entry. The good entry included commit hashes and explicit hypotheses.
* The Melbourne video introduced the DIEP model (Describe, Interpret, Evaluate, Plan) for the reflection part. This seems like a useful way to add structure to the most difficult part.
**Reflection (using DIEP):**
* **(Describe)** I've set up the journal structure and written my first entry.
* **(Interpret)** I see now that this isn't just about taking notes. It's a formal process for thinking about my own work. The emphasis on recording hypotheses *before* the work is done seems particularly powerful. It forces me to have a clear intention.
* **(Evaluate)** This practice seems directly aligned with my goal of "mastery, not familiarity." Having a record of my thought process will be invaluable for defending my architectural choices later. It feels like a lot of discipline, but the potential payoff in accelerated learning is clear.
* **(Plan)** I will commit to creating an entry for every study session. I will use the color-coding technique from the video on my first few entries to ensure I am reflecting and not just describing. My next step is to apply this retroactively to yesterday's lesson.
Step 3: Your Second Entry (Retrospective)
Now, create a second journal entry for the previous lesson on EXPLAIN ANALYZE. This will be a retrospective entry. Go back through the lesson content. Re-run the queries if you need to. Document your investigation of the three query types (highly selective, non-selective, and the join).
- State your hypothesis for each query before you show the
EXPLAIN ANALYZEoutput. - Paste the actual output into the "Observations" section.
- In the "Reflection" section, analyze why the planner made its choice and compare the estimated vs. actual rows and timings. This exercise will not only solidify your new journaling practice but also reinforce your understanding of the query planner.
Conclusion
You have now established a fundamental professional habit. The engineering journal is your personal knowledge base, your debugging partner, and your learning accelerator. It may feel like extra work initially, but it will pay for itself many times over by making your thinking sharper, your memory longer, and your learning curve steeper.
Key Takeaways:
- An engineering journal is a tool for structured thinking, not just note-taking.
- A high-value entry includes your goals, hypotheses, detailed observations (code, outputs), and a structured reflection on the outcomes.
- The DIEP (Describe, Interpret, Evaluate, Plan) model provides a robust framework for the crucial reflection stage.
- This practice is a cornerstone of achieving mastery and being able to defend your engineering decisions.
In our next and final lesson for this module, you will learn about another critical documentation practice: the Architecture Decision Record (ADR). While your journal is the private record of your thought process, an ADR is the public record of a significant system decision. You will see how these two practices complement each other to form a foundation of engineering rigor.