Visualizing Trend Data

From Data to Story: What to Visualize?

Not all information is created equal, especially when it comes to visual media. A dense paragraph of text might be informative, but a well-chosen chart or diagram can convey the core message in a fraction of a second. The art is in selecting what to visualize. This is the essence of data storytelling.

To build a strong foundation, let's watch a video that explains this concept with excellent clarity.

As the video explains, identifying key data points is about asking the right questions of your data and then choosing a visual format that answers those questions clearly. The main categories to look for are:

• Trends over time: How has a metric changed? (Perfect for line charts).
• Comparisons: How does A compare to B? (Excellent for bar charts).
• Distribution/Mix: What are the components of a whole? (Suits area graphs or pie/donut charts).
• Value Reporting: What is the specific number? (Can be a simple table or a bold "stat card").

A Systematic Framework for Identification

Now that we have a conceptual understanding, let's adopt a systematic approach. When our automation pipeline analyzes a trend narrative, what specific patterns should it look for?

An article from Colorado Web Impressions on repurposing blog posts into infographics provides a great, straightforward checklist.

Based on these resources, we can establish a clear set of criteria for our automated agent. We are hunting for:

• Key Statistics: Any specific number, percentage, or metric that quantifies a point (e.g., "a 40% increase in adoption").
• Processes/Workflows: A sequence of steps that can be laid out visually (e.g., "Step 1: Collect data, Step 2: Analyze...").
• Comparisons: Direct comparisons between two or more items.
• Timelines: A series of events with associated dates.
• Key Quotes: A short, impactful statement from a source.
• Core Concepts: A central idea or definition that can be represented with an icon and text.

Automating Identification with an LLM

Manually scanning for these points works, but it doesn't scale. Our goal is automation. To achieve this, we will instruct an LLM to act as a data analyst, scan the trend narrative, and extract these visualizable data points into a structured format.

This is a common and powerful AI workflow. The following video demonstrates a similar process, where an AI first extracts key data from a document and then another AI generates an infographic from it. We are building the first half of that pipeline right now.

To guide our prompt engineering, we can draw inspiration from established prompt libraries.

Activity: Building the identifyVisualDataPoints Function

It's time to build. We'll create a new file, src/identifyVisuals.ts, and implement a function that takes our trend narrative and returns a structured array of visualizable data points.

Given your background as a data engineer, you'll appreciate the importance of a well-defined data structure and traceability. We'll design our output interface to be robust and useful for downstream tasks.

// src/identifyVisuals.ts

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

// We can reuse the TrendInsight type from previous lessons
interface TrendInsight {
  narrative: string;
  keywords: string[];
}

export interface VisualDataPoint {
  id: string; // A unique identifier, e.g., 'stat-01', 'process-01'
  type: 'statistic' | 'comparison' | 'timeline' | 'process' | 'quote' | 'concept';
  title: string; // A short, punchy title for the visual, e.g., "User Growth Explodes"
  data: any; // The structured data itself. e.g., { label: "Q4 Growth", value: "30%" } or ["Step 1", "Step 2"]
  suggested_visualization: 'bar_chart' | 'line_chart' | 'donut_chart' | 'flow_chart' | 'quote_card' | 'icon_and_text';
  source_text: string; // The exact snippet from the narrative this was extracted from, for traceability.
}

/**
 * Creates a prompt to extract visualizable data points from a text.
 * @param trendNarrative - The source text to analyze.
 * @returns A string containing the full prompt for the LLM.
 */
function getVisualIdentificationPrompt(trendNarrative: string): string {
  const prompt = `
    You are an expert Data Analyst and Information Designer. Your task is to analyze the following text about a technology trend and extract key data points that would be suitable for visual representation in an infographic or social media carousel.

    **CONTEXT (Trend Analysis Text):**
    "${trendNarrative}"

    **IDENTIFICATION CRITERIA:**
    Scan the text for the following types of information:
    1.  **Statistics:** Specific numbers, percentages, growth metrics, or quantitative data.
    2.  **Comparisons:** Side-by-side comparisons of features, products, or metrics.
    3.  **Timelines:** A sequence of events or milestones over time.
    4.  **Processes:** A step-by-step workflow.
    5.  **Key Quotes:** Impactful and concise quotes.
    6.  **Core Concepts:** A fundamental definition or idea that can be simplified visually.

    **OUTPUT FORMAT:**
    You MUST respond with ONLY a valid JSON object containing an array of 'VisualDataPoint' objects. Do not include any other text or markdown. The JSON must adhere to the following TypeScript interface:
    \`\`\`typescript
    interface VisualDataPoint {
      id: string; // A unique identifier, e.g., 'stat-01'
      type: 'statistic' | 'comparison' | 'timeline' | 'process' | 'quote' | 'concept';
      title: string; // A short, punchy title for the visual, e.g., "Adoption Rate Skyrockets"
      data: any; // Structured data. For 'statistic', use {label: string, value: string}. For 'process', use string[].
      suggested_visualization: 'bar_chart' | 'line_chart' | 'donut_chart' | 'flow_chart' | 'quote_card' | 'icon_and_text';
      source_text: string; // The exact text snippet from the context that this data was extracted from.
    }
    \`\`\`
    Analyze the text and extract all relevant visual data points you can find.
  `;
  return prompt;
}

/**
 * Identifies visualizable data points from a trend narrative using an LLM.
 * @param insight - An object containing the trend narrative.
 * @returns An array of VisualDataPoint objects, or null on error.
 */
export async function identifyVisualDataPoints(
  insight: TrendInsight
): Promise<VisualDataPoint[] | null> {
  const prompt = getVisualIdentificationPrompt(insight.narrative);

  try {
    console.log(`Identifying visual data points from trend narrative...`);
    const response = await openai.chat.completions.create({
      model: "gpt-4-turbo",
      messages: [{ role: "user", content: prompt }],
      response_format: { type: "json_object" },
      temperature: 0.2, // We want analytical and deterministic extraction
      max_tokens: 2000,
    });

    const rawContent = response.choices[0].message.content;
    if (!rawContent) {
      console.error("LLM returned empty content for visual data points.");
      return null;
    }
    
    // The response is expected to be a JSON object with a single key, e.g., {"visual_data_points": [...]}. We need to extract the array.
    const parsedJson = JSON.parse(rawContent);
    const dataPoints = parsedJson.visual_data_points || parsedJson.data_points || (Array.isArray(parsedJson) ? parsedJson : null);


    if (!dataPoints || !Array.isArray(dataPoints)) {
        console.error("Parsed JSON does not contain a valid array of data points.", parsedJson);
        return null;
    }
    
    console.log("Successfully identified visual data points:", JSON.stringify(dataPoints, null, 2));
    return dataPoints as VisualDataPoint[];

  } catch (error) {
    console.error("Error identifying visual data points:", error);
    return null;
  }
}

Deconstructing the Code

• VisualDataPoint Interface: This is a highly structured interface designed for our pipeline. id and type allow for easy filtering, data contains the clean information, suggested_visualization gives a strong hint to the next agent, and source_text is crucial for verification and debugging.
• getVisualIdentificationPrompt: The prompt is very explicit. It sets the expert persona, defines the exact task, lists clear criteria for what to find, and provides a strict schema for the output, including the TypeScript interface.
• identifyVisualDataPoints Function: This orchestrates the process. We use a low temperature (e.g., 0.2) because this is an analytical extraction task, not a creative one; we want consistency. We also handle the JSON parsing, anticipating that the model might wrap the array in a parent object. The output is a clean array of VisualDataPoint objects—a valuable, structured asset for our content pipeline.

Connecting to the Pipeline

So, how does this new VisualDataPoint[] array fit with the CarouselPlan we created in the last lesson? Think of them as two parallel, complementary assets.

1. The CarouselPlan provides the narrative backbone and slide-by-slide text.
2. The VisualDataPoint[] array acts as a "bank" of pre-identified, high-impact visuals we can draw from.

In a more advanced agentic workflow (which we will build in Module 6), you could design a "director" agent. For each slide in the CarouselPlan, this director would look at the visual_suggestion (e.g., "chart showing growth") and then search the VisualDataPoint array for the most relevant item (e.g., one with type: 'statistic' and a matching title). This detailed VisualDataPoint object would then be passed to the image generation agent.

For now, we have successfully created the structured data needed to make that possible.

Conclusion

In this lesson, you learned how to move beyond simple visual suggestions and systematically identify the most compelling data points within a trend analysis for visual representation.

Key Takeaways:

• Visually compelling data often falls into categories like trends, comparisons, processes, or key statistics.
• We can create a systematic framework to hunt for these data types within a larger text.
• A well-crafted LLM prompt can automate this extraction process, transforming unstructured text into a structured JSON array of VisualDataPoint objects.
• This structured data is a valuable asset that enables more sophisticated and reliable automation in the subsequent visual generation steps.

Preview of the next lesson:
We now have a narrative plan and a bank of data-driven visual concepts. The next logical step is to bring them to life. In our upcoming lesson, we will generate visual assets for carousels and infographics using an image generation API, turning our structured VisualDataPoint objects into actual .png files.