Optimizing Trend Scores with Performance Feedback

1. The Concept of a Self-Improving Agent

Before we dive into the code, let's establish the high-level concept. We are building an "LLM in a loop" system where the agent learns from real-world feedback.

To understand this core idea and its implications, please watch the following video.

As you saw, a feedback loop is a powerful mechanism, but it requires careful design to avoid unintended consequences like reward hacking. Our goal is to steer the agent toward better content, not just maximize a single number at any cost.

2. A Two-Pipeline Architecture: Creation and Learning

A clean way to structure a self-improving system is to separate the content creation logic from the learning logic. Your experience designing event-driven systems will make this pattern feel familiar. We can think of our agent as having two distinct pipelines that share a knowledge base.

This approach is demonstrated beautifully in a real-world open-source project.

Our implementation will mirror this architecture:

1. Creation Pipeline: Your existing agent logic for finding trends, generating, and publishing content.
2. Learning Pipeline: The logic we'll build now, which runs periodically to analyze performance and update a shared "strategy."

The "strategy" we'll be updating is a set of scores that represent the historical performance of different content topics.

3. Implementing the Learning Pipeline

The learning pipeline's job is to take the output of analyzePerformance from our last lesson and distill it into a persistent, evolving score for each topic.

Step 1: Defining a Persistent Strategy Store

First, we need a place to store our evolving topic scores. In a production system, you'd use a dedicated table in your Postgres database. For our purposes, a simple JSON file will serve as our "knowledge base."

Let's define the structure for our topic strategy in a new file, src/monitoring/strategy.ts:

// src/monitoring/strategy.ts

export interface TopicStrategy {
  topic: string;
  // A score representing historical performance, normalized between 0 and 1.
  performanceScore: number; 
  lastUpdatedAt: string; // ISO 8601 timestamp
}

Step 2: Temporal Weighting with Exponential Moving Average (EMA)

If a post about 'GraphQL' goes viral today, our system should favor that topic. But if it was popular six months ago and has since faded, its influence should diminish. We need to weigh recent performance more heavily than past performance. This is called temporal weighting.

A robust and common technique for this is the Exponential Moving Average (EMA). Unlike a simple moving average, an EMA gives exponentially more weight to recent data points.

The formula for an EMA is:
${\text{EMA}}_{\text{today}}=({\text{Value}}_{\text{today}}\times \alpha )+({\text{EMA}}_{\text{yesterday}}\times (1-\alpha ))$
Where $\alpha$ is a smoothing factor between 0 and 1. A higher $\alpha$ gives more weight to recent data.

We can adapt this to update our performanceScore. Let's create a function that takes the latest analytics and updates our strategy store using EMA.

// src/monitoring/strategy.ts (continued)
import * as fs from 'fs/promises';
import { PerformanceSummary } from './analysis'; // From previous lesson

const STRATEGY_DB_PATH = './strategy_store.json';
const SMOOTHING_FACTOR = 0.3; // Alpha for our EMA. Higher = more reactive to recent performance.

async function readStrategyStore(): Promise<TopicStrategy[]> {
  try {
    const data = await fs.readFile(STRATEGY_DB_PATH, 'utf-8');
    return JSON.parse(data) as TopicStrategy[];
  } catch (error: any) {
    if (error.code === 'ENOENT') return [];
    throw error;
  }
}

// The core of our Learning Pipeline
export async function updateTopicPerformanceScores(topicAnalysis: PerformanceSummary): Promise<void> {
  const strategyStore = await readStrategyStore();
  const newStrategyStore: TopicStrategy[] = [];

  for (const topic in topicAnalysis) {
    const analysisData = topicAnalysis[topic];
    const existingStrategy = strategyStore.find(s => s.topic === topic);

    // Normalize the current period's weighted score to a 0-1 scale.
    // This is a simple normalization. A more robust method could use min-max scaling across all topics for the period.
    // For now, let's assume a max possible score to normalize against.
    // e.g., a post with 500 likes, 100 comments, 50 shares = 500*1 + 100*3 + 50*5 = 1050
    // Let's cap our per-post score for normalization.
    const maxPossibleWeightedScore = 2000;
    const currentPeriodScore = Math.min(analysisData.weightedEngagementScore / analysisData.postCount / maxPossibleWeightedScore, 1);
    
    let newPerformanceScore: number;

    if (existingStrategy) {
      // Apply EMA formula
      newPerformanceScore = (currentPeriodScore * SMOOTHING_FACTOR) + (existingStrategy.performanceScore * (1 - SMOOTHING_FACTOR));
      
      const strategyIndex = newStrategyStore.findIndex(s => s.topic === topic);
      if(strategyIndex > -1) newStrategyStore[strategyIndex].performanceScore = newPerformanceScore;

    } else {
      // If it's a new topic, its initial score is just the current period's score.
      newPerformanceScore = currentPeriodScore;
    }

    newStrategyStore.push({
      topic: topic,
      performanceScore: newPerformanceScore,
      lastUpdatedAt: new Date().toISOString(),
    });
  }

  // Add back any topics from the old store that weren't in the new analysis,
  // but decay their score slightly so they fade over time if not posted about.
  const decayFactor = 0.95;
  for (const oldStrategy of strategyStore) {
    if (!newStrategyStore.some(s => s.topic === oldStrategy.topic)) {
      oldStrategy.performanceScore *= decayFactor;
      newStrategyStore.push(oldStrategy);
    }
  }

  await fs.writeFile(STRATEGY_DB_PATH, JSON.stringify(newStrategyStore, null, 2));
  console.log('Updated topic performance scores in the strategy store.');
}

This updateTopicPerformanceScores function is the heart of your learning pipeline. It runs periodically (e.g., daily), consumes the latest analytics, and intelligently updates the long-term performance score for each topic.

4. Integrating Feedback into Trend Scoring

Now that we have a persistent, evolving performanceScore for each topic, we need to use it in our Creation Pipeline.

Back in Module 3, your agent was designed to identify and rank trends. Let's assume you have a function, rankTrends, that assigns a score to potential topics. We will now modify that function to incorporate our new historical performance data.

This is the "Pattern History" signal from the AutoViralAI project. A trend's final score will be a composite of its current "hype" and its historical "performance."

// This is a hypothetical modification of your trend ranking logic from Module 3.
// Let's assume it lives in `src/analysis/trends.ts`

// Assume Trend has this structure
interface Trend {
  topic: string;
  // Score based on recent velocity, e.g., from HackerNews, Reddit
  rawHypeScore: number; 
}

interface RankedTrend extends Trend {
  finalScore: number;
}

// --- MODIFICATION ---
import { readStrategyStore } from '../monitoring/strategy';

export async function rankTrends(trends: Trend[]): Promise<RankedTrend[]> {
  const strategyStore = await readStrategyStore();

  const rankedTrends = trends.map(trend => {
    const topicStrategy = strategyStore.find(s => s.topic === trend.topic);
    
    // If we have historical data, use it. Otherwise, default to a neutral score (0.5).
    const historicalPerformanceScore = topicStrategy ? topicStrategy.performanceScore : 0.5;

    // --- The Core of the Feedback Loop ---
    // Combine raw hype with historical performance. We can weigh them.
    // Let's say current hype is 60% of the score and past performance is 40%.
    const hypeWeight = 0.6;
    const performanceWeight = 0.4;
    
    const finalScore = (trend.rawHypeScore * hypeWeight) + (historicalPerformanceScore * performanceWeight);

    return {
      ...trend,
      finalScore: finalScore,
    };
  });

  // Sort by the new finalScore to get the top trend
  return rankedTrends.sort((a, b) => b.finalScore - a.finalScore);
}

With this change, your agent is no longer just chasing whatever is new and trending. It balances novelty with proven success, using real engagement data to guide its topic selection. A topic that is moderately trending but has historically performed very well for your audience might now be ranked higher than a massively trending topic that your audience has ignored in the past.

Just as Facebook assigns different weights to different user interactions, we are now weighting trends based on their value to our specific goals.

Conclusion

Congratulations! You have officially closed the loop and implemented a self-improving agent. By creating a learning pipeline that updates topic scores with an EMA and integrating those scores back into your trend ranking, your system now learns from past content performance.

Key Takeaways:

• Two-Pipeline Architecture: Separating content Creation from Learning creates a clean, maintainable system.
• Temporal Weighting is Key: Using an Exponential Moving Average (EMA) ensures your agent adapts to changing audience preferences by weighting recent performance more heavily.
• Composite Scoring: The best decisions come from multiple signals. By combining a trend's current "hype" with its historical "performance," your agent makes more nuanced and effective choices.
• Reward Hacking is a Real Risk: A feedback loop will optimize for the metric you give it. Your sophisticated design, which balances multiple signals and includes decay, helps mitigate this risk compared to a naive "maximize likes" approach.

Preview of the next lesson:
We have successfully taught our agent what topics to prioritize. But what about how it writes about them? In our final lesson, "Design a meta-prompting strategy where an LLM suggests improvements to generation prompts based on engagement data," we will explore how to use AI to reflect on content quality itself, creating a second feedback loop that aims to improve the style, structure, and effectiveness of the generated text.