Speed Up Development with Cache and Run

1. The Challenge of Iterative Development

Imagine a workflow that takes 15 minutes to run. The first 10 minutes are spent connecting to a database, downloading a large dataset, and performing several complex data cleansing and joining operations. The final 5 minutes are for creating a summary report.

Now, what if you want to change a simple setting in the final reporting tool? Without caching, you would have to run the entire 15-minute workflow again just to see your change. If you need to make ten small adjustments, you could lose hours of valuable development time.

This is where caching comes in.

2. Caching: Your Workflow's "Save Point"

Caching in Alteryx is the process of storing the data that flows out of a specific tool into a temporary file on your computer. When you run the workflow again, Alteryx can start from this cached "save point" instead of re-running all the tools that came before it.

To understand the value of this feature, let's look at a quick explanation.

As the article explains, by caching the data after the initial complex steps, you can freely experiment with the downstream tools, and each subsequent run will be significantly faster.

You can have multiple cache points in a single workflow, essentially creating multiple "snapshots" of your data along the way.

3. How to Cache Your Workflow

Activating the cache is straightforward. You simply right-click on the tool whose output you want to save.

Once you select "Cache and Run Workflow," Alteryx will:

1. Run the workflow from the beginning up to and including the tool you selected.
2. Write the output of that tool to a temporary .yxdb file.
3. Visually indicate that the preceding tools are cached. This is often described as the tools being "bubble-wrapped" or enclosed in a bubble.

Now, when you run the workflow again (by pressing Ctrl+R or the normal Run button), Alteryx will start with the data from the cache, skipping all the upstream "bubble-wrapped" tools.

For more details on this process and its limitations, the next resource provides a concise summary.

4. Important Rules and Considerations

Caching is a powerful feature, but you must be aware of its rules to use it effectively and avoid getting unexpected results.

Clearing the Cache

There are two main scenarios for clearing a cache:

1. Automatic Clearing: If you make any changes to the configuration of a cached tool or any tool upstream of it, Alteryx intelligently recognizes that the cache is no longer valid. It will automatically clear the cache the next time you run the workflow.
2. Manual Clearing: This is a critical point. Caching does not detect changes in the original input data files. If you cache a workflow that reads from an Excel file, and then a colleague updates that Excel file, running the workflow again will use the old, cached data.
   • To fix this: You must manually clear the cache to force Alteryx to re-read the input file. To do this, right-click the cached tool and select "Clear Cache".

Limitations

As noted in the "TIPS AND TRICKS" document, you cannot cache every tool. The "Cache and Run Workflow" option will be grayed out if:

• The tool has more than one output anchor (e.g., the Join tool).
• The tool is part of a batch or iterative macro loop.
• The tool is an In-DB (In-Database) tool.

Additionally, caching relies on writing an output file. If you have the global workflow option "Disable All Tools that Write Output" checked in the Runtime settings, caching will not work.

5. Practical Exercise

Let's apply this with a quick thought exercise. You don't need to build this in Alteryx, but visualize the steps.

You have a workflow with the following structure:

1. Input Data tool loading a large CSV file.
2. Data Cleansing tool removing nulls and whitespace.
3. Formula tool creating several new columns.
4. Join tool combining the data with a lookup table.
5. Filter tool to select a subset of data.

You have run the workflow, and it works, but now you want to experiment with different filter conditions in the Filter tool (step 5). The first four steps take 10 minutes to run.

Task:

1. Where would you apply the cache to make your work more efficient?
2. What steps would you take to apply it?
3. After applying the cache and running the workflow, you realize you need to change the logic in the Formula tool (step 3). What happens to the cache when you run the workflow next?
4. Later, you are informed that the original CSV file (step 1) has been updated. What must you do to ensure your workflow uses the new data?

Conclusion

In this lesson, we shifted our focus from optimizing workflow runtime to optimizing your personal development time. By using the "Cache and Run Workflow" feature, you can avoid re-running time-consuming upstream processes, allowing for rapid testing and iteration on the later parts of your workflow.

Key Takeaways:

• Purpose: Caching improves development speed by creating "save points" for your data within a workflow.
• How to Use: Right-click a tool and select "Cache and Run Workflow." Cached tools will appear "bubble-wrapped."
• Manual vs. Automatic Clearing: The cache clears automatically if you change an upstream tool's configuration, but you must manually clear the cache if the source input file changes.
• Limitations: Caching is not available for all tools, particularly those with multiple outputs, in macros, or in-database tools.

Preview of the Next Lesson:
Continuing with our theme of managing complex workflows, our next lesson will introduce another powerful organizational and execution feature. You will learn how to disable tool containers to manage the execution of complex workflows, giving you another method to control which parts of your workflow run at any given time.