Coil Optimization with Gradient and Genetic Algorithms

1. The "Hill-Climbing" Approach: Local Optimization

Imagine your design space is a vast, mountainous landscape shrouded in fog. Your objective is to find the lowest point (e.g., to minimize stress). A local optimization algorithm works like a hiker who can only feel the slope directly under their feet. They take a step in the steepest downward direction, reassess, and repeat. This process is often called "hill-climbing" or, in our case, "valley-descending."

Gradient-Based Methods

The most mathematically direct way to find the "steepest direction" is to compute the gradient of the objective function, $\nabla f(\vec{x})$. This vector points in the direction of the greatest increase of the function $f$. To minimize $f$, the algorithm simply takes a step in the direction of $-\nabla f(\vec{x})$.

• Pros: When the design landscape is smooth and has only one valley (i.e., it's a convex problem), gradient-based methods are extremely fast and efficient.
• Cons: They are easily trapped in the first local minimum they find, potentially missing a much deeper valley elsewhere. Furthermore, for our complex multi-physics simulations, calculating the gradient (which requires a "sensitivity analysis") can be computationally expensive or numerically unstable.

Direct Search Methods (Gradient-Free)

What if you can't compute the gradient? Direct search methods provide an alternative. They don't use derivative information but instead evaluate the objective function at a pattern of points to decide where to move next.

A classic example, and one highly relevant to our topic, is the Nelder-Mead simplex method. This is the algorithm used in the dissertation we've been studying.

The Nelder-Mead method works by forming a "simplex" (e.g., a triangle in a 2D design space) of trial points. It evaluates the objective at each vertex and iteratively improves the simplex by reflecting, expanding, or contracting it away from the worst-performing vertex. It's a robust way to find the bottom of the nearest valley, even if the terrain is a bit rough. However, like gradient-based methods, it has no mechanism for escaping that valley to find a better one elsewhere.

2. The Power of Evolution: Genetic Algorithms (GAs)

To overcome the limitations of local search, we turn to a different inspiration: natural selection. Genetic Algorithms (GAs) are global search methods that maintain a "population" of candidate designs. Over many "generations," the population evolves towards optimal solutions.

This approach is especially powerful for magnet design because it can handle complex problems with both continuous variables (like winding angles) and discrete variables (like the number of turns in a block), a significant challenge for traditional methods.

The following paper from CERN demonstrates beautifully why and how GAs are used for designing accelerator magnets.

As you read, the core workflow of a GA is:

1. Initialization: Create a large, random population of designs (chromosomes).
2. Evaluation: Calculate the "fitness" of each design by running a simulation (e.g., FEM) and evaluating the objective function.
3. Selection: Choose parent designs for the next generation. Fitter individuals have a higher probability of being selected.
4. Reproduction:
   • Crossover: Combine parts of two parent chromosomes to create one or more offspring. This is the primary exploration mechanism, mixing and matching good features from different designs.
   • Mutation: Introduce small, random changes to an offspring's chromosome. This maintains genetic diversity and prevents the algorithm from converging too early.
5. Repeat: The new generation replaces the old, and the cycle repeats.

The remarkable outcome of this process is that GAs can uncover non-intuitive, high-performance designs that a human designer, or a local optimizer, might never find. The CERN paper showcases this by finding a novel 6-block coil design that outperformed the existing 5-block design.

3. Multi-Objective Genetic Algorithms (MOGAs)

In our previous lesson, we established that real-world design involves trading off between multiple, often conflicting, objectives (e.g., maximizing field vs. minimizing stress). A simple GA that optimizes for a single fitness value isn't enough. We need a method that can find the entire Pareto front of optimal trade-offs.

Multi-Objective Genetic Algorithms (MOGAs) achieve this by changing how individuals are selected. Instead of a single fitness score, individuals are ranked based on the concept of Pareto dominance.

A design A dominates a design B if A is better than or equal to B in all objectives, and strictly better in at least one objective. The goal of a MOGA is to find the set of non-dominated solutions.

A key challenge in this process is ensuring the final population is spread out across the entire Pareto front, giving the designer a diverse set of trade-offs to choose from. The following video explains this beautifully.

Algorithms like NSGA-II (Non-dominated Sorting Genetic Algorithm II) are industry-standard tools that implement exactly these concepts: non-dominated sorting to identify the Pareto fronts, and crowding distance to maintain diversity along those fronts.

4. A Practical Optimization Workflow

In practice, designers often combine the strengths of both local and global search methods. A typical workflow for designing a novel magnet might look like this:

1. Global Exploration: Use a Multi-Objective Genetic Algorithm (e.g., NSGA-II) to perform a broad search of the design space. This computationally intensive step might run for hours or days, calling your FEM solver thousands of times to evaluate different designs. The output is not a single answer but the Pareto front of promising candidate topologies.
2. Analysis and Selection: The engineer examines the Pareto front, plotting the trade-offs (e.g., peak field vs. coil mass vs. peak stress). Based on the project's specific priorities, one or a few candidate designs are selected from the front for further refinement.
3. Local Refinement: Starting from a promising design found by the GA, a local search algorithm (like Nelder-Mead or a gradient-based method) is used to "polish" the solution. This fine-tunes the continuous parameters to find the precise local minimum in that region of the design space.

This hybrid approach gives you the best of both worlds: the creative, exploratory power of GAs and the fast, precise convergence of local optimizers. The Rembeczki dissertation is a great example of the latter part of this process, applying a local optimizer to refine a well-defined "variable pitch" concept.

Conclusion

Today, we've opened the black box of design optimization and looked at the algorithms inside. You now understand the fundamental differences between local "hill-climbers" and global "evolutionary explorers" and how they are applied to the problem of magnet design.

Key Takeaways:

• Local vs. Global Search: Gradient-based and direct search methods (like Nelder-Mead) are efficient for refining a design locally but can get stuck in suboptimal solutions.
• Genetic Algorithms (GAs): GAs use principles of evolution (population, selection, crossover, mutation) to perform a robust global search, making them ideal for discovering novel designs with mixed continuous and discrete variables.
• Multi-Objective GAs (MOGAs): By using Pareto dominance ranking and diversity-preservation techniques like crowding distance, MOGAs can identify the entire family of optimal trade-off solutions for problems with conflicting objectives.
• Hybrid Workflow: A powerful strategy combines a GA for initial global exploration with a local optimizer for final refinement of the chosen design.

Next Lesson Preview

We have now learned how to build complex multi-physics simulations and use sophisticated algorithms to find optimal designs based on those simulations. But this all hinges on one critical assumption: that our simulations are an accurate reflection of reality. How can we be sure?

In our final lesson of this module, we will tackle this question head-on. The topic is "Validate simulation results against analytical solutions for simple geometries and available experimental data." We will explore the essential engineering practice of Verification and Validation (V&V), which provides the confidence needed to trust our simulation results and, by extension, the designs they produce.