Similarity-Aware Surrogate-Assisted NAS with Graph Neural Networks

 Similarity surrogate-assisted evolutionary neural architecture search based on graph neural networks sits right at the intersection of efficiency and intelligence in modern AutoML. As neural architectures grow more complex, evaluating every candidate through full training becomes painfully expensive. This approach tackles that bottleneck by using a surrogate model that can predict performance, allowing the evolutionary search to focus its energy on the most promising architectures instead of wasting computation on weak candidates.

At the core of this idea is the representation of neural architectures as graphs. Layers become nodes, connections become edges, and the entire network structure is naturally encoded in a graph format. Graph neural networks (GNNs) are especially well-suited here because they can learn structural patterns and relationships directly from this representation. By processing these architecture graphs, the GNN surrogate learns how different design choices influence performance metrics such as accuracy, latency, or energy consumption.

The “similarity” aspect adds another smart layer to the process. Instead of treating each architecture in isolation, the surrogate model leverages similarity measures between architectures—often based on graph embeddings. Architectures that are structurally similar tend to exhibit similar performance, and the surrogate exploits this intuition to make more accurate predictions, even with limited training data. This significantly improves sample efficiency, especially in the early stages of the evolutionary search.

Within the evolutionary framework, the surrogate model guides selection, mutation, and crossover operations. Poorly predicted architectures can be filtered out early, while promising ones are refined and evolved further. Periodically, a small subset of architectures is fully trained to correct and update the surrogate, ensuring that prediction errors do not accumulate and mislead the search over time.

Overall, this combination of similarity-aware surrogates, evolutionary optimization, and graph neural networks creates a powerful and scalable NAS strategy. It balances exploration and exploitation more effectively than brute-force methods, reduces computational cost, and opens the door to searching much larger and richer architecture spaces—making high-performance neural design more accessible and sustainable.