Adam A. Holmes

adamaholmes@gmail.com · Google Scholar · GitHub · LinkedIn

Adam Holmes

Hi! I’m a computational physicist and AI researcher (Ph.D. Theoretical Physics, Cornell). I work on hard search problems — where the space of possibilities is far too large to enumerate, and the whole game is deciding what to look at next.

A common thread in my work has always been an approach to such problems: solve as much of it as you can with efficient exact methods, and fall back on learned or statistical ones for the remainder. The exact layer reduces what has to be learned, and improves the training signal for what’s left.

I started in quantum many-body physics, inventing new deterministic, stochastic, and semistochastic algorithms for high-precision first-principles calculations, using only the fundamental laws of quantum mechanics (2,400+ citations). Since then I’ve worked on large language model (LLM) efficiency, game playing, theorem proving, and chip design. Along the way I’ve built production AI systems since 2018 (Transformer-based semantic search, before Google’s BERT model made the approach standard), run my algorithms on some of the largest supercomputers in the world at Lawrence Livermore, and built quantitative models for systematic trading at Citadel.

I’m now looking for a role in AI research or systems engineering at an established or early-stage AI lab.


Large Language Models

Inference is bottlenecked by memory movement: for every token it generates, the model re-reads its key–value (KV) cache, the stored attention inputs for all earlier tokens. Every way of shrinking that cache is lossy, so the question is how much quality you buy back.

Sparse KV-Cache Reads

I developed a training-free method that speeds up long-context decoding by reading only part of the KV cache: it clusters the cached keys by direction and reads only the clusters that a cheap summary score ranks highest, using GPU kernels I wrote. Reading 20% of a 32,768-token cache, attention is 3× faster than FlashInfer, used by SGLang. The idea came from reading about SANTA and MagicPIG, which sample the cache instead; to come as close to the exact model as SANTA-style sampling, it reads 8% of the cache where sampling reads 51%. It is close to ClusterKV, which also clusters keys with k-means.

PyTorch · Triton

Error from the exact model versus percent of the cache read, for this method and systematic sampling

Error versus cache reads on Qwen3-4B at 8,192 tokens, as total variation distance (TVD) from the exact model’s next-token distribution, with 95% bootstrap intervals over 8 text chunks. The two sampling points use 64 and 256 samples.

Inference Engine + Post-hoc MLA

I studied converting a trained model’s attention to a compressed form, multi-head latent attention (MLA), after training, using a from-scratch single-GPU inference engine I wrote for Qwen3, Alibaba’s open-weight model family.

To recover the quality lost to compression, I train a small adapter that pulls the model’s output distribution back toward the original’s. Targeting the total-variation distance between the exact and approximate token distributions beats the standard Kullback–Leibler (KL) objective on every fidelity measure. The adapter merges into the weights, so it costs nothing at inference.

PyTorch · Triton

Neurosymbolic AI

Neurosymbolic Chess Engine

Self-play engines like AlphaZero learn everything from scratch, including positions a classical solver settles in microseconds. I developed an engine that searches classically first, and rewards any position an exact method can settle, such as a forced mate in N moves, rather than only checkmate. The training signal is denser, and unlike a learned reward model it cannot be gamed. It reaches ~600 Elo above an identically-trained purely neural run, in 18 generations rather than 28.

Rust · Monte Carlo Tree Search · PyTorch

Elo rating by training generation for the neurosymbolic and purely neural runs

Elo by training generation for both runs, with 95% bootstrap intervals, from an 18-model tournament of 6,579 games.

Search & Optimization

Electronic Design Automation

The arrangement of a chip’s large memory blocks (macros) largely determines the speed, power, and routability of everything placed after them. I built a macro placer that runs on the GPU, to optimize a function of wirelength, density (how crowded the blocks are), and congestion (how crowded the wiring is). Only the first two can be written as differentiable scores, so the placer works in two stages: it optimizes wirelength and density by gradient descent, legalizes the result so that no blocks overlap, then runs simulated annealing using the full score, including congestion.

After extensive Bayesian optimization of the hyperparameters, I found that the gradient stage works best when it optimizes wirelength alone at first and only gradually adds the density term, which is why the blocks in the animation collapse together and then slowly spread out. I ran it on a public challenge’s 17 benchmarks, with one hour of compute each, after the challenge had closed. It scored 33% better than RePlAce, a standard placement algorithm, with zero overlaps, which would have placed 4th. Full write-up.

PyTorch · GPU · Simulated Annealing

One complete run on benchmark ibm18: the layout as it spreads, legalizes and improves (left), and the score per frame, with the reference placement marked (right).

MMR-Elites

Often you want a diverse set of good solutions rather than the single best, but selecting on quality alone gives redundancy, because the best candidates cluster together. I developed MMR-Elites, which treats keeping such a set as submodular maximization, where each added item is worth less the more you already have, which makes greedy selection near-optimal. Borrowing Maximum Marginal Relevance (MMR) from information retrieval, it needs no grid at all, only a distance between solutions, and uses fixed O(K) memory and O(K log K) selection. That avoids the curse of dimensionality in MAP-Elites, the standard method, which keeps the best solution in each cell of a grid over behavior space, so its number of cells grows exponentially with the number of behavior dimensions. Choosing a varied, high-quality subset of LLM samples is an analogous problem.

Rust · PyO3 · Python

Multi-Agent Path Planning

I built a two-layer system for optimal multi-robot navigation. A global planner (Conflict-Based Search, CBS) computes provably optimal, collision-free routes for every robot before anything moves; a local controller (Optimal Reciprocal Collision Avoidance, ORCA) adjusts each robot’s velocity moment to moment for whatever the plan couldn’t anticipate.

Rust · PyO3 · CBS · ORCA

The test case from the ORCA paper, with local avoidance only: twelve agents on a circle each head for the opposite point. A red ring marks an agent being deflected.

Quantum Many-Body Algorithms

I like to think of quantum many-body physics as a graph search problem, but an unusually challenging one because it is a graph too large to even store! The nodes are electron configurations, and a molecule’s state is a weighted combination of them. Earlier methods generated enormous numbers of candidate configurations and tested each one. During my Ph.D. I developed a physics-informed heuristic that jumps straight to the ones that matter, called Heat-Bath Configuration Interaction, or HCI (Holmes et al., JCTC 2016). “Configuration interaction” is the field’s term for representing a state this way. The heuristic is the deterministic analogue of the heat-bath sampling algorithm I invented previously, which is why I named the method after it.

With my colleagues I then removed the memory bottleneck in perturbation theory, the step that accounts for configurations left out, by pairing a deterministic approximation built on that heuristic with stochastic sampling that corrects it (Sharma, Holmes et al., JCTC 2017). Together these became Semistochastic HCI (SHCI), now a benchmark algorithm in electronic structure theory, implemented in major quantum chemistry packages. With it we computed near-exact energy curves for fourteen states of the carbon dimer (~10²¹ configurations), now a reference for quantum computing and neural-network methods. We also computed a near-exact binding curve for the chromium dimer (~10⁴² configurations), where most methods fail badly (Li, Yao, Holmes et al., Phys. Rev. Res. 2020).

Matrix elements are precomputed and sorted by magnitude, so for each candidate the algorithm walks the list only until it drops below a threshold set by the current coefficient. Blue is generated; green is never touched. Figure from Smith, Mussard, Holmes & Sharma, *JCTC* 2017 (open access).

Selected papers


Tools

Small tools I built for my own workflow; both are open source.

butwhy.nvim

One of my favorite uses of LLMs is helping me understand things. I built a plugin for the Neovim text editor that explains any text I highlight, whether prose, code, or a LaTeX equation, in a pop-up beneath it, pitched to a short description of my background. If that’s still unclear, asking “but why?” re-explains it one level simpler, as many times as needed. It works with hosted or local models.

Selecting a line of NumPy code, explaining it, then asking for simpler explanations twice

picat

I often do remote development work, sometimes over slow coffee-shop Wi-Fi, and like to view images on the remote machine without copying them locally first. So I built picat (progressive icat, after the image viewer built into the kitty terminal), which shows a blurry preview immediately and then sharpens it, sending the most detailed parts first. Over a 10 Mbit/s link, a 3024×4032 photo, scaled to fit a 1000×1400-pixel window, is sharp in 0.8 s, where kitty’s own viewer shows nothing until 4.1 s.

kitten icat and picat side by side, showing the same photo over a 10 Mbit/s link