# Williams’ Law research program

> A research program examining whether cumulative algorithmic innovation can compound AI performance beyond hardware scaling alone.

[Canonical HTML page](https://trustcyber.ca/research/williams-law/)

- Author: Junior Williams
- Status: Proposed framework; empirical validation is not established
- Evidence register updated: 2026-08-30

## Proposed formulation

`P(H, A) = P(H0, 0)(H/H0)^alpha exp(lambda A)`
H is hardware capacity. A is a latent innovation index. Alpha and lambda are model parameters.

## Current evidence boundary

The Stage 1 protocol is prospective, reports no confirmatory results, does not estimate A, and does not validate Williams’ Law.
Williams-authored examples and papers establish the proposed framework; they are not independent validation.

## Governed propositions

- **CLM-001 · theory · unverified:** Williams’ Law proposes that cumulative algorithmic innovation can compound AI performance beyond hardware scaling alone. Accurate as a statement of the proposed framework. Independent efficiency trends motivate the research question but do not validate a universal law.
- **CLM-002 · theory · verified:** The source formulation is P(H, A) = P(H0, 0)(H/H0)^alpha exp(lambda A). Verified as source attribution; verification does not establish the empirical truth of the equation.
- **CLM-003 · interpretation · verified:** Historical examples in the foundational papers are illustrative source examples rather than independent validation of Williams’ Law. Governing evidence distinction for all downstream material.
- **CLM-004 · theory · verified:** The algorithmic innovation index A remains a latent conceptual construct in the foundational framework. The Stage 1 protocol explicitly states that it does not estimate A.
- **CLM-005 · protocol · verified:** The Stage 1 protocol is prospective: it reports no confirmatory model-response dataset, completed confirmatory analysis, empirical result, or verified preregistration. This status must remain visible wherever the protocol is described.
- **CLM-006 · protocol · verified:** The Stage 1 protocol prospectively compares composed inference policies using paired accuracy and resource estimands under bounded hosted-inference conditions. This describes the planned design, not an executed experiment.
- **CLM-007 · interpretation · verified:** A positive Stage 1 result would support only a bounded claim about the tested policies, models, prompts, datasets, endpoints, and execution period; it would not validate the latent index A or the universal law. Directly follows the protocol’s prospective supersession and interpretation boundaries.
- **CLM-010 · historical-trend · verified:** Hernandez and Brown documented substantial historical reductions in compute needed to reach fixed performance thresholds in selected neural-network domains. Supports a bounded algorithmic-efficiency trend, not the exponential form of Williams’ Law.
- **CLM-011 · historical-trend · verified:** Ho and colleagues estimate algorithmic progress in language models across multiple evaluations and compute-equivalent comparisons. Contextual evidence for measuring algorithmic progress; it does not identify the Williams’ Law index A.
- **CLM-012 · interpretation · mixed:** Primary research documents many bounded architecture, training, compression, and inference-policy efficiency shifts, but the collection does not by itself establish a universal exponential law of algorithmic innovation. The relation is thematic and mechanism-diverse; causal attribution and commensurability require narrower designs.

## Foundational papers

- [Defining Williams' Law: The Power of Algorithmic Innovation](https://doi.org/10.5281/zenodo.14946951): Performance gains come from disciplined algorithmic innovation, not only larger models.
- [Think Smarter, Not Harder: Algorithmic Innovation as the Key to Exponential AI Performance](https://doi.org/10.5281/zenodo.14957577): Practical pathways to increase AI efficiency and performance through structured design choices.

## Related works: Williams’ Law program

- **REL-001 · contextualizes:** [Defining Williams’ Law: The Power of Algorithmic Innovation](https://zenodo.org/records/14946951) — Junior Williams (2025). Foundational Williams-authored source; establishes the proposed framework but is not independent evidence.
- **REL-002 · contextualizes:** [Think Smarter, Not Harder: Algorithmic Innovation as the Key to Exponential AI Performance](https://zenodo.org/records/14957577) — Junior Williams (2025). Companion conceptual source; part of the program rather than independent validation.

## Related works: Scaling and efficiency

- **REL-003 · contextualizes:** [Cramming More Components onto Integrated Circuits](https://ieeexplore.ieee.org/document/658762) — Gordon E. Moore (1965). Historical hardware-scaling comparator; it does not define AI performance or isolate algorithmic effects.
- **REL-004 · complicates:** [Scaling Laws for Neural Language Models](https://arxiv.org/abs/2001.08361) — Jared Kaplan et al. (2020). Empirical power laws across model size, data, and compute show that resource scaling and algorithm choices are intertwined.
- **REL-005 · complicates:** [Training Compute-Optimal Large Language Models](https://arxiv.org/abs/2203.15556) — Jordan Hoffmann et al. (2022). Compute-optimal allocation between parameters and data complicates simple hardware-versus-algorithm narratives.
- **REL-006 · supports:** [Measuring the Algorithmic Efficiency of Neural Networks](https://arxiv.org/abs/2005.04305) — Danny Hernandez and Tom B. Brown (2020). Documents fixed-performance compute reductions in selected domains; supports a bounded efficiency trend, not a universal law.
- **REL-007 · supports:** [Algorithmic Progress in Language Models](https://arxiv.org/abs/2403.05812) — Anson Ho et al. (2024). Measures algorithmic progress using compute-equivalent comparisons; relevant to operationalization but does not identify A.
- **REL-008 · contextualizes:** [Compute Trends Across Three Eras of Machine Learning](https://arxiv.org/abs/2202.05924) — Jaime Sevilla et al. (2022). Provides historical compute-growth context and cautions against attributing capability trends to one input.
- **REL-009 · challenges:** [Are Emergent Abilities of Large Language Models a Mirage?](https://arxiv.org/abs/2304.15004) — Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo (2023). Shows how metric choice can create apparent discontinuities, challenging overinterpretation of breakthrough narratives.

## Related works: Architectures and training

- **REL-010 · contextualizes:** [ImageNet Classification with Deep Convolutional Neural Networks](https://proceedings.neurips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks) — Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton (2012). Canonical architecture-and-training milestone used in the historical source corpus.
- **REL-011 · supports:** [Deep Residual Learning for Image Recognition](https://arxiv.org/abs/1512.03385) — Kaiming He et al. (2015). Residual connections provide a bounded example of architecture changing trainability and performance.
- **REL-012 · contextualizes:** [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://arxiv.org/abs/1409.1556) — Karen Simonyan and Andrew Zisserman (2014). Depth-focused architecture evidence helps trace how design choices interact with compute and data.
- **REL-013 · supports:** [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://arxiv.org/abs/1905.11946) — Mingxing Tan and Quoc V. Le (2019). Demonstrates a bounded architecture-scaling method with explicit efficiency comparisons.
- **REL-014 · supports:** [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://arxiv.org/abs/1704.04861) — Andrew G. Howard et al. (2017). Shows architecture-level resource tradeoffs under deployment constraints.
- **REL-015 · contextualizes:** [An Image Is Worth 16×16 Words: Transformers for Image Recognition at Scale](https://arxiv.org/abs/2010.11929) — Alexey Dosovitskiy et al. (2020). Architecture transfer across domains illustrates innovation, while dependence on pretraining scale complicates attribution.
- **REL-016 · complicates:** [A ConvNet for the 2020s](https://arxiv.org/abs/2201.03545) — Zhuang Liu et al. (2022). Modernized convolutional design shows that implementation recipes and architecture families co-evolve.
- **REL-017 · supports:** [Attention Is All You Need](https://arxiv.org/abs/1706.03762) — Ashish Vaswani et al. (2017). A central bounded example of an architectural idea changing parallelism and sequence modeling.
- **REL-018 · contextualizes:** [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/abs/1810.04805) — Jacob Devlin et al. (2018). Pretraining objectives and transfer are algorithmic contributions entangled with data and compute.
- **REL-019 · complicates:** [Language Models Are Few-Shot Learners](https://arxiv.org/abs/2005.14165) — Tom B. Brown et al. (2020). Few-shot capability at scale demonstrates interaction among architecture, training data, compute, and prompting.
- **REL-020 · supports:** [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://arxiv.org/abs/1701.06538) — Noam Shazeer et al. (2017). Conditional computation is a bounded example of increasing capacity without proportional per-example compute.
- **REL-021 · supports:** [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://arxiv.org/abs/2101.03961) — William Fedus, Barret Zoph, and Noam Shazeer (2021). Sparse routing provides an explicit capacity–compute tradeoff relevant to algorithmic efficiency.
- **REL-022 · supports:** [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://arxiv.org/abs/2112.06905) — Nan Du et al. (2021). Another bounded sparse-model comparison; does not isolate a universal innovation index.
- **REL-026 · contextualizes:** [Adam: A Method for Stochastic Optimization](https://arxiv.org/abs/1412.6980) — Diederik P. Kingma and Jimmy Ba (2014). Optimization methods are algorithmic, but their effects depend on tasks, schedules, and baselines.
- **REL-027 · contextualizes:** [Highly Accurate Protein Structure Prediction with AlphaFold](https://www.nature.com/articles/s41586-021-03819-2) — John Jumper et al. (2021). A major application breakthrough, but causal attribution spans architecture, data, training, and domain structure.
- **REL-028 · contextualizes:** [Denoising Diffusion Probabilistic Models](https://arxiv.org/abs/2006.11239) — Jonathan Ho, Ajay Jain, and Pieter Abbeel (2020). A generative-model innovation in the source corpus; it is an example rather than a law-level test.

## Related works: Systems and efficiency

- **REL-023 · supports:** [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://arxiv.org/abs/2205.14135) — Tri Dao et al. (2022). System-aware algorithm design changes realized speed and memory without changing model semantics.
- **REL-024 · supports:** [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685) — Edward J. Hu et al. (2021). Provides bounded parameter and memory efficiency gains for adaptation.
- **REL-025 · supports:** [Distilling the Knowledge in a Neural Network](https://arxiv.org/abs/1503.02531) — Geoffrey Hinton, Oriol Vinyals, and Jeff Dean (2015). Knowledge distillation is a bounded example of transferring capability into a cheaper model.

## Related works: Reasoning and inference

- **REL-029 · supports:** [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://arxiv.org/abs/2201.11903) — Jason Wei et al. (2022). Establishes the primary reasoning-representation comparator in the Stage 1 protocol.
- **REL-030 · supports:** [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://arxiv.org/abs/2203.11171) — Xuezhi Wang et al. (2022). Motivates repeated sampling and aggregation while making resource accounting essential.
- **REL-031 · supports:** [Let’s Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs](https://aclanthology.org/2023.emnlp-main.761/) — Pranjal Aggarwal et al. (2023). Connects adaptive sampling to efficiency and helps frame secondary policy analyses.
- **REL-032 · contextualizes:** [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://arxiv.org/abs/2305.10601) — Shunyu Yao et al. (2023). Expands inference computation through search; illustrates capability–resource tradeoffs.
- **REL-033 · contextualizes:** [ReAct: Synergizing Reasoning and Acting in Language Models](https://arxiv.org/abs/2210.03629) — Shunyu Yao et al. (2022). Shows that inference policy can alter task behavior, though tool use adds new resources and confounds.
- **REL-034 · supports:** [Chain of Draft: Thinking Faster by Writing Less](https://arxiv.org/abs/2502.18600) — Silei Xu et al. (2025). Defines the concise reasoning representation tested prospectively in Stage 1.
- **REL-035 · contextualizes:** [Training Verifiers to Solve Math Word Problems](https://arxiv.org/abs/2110.14168) — Karl Cobbe et al. (2021). Introduces GSM8K, the primary benchmark specified by the Stage 1 protocol.
- **REL-036 · contextualizes:** [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://arxiv.org/abs/2205.10625) — Denny Zhou et al. (2022). A decomposition policy illustrating how prompt structure can change task performance.
- **REL-037 · complicates:** [PAL: Program-Aided Language Models](https://arxiv.org/abs/2211.10435) — Luyu Gao et al. (2022). Offloading computation to an interpreter complicates token-only resource comparisons.
- **REL-038 · complicates:** [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://arxiv.org/abs/2211.12588) — Wenhu Chen et al. (2022). Separates reasoning representation from execution and highlights construct-boundary choices.
- **REL-039 · contextualizes:** [Reflexion: Language Agents with Verbal Reinforcement Learning](https://arxiv.org/abs/2303.11366) — Noah Shinn et al. (2023). Iterative feedback is another inference-time policy with distinct resource and state requirements.
- **REL-040 · contextualizes:** [STaR: Bootstrapping Reasoning with Reasoning](https://arxiv.org/abs/2203.14465) — Eric Zelikman et al. (2022). Connects generated rationales to training, underscoring the difference between training-time and inference-time innovation.
- **REL-041 · supports:** [Scaling LLM Test-Time Compute Optimally Can Be More Effective Than Scaling Model Parameters](https://arxiv.org/abs/2408.03314) — Charlie Snell et al. (2024). Directly motivates resource-aware comparison of inference policies, without validating Williams’ Law.
- **REL-042 · supports:** [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://arxiv.org/abs/2407.21787) — Bradley Brown et al. (2024). Provides broad repeated-sampling evidence relevant to the Stage 1 reinvestment question.

## Related works: Methods and reproducibility

- **REL-043 · contextualizes:** [Bioequivalence Trials, Intersection–Union Tests and Equivalence Confidence Sets](https://projecteuclid.org/euclid.ss/1032280304) — Roger L. Berger and Jason C. Hsu (1996). Methodological basis for intersection–union logic used in compound Stage 1 decisions.
- **REL-044 · contextualizes:** [A Simple Sequentially Rejective Multiple Test Procedure](https://www.jstor.org/stable/4615733) — Sture Holm (1979). Controls familywise error across the Stage 1 co-primary decisions.
- **REL-045 · contextualizes:** [Equivalence Test and Confidence Interval for the Difference in Proportions for the Paired-Sample Design](https://onlinelibrary.wiley.com/doi/10.1002/%28SICI%291097-0258%2819980430%2917%3A8%3C891%3A%3AAID-SIM780%3E3.0.CO%3B2-B) — Toshiro Tango (1998). Paired-proportion noninferiority and equivalence methodology relevant to accuracy contrasts.
- **REL-046 · contextualizes:** [Recommended Tests and Confidence Intervals for Paired Binomial Proportions](https://doi.org/10.1002/sim.6148) — Morten W. Fagerland, Stian Lydersen, and Petter Laake (2014). Supports selection and interpretation of paired-binomial procedures.
- **REL-047 · contextualizes:** [Permutation P-values Should Never Be Zero](https://doi.org/10.2202/1544-6115.1585) — Belinda Phipson and Gordon K. Smyth (2010). Relevant to finite resampling inference and transparent p-value calculation.
- **REL-048 · supports:** [Improving Reproducibility in Machine Learning Research](https://jmlr.org/papers/v22/20-303.html) — Joelle Pineau et al. (2021). Supports explicit artifacts, checklists, and execution provenance in the empirical program.
- **REL-049 · contextualizes:** [Bootstrap Methods and Their Application](https://www.cambridge.org/core/books/bootstrap-methods-and-their-application/ED2FD043579F27952363566DC09CBD6A) — A. C. Davison and D. V. Hinkley (1997). General bootstrap foundation for resource-ratio uncertainty procedures.
- **REL-050 · contextualizes:** [An Introduction to the Bootstrap](https://www.routledge.com/An-Introduction-to-the-Bootstrap/Efron-Tibshirani/p/book/9780412042317) — Bradley Efron and Robert J. Tibshirani (1993). General resampling reference for the Stage 1 statistical design.
- **REL-051 · contextualizes:** [What Teachers Should Know about the Bootstrap](https://doi.org/10.1080/00031305.2015.1089789) — Tim C. Hesterberg (2015). Practical guidance on null-centered resampling and bootstrap interpretation.
- **REL-052 · supports:** [OSF Registrations and Preregistrations](https://help.osf.io/article/330-welcome-to-registrations) — Center for Open Science (2026). Official registration guidance for the immutable prospective record required before confirmation.