Williams’ Law research program
A research program examining whether cumulative algorithmic innovation can compound AI performance beyond hardware scaling alone.
- Author: Junior Williams
- Status: Proposed framework; empirical validation is not established
- Evidence register updated: 2026-08-30
Proposed formulation
P(H, A) = P(H0, 0)(H/H0)^alpha exp(lambda A)
H is hardware capacity. A is a latent innovation index. Alpha and lambda are model parameters.
Current evidence boundary
The Stage 1 protocol is prospective, reports no confirmatory results, does not estimate A, and does not validate Williams’ Law. Williams-authored examples and papers establish the proposed framework; they are not independent validation.
Governed propositions
- CLM-001 · theory · unverified: Williams’ Law proposes that cumulative algorithmic innovation can compound AI performance beyond hardware scaling alone. Accurate as a statement of the proposed framework. Independent efficiency trends motivate the research question but do not validate a universal law.
- CLM-002 · theory · verified: The source formulation is P(H, A) = P(H0, 0)(H/H0)^alpha exp(lambda A). Verified as source attribution; verification does not establish the empirical truth of the equation.
- CLM-003 · interpretation · verified: Historical examples in the foundational papers are illustrative source examples rather than independent validation of Williams’ Law.
- CLM-004 · theory · verified: The algorithmic innovation index A remains a latent conceptual construct in the foundational framework. The Stage 1 protocol explicitly states that it does not estimate A.
- CLM-005 · protocol · verified: The Stage 1 protocol is prospective: it reports no confirmatory model-response dataset, completed confirmatory analysis, empirical result, or verified preregistration.
- CLM-006 · protocol · verified: The Stage 1 protocol prospectively compares composed inference policies using paired accuracy and resource estimands under bounded hosted-inference conditions. This describes the planned design, not an executed experiment.
- CLM-007 · interpretation · verified: A positive Stage 1 result would support only a bounded claim about the tested policies, models, prompts, datasets, endpoints, and execution period; it would not validate the latent index A or the universal law. Directly follows the protocol’s prospective supersession and interpretation boundaries.
- CLM-010 · historical-trend · verified: Hernandez and Brown documented substantial historical reductions in compute needed to reach fixed performance thresholds in selected neural-network domains. Supports a bounded algorithmic-efficiency trend, not the exponential form of Williams’ Law.
- CLM-011 · historical-trend · verified: Ho and colleagues estimate algorithmic progress in language models across multiple evaluations and compute-equivalent comparisons. Contextual evidence for measuring algorithmic progress; it does not identify the Williams’ Law index A.
- CLM-012 · interpretation · mixed: Primary research documents many bounded architecture, training, compression, and inference-policy efficiency shifts, but the collection does not by itself establish a universal exponential law of algorithmic innovation. The relation is thematic and mechanism-diverse; causal attribution and commensurability require narrower designs.
Foundational papers
- Defining Williams' Law: The Power of Algorithmic Innovation: Performance gains come from disciplined algorithmic innovation, not only larger models.
- Think Smarter, Not Harder: Algorithmic Innovation as the Key to Exponential AI Performance: This preprint examines algorithmic efficiency through Chain of Draft prompting and a mathematical framework relating AI performance to hardware capacity and algorithmic innovation.
Related works: Foundational Williams’ Law papers
- REL-001 · contextualizes: Defining Williams’ Law: The Power of Algorithmic Innovation — Junior Williams (2025). Foundational Williams-authored source; establishes the proposed framework but is not independent evidence.
- REL-002 · contextualizes: Think Smarter, Not Harder: Algorithmic Innovation as the Key to Exponential AI Performance — Junior Williams (2025). Companion conceptual source; part of the program rather than independent validation.
Related works: Scaling and efficiency
- REL-003 · contextualizes: Cramming More Components onto Integrated Circuits — Gordon E. Moore (1965). Historical hardware-scaling comparator; it does not define AI performance or isolate algorithmic effects.
- REL-004 · complicates: Scaling Laws for Neural Language Models — Jared Kaplan et al. (2020). Empirical power laws across model size, data, and compute show that resource scaling and algorithm choices are intertwined.
- REL-005 · complicates: Training Compute-Optimal Large Language Models — Jordan Hoffmann et al. (2022). Compute-optimal allocation between parameters and data complicates simple hardware-versus-algorithm narratives.
- REL-006 · supports: Measuring the Algorithmic Efficiency of Neural Networks — Danny Hernandez and Tom B. Brown (2020). Documents fixed-performance compute reductions in selected domains; supports a bounded efficiency trend, not a universal law.
- REL-007 · supports: Algorithmic Progress in Language Models — Anson Ho et al. (2024). Measures algorithmic progress using compute-equivalent comparisons; relevant to operationalization but does not identify A.
- REL-008 · contextualizes: Compute Trends Across Three Eras of Machine Learning — Jaime Sevilla et al. (2022). Provides historical compute-growth context and cautions against attributing capability trends to one input.
- REL-009 · challenges: Are Emergent Abilities of Large Language Models a Mirage? — Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo (2023). Shows how metric choice can create apparent discontinuities, challenging overinterpretation of breakthrough narratives.
Related works: Architectures and training
- REL-010 · contextualizes: ImageNet Classification with Deep Convolutional Neural Networks — Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton (2012). Canonical architecture-and-training milestone used in the historical source corpus.
- REL-011 · supports: Deep Residual Learning for Image Recognition — Kaiming He et al. (2015). Residual connections provide a bounded example of architecture changing trainability and performance.
- REL-012 · contextualizes: Very Deep Convolutional Networks for Large-Scale Image Recognition — Karen Simonyan and Andrew Zisserman (2014). Depth-focused architecture evidence helps trace how design choices interact with compute and data.
- REL-013 · supports: EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks — Mingxing Tan and Quoc V. Le (2019). Demonstrates a bounded architecture-scaling method with explicit efficiency comparisons.
- REL-014 · supports: MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications — Andrew G. Howard et al. (2017). Shows architecture-level resource tradeoffs under deployment constraints.
- REL-015 · contextualizes: An Image Is Worth 16×16 Words: Transformers for Image Recognition at Scale — Alexey Dosovitskiy et al. (2020). Architecture transfer across domains illustrates innovation, while dependence on pretraining scale complicates attribution.
- REL-016 · complicates: A ConvNet for the 2020s — Zhuang Liu et al. (2022). Modernized convolutional design shows that implementation recipes and architecture families co-evolve.
- REL-017 · supports: Attention Is All You Need — Ashish Vaswani et al. (2017). A central bounded example of an architectural idea changing parallelism and sequence modeling.
- REL-018 · contextualizes: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding — Jacob Devlin et al. (2018). Pretraining objectives and transfer are algorithmic contributions entangled with data and compute.
- REL-019 · complicates: Language Models Are Few-Shot Learners — Tom B. Brown et al. (2020). Few-shot capability at scale demonstrates interaction among architecture, training data, compute, and prompting.
- REL-020 · supports: Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer — Noam Shazeer et al. (2017). Conditional computation is a bounded example of increasing capacity without proportional per-example compute.
- REL-021 · supports: Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity — William Fedus, Barret Zoph, and Noam Shazeer (2021). Sparse routing provides an explicit capacity–compute tradeoff relevant to algorithmic efficiency.
- REL-022 · supports: GLaM: Efficient Scaling of Language Models with Mixture-of-Experts — Nan Du et al. (2021). Another bounded sparse-model comparison; does not isolate a universal innovation index.
- REL-026 · contextualizes: Adam: A Method for Stochastic Optimization — Diederik P. Kingma and Jimmy Ba (2014). Optimization methods are algorithmic, but their effects depend on tasks, schedules, and baselines.
- REL-027 · contextualizes: Highly Accurate Protein Structure Prediction with AlphaFold — John Jumper et al. (2021). A major application breakthrough, but causal attribution spans architecture, data, training, and domain structure.
- REL-028 · contextualizes: Denoising Diffusion Probabilistic Models — Jonathan Ho, Ajay Jain, and Pieter Abbeel (2020). A generative-model innovation in the source corpus; it is an example rather than a law-level test.
Related works: Systems and efficiency
- REL-023 · supports: FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness — Tri Dao et al. (2022). System-aware algorithm design changes realized speed and memory without changing model semantics.
- REL-024 · supports: LoRA: Low-Rank Adaptation of Large Language Models — Edward J. Hu et al. (2021). Provides bounded parameter and memory efficiency gains for adaptation.
- REL-025 · supports: Distilling the Knowledge in a Neural Network — Geoffrey Hinton, Oriol Vinyals, and Jeff Dean (2015). Knowledge distillation is a bounded example of transferring capability into a cheaper model.
Related works: Reasoning and inference
- REL-029 · supports: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models — Jason Wei et al. (2022). Establishes the primary reasoning-representation comparator in the Stage 1 protocol.
- REL-030 · supports: Self-Consistency Improves Chain of Thought Reasoning in Language Models — Xuezhi Wang et al. (2022). Motivates repeated sampling and aggregation while making resource accounting essential.
- REL-031 · supports: Let’s Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs — Pranjal Aggarwal et al. (2023). Connects adaptive sampling to efficiency and helps frame secondary policy analyses.
- REL-032 · contextualizes: Tree of Thoughts: Deliberate Problem Solving with Large Language Models — Shunyu Yao et al. (2023). Expands inference computation through search; illustrates capability–resource tradeoffs.
- REL-033 · contextualizes: ReAct: Synergizing Reasoning and Acting in Language Models — Shunyu Yao et al. (2022). Shows that inference policy can alter task behavior, though tool use adds new resources and confounds.
- REL-034 · supports: Chain of Draft: Thinking Faster by Writing Less — Silei Xu et al. (2025). Defines the concise reasoning representation tested prospectively in Stage 1.
- REL-035 · contextualizes: Training Verifiers to Solve Math Word Problems — Karl Cobbe et al. (2021). Introduces GSM8K, the primary benchmark specified by the Stage 1 protocol.
- REL-036 · contextualizes: Least-to-Most Prompting Enables Complex Reasoning in Large Language Models — Denny Zhou et al. (2022). A decomposition policy illustrating how prompt structure can change task performance.
- REL-037 · complicates: PAL: Program-Aided Language Models — Luyu Gao et al. (2022). Offloading computation to an interpreter complicates token-only resource comparisons.
- REL-038 · complicates: Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks — Wenhu Chen et al. (2022). Separates reasoning representation from execution and highlights construct-boundary choices.
- REL-039 · contextualizes: Reflexion: Language Agents with Verbal Reinforcement Learning — Noah Shinn et al. (2023). Iterative feedback is another inference-time policy with distinct resource and state requirements.
- REL-040 · contextualizes: STaR: Bootstrapping Reasoning with Reasoning — Eric Zelikman et al. (2022). Connects generated rationales to training, underscoring the difference between training-time and inference-time innovation.
- REL-041 · supports: Scaling LLM Test-Time Compute Optimally Can Be More Effective Than Scaling Model Parameters — Charlie Snell et al. (2024). Directly motivates resource-aware comparison of inference policies, without validating Williams’ Law.
- REL-042 · supports: Large Language Monkeys: Scaling Inference Compute with Repeated Sampling — Bradley Brown et al. (2024). Provides broad repeated-sampling evidence relevant to the Stage 1 reinvestment question.
Related works: Methods and reproducibility
- REL-043 · contextualizes: Bioequivalence Trials, Intersection–Union Tests and Equivalence Confidence Sets — Roger L. Berger and Jason C. Hsu (1996). Methodological basis for intersection–union logic used in compound Stage 1 decisions.
- REL-044 · contextualizes: A Simple Sequentially Rejective Multiple Test Procedure — Sture Holm (1979). Controls familywise error across the Stage 1 co-primary decisions.
- REL-045 · contextualizes: Equivalence Test and Confidence Interval for the Difference in Proportions for the Paired-Sample Design — Toshiro Tango (1998). Paired-proportion noninferiority and equivalence methodology relevant to accuracy contrasts.
- REL-046 · contextualizes: Recommended Tests and Confidence Intervals for Paired Binomial Proportions — Morten W. Fagerland, Stian Lydersen, and Petter Laake (2014). Supports selection and interpretation of paired-binomial procedures.
- REL-047 · contextualizes: Permutation P-values Should Never Be Zero — Belinda Phipson and Gordon K. Smyth (2010). Relevant to finite resampling inference and transparent p-value calculation.
- REL-048 · supports: Improving Reproducibility in Machine Learning Research — Joelle Pineau et al. (2021). Supports explicit artifacts, checklists, and execution provenance in the empirical program.
- REL-049 · contextualizes: Bootstrap Methods and Their Application — A. C. Davison and D. V. Hinkley (1997). General bootstrap foundation for resource-ratio uncertainty procedures.
- REL-050 · contextualizes: An Introduction to the Bootstrap — Bradley Efron and Robert J. Tibshirani (1993). General resampling reference for the Stage 1 statistical design.
- REL-051 · contextualizes: What Teachers Should Know about the Bootstrap — Tim C. Hesterberg (2015). Practical guidance on null-centered resampling and bootstrap interpretation.
- REL-052 · supports: OSF Registrations and Preregistrations — Center for Open Science (2026). Official registration guidance for the immutable prospective record required before confirmation.