Summary：AI Language Models Predict Social Science Experiments, Leaving Experts Stunned **Introduction** Wh

referrerpolicy="no-referrer"
style="max-width:100%;height:auto;display:block;margin:0 auto;">

AI Language Models Predict Social Science Experiments, Leaving Experts Stunned

**Introduction**
When researchers at Stanford and MIT teamed up to test whether artificial intelligence could anticipate the outcomes of social‑science studies, they expected a modest bump in forecasting skill. What they found instead was a striking parity: large language models (LLMs) predicted experimental results almost as well as a panel of seasoned human forecasters, even for papers that appeared after the models’ training cutoff. The discovery has sparked both excitement and skepticism across academia, prompting a fresh look at how machine‑learning tools might reshape the way we design and interpret empirical work.

**Key Developments**
The study, published in *Nature Human Behaviour*, fed abstracts and methodology sections from 150 published experiments into three state‑of‑the‑art LLMs. The models were asked to estimate the direction and magnitude of treatment effects, producing probability distributions that were later compared to the actual reported outcomes. Across psychology, economics, and political science subsets, the LLMs’ mean absolute error hovered around 0.21 standard deviations—virtually indistinguishable from the 0.19 error of a group of 30 expert forecasters who had access to the full papers. Notably, the AI tended to overestimate effect sizes by roughly 10 %, a bias the authors attribute to the models’ propensity to favor statistically significant patterns present in their training corpora.

**Industry Analysis**
From a scholarly perspective, the result challenges the notion that human intuition about context, theory, and methodological nuance is irreplaceable. It suggests that LLMs have absorbed enough statistical regularities from vast text corpora to mimic the heuristic shortcuts experts use when guessing experimental outcomes. For research funders and journal editors, the finding raises practical questions: could AI‑assisted