The Open-Source AI Safety Lab Nobody Is Talking About

EleutherAI is a nonprofit AI safety research organisation that has published work on model interpretability, red-teaming methodology, and training data transparency that has influenced practice at every major AI laboratory. Most people in the technology industry have not heard of it.

The organisation was founded in 2020 by a group of researchers who were frustrated that frontier AI research was happening behind closed doors at well-funded labs while the broader scientific community had limited ability to study, critique, or replicate it. Their response was to build the infrastructure for open AI research: open training datasets, open model weights, open evaluation frameworks, and published research under open access terms.

Their dataset, The Pile, was used to train many of the early open-source language models. Their evaluation framework, lm-evaluation-harness, is the standard tool for language model evaluation across academia. Their interpretability research has been cited in papers from DeepMind, Anthropic, and MIT.

The funding picture is less encouraging than the research record. EleutherAI operates on a budget that is a small fraction of what major AI laboratories spend on a single training run. The gap between their research impact and their resource base is not sustainable indefinitely.

“The scientific community needs independent actors who are not beholden to commercial interests to study these systems critically,” says one of the organisation’s researchers. “Right now, we exist. Whether we continue to exist depends on whether people who care about that decide to fund it.”