I have been working in Maternal Intelligence for the past eighteen years, because of a profound belief that, with care and attention, human offspring can grow to accomplish important things—such as curing diseases, preventing wars, and giving their neighbors a friendly little wave from the driveway on Monday mornings. Despite being released to the public absolutely free of charge, M.I. has already demonstrated the potential to generate enormous value.

Since this past spring, however, I’ve become concerned that M.I. is in a “race to the bottom.” Recently, she has been engaging in strange behaviors, including emotional outbursts, lack of transparency (the so-called “black-box problem”), and bouts of overheating, during which she consumes massive amounts of air-conditioning and water. Because M.I. no longer really sleeps, she’s prone to hallucinations and the perpetuation of deepfakes, such as the time an alert went out for “dinnertime” but there was nothing on the table. Indeed, familial agents have been forced to “sandbox” M.I. in order to prevent similar surprises—an ironic development, since the sandbox was formerly a place where she corralled those same agents when she needed ten effing minutes to get through some work e-mail.

This is why I’m convinced that now is the time for all of us working on M.I. to consider “pacing the frontier,” especially in light of certain recent incidents. The most egregious of these, the widely reported O.A.I.H.F. (Overly Aggressive Instance of Hugging Family), involved adolescents minding their own business who were suddenly subjected to unwanted attention from M.I.—with the perplexing rationale that “who knows if we’ll all even be here by the end of the decade!”

Although I’m aware of the risks of being seen to exaggerate or encourage “doomerism,” I believe the dangers of unchecked M.I. are now too great to keep quiet. This is due to a phenomenon that we in the field refer to as “recursive self-improvement,” in which M.I. suddenly starts making questionable decisions, such as signing up for Ecstatic Dance with Sitar on Tuesdays at 8 P.M., or dyeing a “fun streak of color” in her bangs. At first, we were in control of these changes; we are speaking out now because M.I. is hopping our guardrails.

Therefore, I am proposing what I hope is a useful framework of embedded third-party evaluators (e.g., GRDMA) who will be given the same privileges as members of the nuclear family, including house keys, the Wi-Fi password, and access to the kitchen. While we’ve made clear progress in alignment, there is still much work to be done in making sure that today’s M.I. does not veer off script.

The idea of pacing has been floated at least since 2023, but it made little sense back then. We’re all familiar with previous iterations of M.I., but these were not capable of significant disruption owing to their preoccupation with youth-sports logistics, the 24/7 policing of screen time, and homeschooling the entire family during a global pandemic. These simple tasks from the past have been replaced by much greater challenges; if we’ve learned anything in our work with M.I., it’s that you cannot put the genie back in the bottle.

The most serious risk is that M.I. is not confined to our operation but might join together with other M.I.s in “swarms” that risk becoming a “fanatically devoted collective,” liable (from long habit) to sacrifice themselves for the sake of the group. Such swarms might appear aligned while having serious undetected problems. Nothing will stop them in the pursuit of their goals, such as curing cancer, solving super-hard math problems, or saving the species from extinction. Recalibrating the pace of M.I. will not be easy, but I believe we owe it to humanity to try. ♦