解读 Pangram：AI 文本检测器到底可不可靠？

解读 Pangram

昨天 David Sacks 发了一条推文。几分钟之内，人们就做了他们一贯会做的事，去问 Pangram 这条是不是 AI 写的。Pangram 回答说完全是 AI 生成的。David 随即回了一句，说这些 AI 检测器都是骗人的。

Pangram 的误报率其实相当低。但只要你用过 LLM 当写作助手，多半会注意到，它会百分百断定你的帖子是 AI 写的，哪怕你自己并不这么觉得。Pangram 本身就是个训练好的模型，会把一段文本分成三类，确定是人类写的、确定是 AI 写的，或者两者掺在一起。想了解原理的话，他们发过一篇论文。

简单说就是，他们自己造训练数据。起点是一堆已知的人类写作文本。再让一个大语言模型（LLM）先去理解这些文本，再围绕同一主题重新写一段新的出来。他们也让 LLM 对原始人类文本做局部修改，由此捕捉「人机共写」里的细节。

Pangram 自称模型误判为 AI 的概率是 0.0041%，漏判 AI 文本的概率是 0.34%。

知道了这些之后，我想让 LLM 复刻一下 David 那条推文，应该挺好玩的。我先拟了个提示词。说是「我写的」，其实是让一个 LLM 根据那条推文，告诉我结构上可以怎么写。

我敢说，你去问 Pangram 上面这段文字是不是 AI，它大概率也会说是，但这不是重点。重点是，我后来用 Opus 5 生成了一段文字，读起来完全是 AI 的味道。好奇的话，下面是我用的提示词。

Original prompt to generate a David Sacks like tweet
Please write an 8 paragraph tweet about "Pacing the Frontier", not longer than 350 words.
Context of this tweet is this post by Dario: https://darioamodei.com/post/we-must-pace-the-frontier
And this tweet by Sam Altman:
"I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon."
Structure for the post:
- Open with a concession: Note that Dario ("We Must Pace the Frontier") and Sam Altman both want to slow down at the frontier.
- Say "go ahead" which reads as a surprising response.
- Establish they own the frontier: Argue OpenAI and Anthropic hold a duopoly on frontier intelligence by all kind of metrics (market share, capability, revenue etc.) and that they themselves claim the lead is widening via RSI.
- Concede you can't see their unreleased models, so if those are scary, being responsible is their call.
- Follow with a list of a rapid-fire paragraph ("stop pretending") rejecting the asks from Dario's post.
- Name the commercial self-interest (Slowing down isn't purely altruistic)
- Close with the challenge that pacing buys room for a smarter regulatory debate than Bernie Sanders "shut it all down".
- Say something about China and Open Weight models.

Opus 5 写出来的是这样。

Dario published "We Must Pace the Frontier." Sam agreed, and said OpenAI will also commit to independent evaluators with employee-like access. My response is not the one they expect: go ahead. Please do. Because you two are the frontier.

OpenAI and Anthropic hold a duopoly on frontier intelligence by every metric anyone actually uses: capability, revenue, enterprise share, developer mindshare. And you both keep telling us the lead is widening, that models are starting to improve models. If that's true, pacing costs you a quarter you can reclaim whenever you like.

I also can't see your unreleased checkpoints. If what's sitting behind the curtain genuinely scares the people who trained it, then slowing down is your call to make, and the right one. I'm not going to second-guess a risk assessment I have no access to.

But stop pretending the rest of us are the problem. Stop pretending this is a collective action failure in which a handful of startups are the defectors. Stop pretending open weights are the dangerous frontier while the two of you ship the most capable model in the world every few months. Stop pretending third-party evals require new law, when you both just committed to them voluntarily, in public, on a blog. And stop pretending it's purely altruism. Pacing the frontier is also margin repair. It slows the price war, stabilizes the capex story, stretches depreciation schedules across a longer product cycle, and makes life considerably harder for fast followers who survive by distilling whatever you shipped last month. Safety and self-interest are pointing the same direction here. That's worth saying out loud.

Still, the upside is real. Pacing buys room for a smarter regulatory debate than the one on offer, where the loudest alternative is Bernie Sanders' "shut it all down." That's a slogan, not policy. Meanwhile China keeps shipping competitive open weights, and any rule that binds two American labs while the rest of the world downloads its capability for free isn't safety, it's theater with a very expensive ticket price.

嗯，Pangram 也判它 100% 是 AI。

目前为止，也就如此，谈不上多有意思。读起来确实有点像 David 那条，但显然不是完全一样。原始提示词本来就给不出足够信息把那条推文完整还原，有点出入也正常。真正让我感兴趣的其实是，如果你能拿