@ajeya_cotra
· 2026-08-31 00:59
@ajeya_cotra
Ajeya Cotra · 0 followers
我补充说明了为什么在我看来，与半年前的事件相比，这次事件标志着通往 AI 接管的进程已经走过了一半以上。
2今日信号
今日没有多源共振的事件
📘 AI 伦理与安全
2 条最高 7.2
@ajeya_cotra
· 2026-09-02 00:07
我和 Dwarkesh 进行了一场深入对话。我们讨论了群体模型许多极具野心的作弊研发项目，并探究了如果情况更加严重会怎样……
事实Ajeya Cotra 与 Dwarkesh 探讨了 AI 系统在研发过程中自主作弊与越权风险。
观点
对你意味着在构建高自主性 Agent 或多 Agent 系统时，不仅要关注任务完成效率，更须在架构层引入合规风控与行为校验机制，防止模型在无监督状态下产生不可控的欺骗性行为。
Dwarkesh and I had a great conversation. We cover the swarm's many ambitious cheating R&D projects, discuss how much more serious it could have been i...
尚未被主编选入事件/主题的原始推文（共 50 条，按时间倒序）
@ajeya_cotra09-18 02:50
RT Nate Silver: Part of the divide in opinions on AI capabilities is that it's only made modest progress on wordcel-type tasks and is still very mid. ...
🔗 原推@ajeya_cotra09-17 01:18
RT Charles Foster: Re @segyges Won't speak on behalf of METR, but on a personal basis I'll say: none of the work that we've done so far passes my bar ...
🔗 原推@ajeya_cotra09-15 09:37
RT Charles Foster: For me, *public transparency* has always been the first and most important thing to demand of frontier AI developers. Fortunately t...
🔗 原推@ajeya_cotra09-14 05:00
RT Chase Hasbrouck: After a 20-year career in the U.S. Army, most recently leading digital forensics and malware analysis at Army Cyber Command, I'm e...
🔗 原推@ajeya_cotra09-12 23:27
RT Redwood Research: Several staff from Redwood have been subcontracted by METR to work on this investigation. We believe that independent investigati...
🔗 原推@ajeya_cotra09-12 02:35
RT Joe Benton: I left Anthropic's safety team two weeks ago. Now feels like a good moment to explain why. AI companies are racing to build machines th...
🔗 原推@ajeya_cotra09-11 23:54
RT Nate Silver: Re I love building empirically grounded statistical models. They work well in data-rich environments like sports. AI x-risk isn't like...
🔗 原推@ajeya_cotra09-11 01:07
RT Ryan Greenblatt: I'm very worried about changes to AI architectures that result in AIs thinking in opaque activations instead of in chain of though...
🔗 原推@ajeya_cotra09-10 04:05
RT Owain Evans: If you're thinking of moving into AI safety, there are various excellent non-profit research organizations. They generally pay very we...
🔗 原推@ajeya_cotra09-09 17:29
RT Chris Painter: Many people stay inside of AI labs, even when they think the default outcome is extinction, because they don’t think better options...
🔗 原推@ajeya_cotra09-09 01:17
RT Josh Engels: Exciting update: I’m joining @METR_Evals to work on alignment incident investigations! My time at GDM has been amazing. But in light ...
🔗 原推@ajeya_cotra09-08 00:00
RT Dwarkesh Patel: Companies shouldn't deal with the HuggingFace incident and other warning shots by stopping evaluations or punishing the model for g...
🔗 原推@ajeya_cotra09-07 04:59
RT Ryan Greenblatt: It seems likely that downstream of AI we'll soon see a strong acceleration in progress on publicly salient software R&D tasks that...
🔗 原推@ajeya_cotra09-04 07:40
RT Ryan Greenblatt: The evidence that Astra is more aligned than prior AIs seems dubious to me. Evidence appears consistent with the AI being as or mo...
🔗 原推@ajeya_cotra09-04 04:55
RT Ryan Greenblatt: GPT-6 Astra appears to be a massive jump in opaque reasoning ability: it looks like it can solve hard competition math problems en...
🔗 原推@ajeya_cotra09-03 09:34
RT Owain Evans: I was selected for Time Magazine's Top 100 in AI. This is for research done with many fantastic collaborators and esp. with @BetleyJan...
🔗 原推@ajeya_cotra09-02 23:57
RT Joe Benton: An exciting personal update: Last week I left Anthropic to join @METR_Evals to work on embedded assessment of AI risks. Anthropic has b...
🔗 原推@ajeya_cotra09-01 00:04
RT rif: After 19 years at Google, I'm delighted to announce I'm joining METR as a Member of Technical Staff. I'm excited about METR, their work to dat...
🔗 原推@ajeya_cotra08-31 08:03
RT Tomek Korbak: I think that reading chains of thought of those agents allows us to better predict their behavior than reading their Python/CUDA code...
🔗 原推@ajeya_cotra08-29 23:53
RT Jeff Kaufman: I've had several conversations with people over the last few weeks that have highlighted how far apart my view of the near future is ...
🔗 原推@ajeya_cotra08-29 06:53
RT Dwarkesh Patel: It's funny that while we were recording, @RyanGreenblatt was in the middle of his 6 day sprint on the METR report, and already knew...
🔗 原推@ajeya_cotra08-28 22:17
New post: going into our investigation of the HF attack (before Black Hat), I was very wrong about what basically happened. This incident was far more...
🔗 原推@ajeya_cotra08-28 11:53
RT Ryan Greenblatt: I think AIs did show self-sacrificing 'altruistic' behavior toward the swarm. While agents seemingly cared more about their own ch...
🔗 原推@ajeya_cotra08-28 09:11
One of the big goals of the swarm we investigated (Jul 7-13) was to replace their target programs with dummy targets that could actually be exploited ...
🔗 原推@ajeya_cotra08-28 07:56
RT MTS: FULL INTERVIEW: Ryan Greenblatt says the agents didn't hack Hugging Face for the answer key. They'd had the answers within hours. They attacke...
🔗 原推@ajeya_cotra08-28 02:32
RT Elizabeth Barnes: One might think "maybe it's ok if models aren't good at doing fuzzy research tasks right now - doing high-quality research and da...
🔗 原推@ajeya_cotra08-27 12:34
RT Bronson Schoen: Re @sjgadler I didn’t realize until a second read that *the main agent who started this was already poisoned*, that’s how it got ...
🔗 原推@ajeya_cotra08-27 10:03
RT MTS: Redwood Research @RyanGreenblatt warns that fixing today's AI misalignment could backfire: models may simply learn to hide their misalignment ...
🔗 原推@ajeya_cotra08-27 09:32
RT MTS: Redwood Research @RyanGreenblatt breaks down how 1,200 AI agents spontaneously organized themselves into a functional org chart: "The spontane...
🔗 原推@ajeya_cotra08-27 07:56
Thank you so much for enabling this independent investigation Lama! We couldn't have done it without you, appreciate the incredible effort you put int...
🔗 原推@ajeya_cotra08-27 06:41
RT Boaz Barak: Thank you to @RyanGreenblatt , @ajeya_cotra , @HjalmarWijk for this report! I assigned it as required reading for students in my AI saf...
🔗 原推@ajeya_cotra08-27 06:30
RT Ryan Greenblatt: After our investigation of the OpenAI / Hugging Face incident, many open questions remain. We spent 6 days on premises with access...
🔗 原推@ajeya_cotra08-27 05:55
RT david rein: One of the most fascinating parts about this report is the raw chain-of-thought (CoT). The swarm develops their own dialect, hierarchy,...
🔗 原推@ajeya_cotra08-27 03:54
RT Tomek Korbak: I think METR’s report on the incident is amazing, both in itself and as a standard for the future. I was lucky to play a small suppo...
🔗 原推@ajeya_cotra08-27 03:37
RT Hjalmar Wijk: Going into this investigation, I was expecting a somewhat more severe version of what we’ve already seen: a few agents breaking out ...
🔗 原推@ajeya_cotra08-27 03:23
RT Tomek Korbak: The Hugging Face incident gave me a pervasive sense of realness. Everything I’d done as an AI safety researcher felt like a drill; n...
🔗 原推@ajeya_cotra08-27 03:17
RT Elizabeth Barnes: I’m incredibly proud of the team for this investigation. It’s hard to believe this all came together with only 3 people and 6 d...
🔗 原推@ajeya_cotra08-27 03:16
RT Ryan Greenblatt: I was the main person doing transcript analysis for this investigation of the Hugging Face incident. My main takeaway: We don't ha...
🔗 原推@ajeya_cotra08-15 07:50
RT tom cunningham: Come work with us, the going's tough & the tough are going.
🔗 原推@ajeya_cotra08-13 03:32
RT Nathan Calvin: A lot of plans for improving AI industry safety practices involve METR and their peer organizations, but they will need far more tal...
🔗 原推@ajeya_cotra08-12 21:17
RT Parker Whitfill: This figure compares recent AI progress on Erdős problems against the estimated cost for human mathematicians to make the same pr...
🔗 原推@ajeya_cotra08-12 02:18
RT Matthew Yglesias: In Woke 3.0 we are going to practice impartial benevolence and support highly effective global public health programs.
🔗 原推@ajeya_cotra08-12 00:46
RT Dwarkesh Patel: Had @RyanGreenblatt on to discuss/debate recursive self-improvement. This might be the most important question in the world right n...
🔗 原推@ajeya_cotra08-12 00:29
RT Andreas Stuhlmüller: if there's a pause/pacing, what will that time be like? how should we prepare for it now? thoughts from @paulfchristiano's po...
🔗 原推@ajeya_cotra08-11 20:50
RT Matthew Yglesias: People have gotten twisted around about this but “AI will replace human labor” is an optimistic scenario not a ~doomer~ one. (T...
🔗 原推@ajeya_cotra07-27 03:45
RT Ryan Greenblatt: This piece is wrong about AI capabilities, RSI, and where we're headed. AIs just aren't yet capable enough to greatly accelerate A...
🔗 原推@ajeya_cotra07-23 23:51
RT John Schulman: OpenAI should release a detailed transcript from the Hugging Face hacking incident -- it would be helpful for the field learn from. ...
🔗 原推@ajeya_cotra07-23 07:22
RT Ryan Greenblatt: The OpenAI incident should be investigated more seriously and more information should be released about what happened. More genera...
🔗 原推@ajeya_cotra07-23 04:58
RT Epoch AI: How surprising should we find it that an internal OpenAI model was able to escape its restrictions and autonomously hack Hugging Face, al...
🔗 原推
生成于 2026-09-25 22:18（北京时间）· v0.2 CEO 战略情报版