跳到正文

技术方向

安全对齐 最新动态

让 AI 可控、可信、不被滥用:越狱与防御、模型行为研究、安全评测与治理框架。

86 条精选近 30 天 78 条共收录 431 条

更新

精选归档 · 第 5 页

9月13日周日第 81–86 条
  1. Alex Albert60

    Alex Albert 推荐 Dario Amodei 新文《We Must Pace the Frontier》,该文主张 AI 行业应放缓,并提出三步计划,Anthropic 单方面承诺先落实第一步,即让第三方评估员以员工级权限长期访问其系统,核查安全措施执行、上报事故并评估训练中的模型对齐。Alex Albert 认为这种嵌入评估员在科技界少见,但在银行业和核电站已有常驻监管先例,是务实的第一步。

    引用Dario Amodei@DarioAmodei

    We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training. You can read the full post here: https://darioamodei.com/post/we-must-pace-the-frontier

    推荐理由:借银行业与核电站的常驻监管先例,说明第三方评估员嵌入实验室为何是可落地的安全步骤。

9月2日周三
  1. Alex Albert60

    Anthropic 随 Fable 5.1 推出 Enterprise Frontier Safeguards(EFS),为企业客户提供与零数据保留同级的隐私保护,同时防范对抗性使用,今年秋季起分阶段上线。Alex Albert 称 EFS 是面向智能体时代的 ZDR++,公司数据留在自有云中,由自动监控层向团队标记风险模式,可视为智能体的可观测性与风险缓解层。

    引用Claude@claudeai

    We’re also introducing Enterprise Frontier Safeguards (EFS), which give enterprise customers complete privacy (the same as zero data retention), while still being state-of-the-art at preventing adversarial use. EFS rolls out in phases, starting this fall. https://www.anthropic.com/news/enterprise-frontier-safeguards

    推荐理由:Anthropic 员工解释 EFS 如何在零数据保留之上为智能体加一层监控,读者可借此理解企业采用 AI 时的隐私与风险权衡。

8月28日周五
8月14日周五
  1. Z.ai78

    智谱宣布正在为 GLM-5.3 的开放发布做准备,称这是其迄今网络安全能力最强的模型,将先由安全合作伙伴在受控环境中评估,随后扩大访问与 API 可用性,并在完成安全评估后公开完整模型权重。

    推荐理由:原文给出了 GLM-5.3 在三个安全基准上的对比数据和分阶段开放权重计划,读者可据此判断开源模型在攻防双用途上的能力边界。

3月31日周二
  1. Andrej Karpathy72

    Karpathy 转发安全警报称,npm 上最流行的 HTTP 客户端库 axios 遭遇新的供应链攻击,最新版本 axios@1.14.1 引入了一个当天才出现的包 plain-crypto-js@4.2.1。

    引用Feross@feross

    🚨 CRITICAL: Active supply chain attack on axios -- one of npm's most depended-on packages. The latest axios@1.14.1 now pulls in plain-crypto-js@4.2.1, a package that did not exist before today. This is a live compromise. This is textbook supply chain installer malware. axios has 100M+ weekly downloads. Every npm install pulling the latest version is potentially compromised right now. Socket AI analysis confirms this is malware. plain-crypto-js is an obfuscated dropper/loader that: • Deobfuscates embedded payloads and operational strings at runtime • Dynamically loads fs, os, and execSync to evade static analysis • Executes decoded shell commands • Stages and copies payload files into OS temp and Windows ProgramData directories • Deletes and renames artifacts post-execution to destroy forensic evidence If you use axios, pin your version immediately and audit your lockfiles. Do not upgrade.

    推荐理由:作者以自身系统排查为例,说明未锁定版本的依赖如何让一次投毒扩散到普通用户。