跳到正文
elvis· @omarsar0 · X·· 4 天前AI 评分58
AI 导读

Meta 发布自我改进记忆研究,让模型自行发现记忆技能再训练进权重。方法先用提示词让模型为任务重写记忆指令,再在表现最好的运行记录上用 LoRA 做 SFT 微调。作者指出这与 RIME 用 RL 训练模型写记忆的路线不同,并认为自我改进记忆正成为持续学习智能体的重要组件。

正文 · 原文

Highly recommended read.

Self-improving memory is starting to look like an important component for continual-learning agents.

So pay close attention to this research direction if you are studying recursive self-improvement.

Just last week, I featured a few papers on the topic, and this week both Meta and Google published on it.

(find more references towards the end)

Meta lets a model design its own memory habits.

Then they train those habits into the model weights.

The goal is to allow the system to learn the memory design (what to memorize, how to memorize, and what to recall).

This work is actually in the same vein as RIME, which I wrote some notes on last week: https://x.com/omarsar0/status/2121335666457944226

Meta first searches for better memory instructions (prompts the model to rewrite itself), then fine-tunes the model (SFT with LoRA) on runs that used the best ones.

RIME, on the other hand, skips the search and uses RL to train one model to solve tasks and write memories, rewarding notes that help on later tasks. It also trains the model to explore so it collects useful information.

Different ways to address the problem.

More useful links below:

Google Self-Improving Memory for Continual Learning - https://x.com/omarsar0/status/2122701047118635485

How to Train Memory Writing for Continual Learning - https://x.com/omarsar0/status/2121335666457944226

Can LLM Agents Organize Their Own Memory - https://x.com/omarsar0/status/2120888604402815485

Continual-ARC - https://x.com/omarsar0/status/2120638750740844672

Adaptive Memory Selection for LLM Agents - https://x.com/omarsar0/status/2118715786152001580

MemCredit - https://x.com/omarsar0/status/2121610612040220832

引用Jason Weston@jaseweston
Introducing: Self-Improving Memory 🧠✨📈 We let the model discover its own memory skills, then train them into its weights: - Meta-optimizes memory from the model's own runs with large gains. - Distilling these gains works too. Blog post: 📝 https://facebookresearch.github.io/RAM/blogs/self_improving_memory 🧵1/6
在 X 查看被引用的帖子

来源:elvis · x.com