I models can learn to read without a tokenizer.

It's called in-context tokenization: context changes how the model groups information into chunks.

And the fact that it can work for DNA prediction is mind-blowing!

@cartesia gave its new H-Net model a made-up word. At first, it broke it into little pieces. But the more it saw the word, the more of it it grouped together. The same thing happened with patterns and names that kept showing up in code.

Then Cartesia trained a separate H-Net on bacterial DNA, and showed it a human gene. And somehow, it picked up on the gene’s structure too. That's wild.

So, here's a question: Can we build systems that learn how to organize each kind of data for themselves? Let's see ↓

In this video, I also get into how H-Net works, how it compares to SmolLM3, and why it needed much less training data to reach similar results -> https://www.youtube.com/watch?v=Q9mGVb_T_Tk