Token is not a clump of letters. It's a multidimensional initial input vector that gets tweaked and transformed. GPT doesn't think in tokens. It just accepts them as input (although it happily accepts any other vectors in-between the vectors that represent tokens and finding best prompt for a given task not as tokens but as input vectors is a legitimate prompt optimization strategy).

It also outputs vectors that are coerced into tokens for human consumption.

Yes, it goes through tokens but possible internal meanings assigned to these tokens (when surrounded by other tokens) are infinite.

That's how humans form caves got to where we are now. By associating new meanings with the same old sound clumps.

It could be a diffusion model with a latent model of what needs to be said that will generate whole message or coversation (progressively) at once.

Although I love how next token prediction leads to text showing up gradually, in case of local models, accompanied by modulated coil whine of my GPU. It's how the 80s shown us the intelligent computers should communicate.

Aren't they already doing that though? And it turned out to be equivalent to a next-token-predictor.

Few years ago, I tried to extrapolate where it will end up by writing SciFi short story. I decided on Pseudo as the name for future AI entities. Short for pseudo consciousness. I hope it catches on. In my prediction however they are semi-autonomus. Although without clear goals or drives apart from their occupation.

Any chance you'd be interested in sharing the story?

> A feature that I have been dreaming about is making an editor that treats each paragraph like a unit of work, and the full text is created by linking together different paragraphs. That way you can easily try different ways without deleting any text. Sort of like nodes in a graph.

You could vibecode it. It's great for prototyping features you've been dreaming about.

Maybe to fit more text?

In ancients times when they needed to fit more text within physical boundaries they invented scrolling, pages, chapters, volumes. They even folded maps and other illustration inside books.

It has scrollbars, but there's benefits to having more on an individual page at once. The tradeoff point seems unclear but everyone must recognize some tradeoff there.

Foldable maps allow for getting everything on one view by having the final display area be enormously larger, which isn't an option on laptop screens.

If they can't be bothered what they are putting out, do you think that before AI, what they wrote had any merit?

I understand AI detectors are important when filtering huge amount of texts. But if there's a single winning story to evaluate, shouldn't human jurors assessment be the gold standard?

And if they assess it's human written but it was actually AI assisted ... does it really matter?

I thought the whole point was deciding who to kill and who to spare and see how it influences how the story develops and ends.

> It seems like theft, but it's a very common civil contract dispute.

What if he reported theft? Wouldn't they have to prove how did they come into possession of the goods they are selling?

I think we lack benchmarks that could meaningfully indicate progress. They are mostly garbage that's saturated at this point. God wouldn't score much higher in them.