StoryAI is the next step up from AttentionAI. They work the same way — but
StoryAI is bigger and thinks in two rounds. Here is the key difference:
AttentionAI has one round. It reads your words once (attention), does a
little reasoning (feed-forward), and guesses the next word. That is enough for
short, simple sentences.
StoryAI has two rounds. It does the same attention-plus-reasoning
step, then does it all again. The first round gets a rough idea of what
the words mean together. The second round reads that idea and sharpens it into a
much better guess. Two rounds let it follow more complicated stories that
one round would trip over. That is why we call them Attention 1 and
Attention 2 in the picture — each round has its own attention step
followed by its own reasoning step.
Attention — in both rounds. In each round it matches every word's
query against the keys of all the words and decides how much to focus on each one. Those focus weights are
the glowing lines you see: the stronger the glow, the more that word matters for
what comes next. Each round uses single-head attention — one spotlight
instead of the many "heads" a real transformer uses.
Word order matters. StoryAI adds a position signal to every word —
a small set of numbers that says where the word sits in the sentence. That's how
it tells "the cat ate the fish" apart from "the fish ate the cat".
Why it can finish your story right away. StoryAI is pre-trained — it
has already read lots of little stories. With about 16,000 changeable
numbers (vs AttentionAI's 2,000), it invents new endings by combining
what it knows, predicting one word at a time until your story is finished.