We’ve gone from tiny context windows to 1 million tokens in a couple of years. At this rate LLMs will exceed human context and grow to have full context.