AI Research Circle: Memory & Long-Context LLMs

We surveyed practical approaches to long-context LLMs including RoPE scaling, sparse attention, and state-space alternatives, then mapped the real engineering tradeoffs: when expanding context windows makes sense versus when retrieval-augmented generation still wins. The takeaway: accepting tokens isn't the same as using them well.