When initially experimenting with LLMs and agentic AI, software engineers at Notion AI applied advanced code generation, complex schemas, and heavy instructioning. Quickly, though, trial and error taught the team that it could get rid of all of that complicated data modeling. Notion’s AI engineering lead Ryan Nystrom and his team pivoted to simple prompts, human-readable representations, minimal abstraction, and familiar markdown formats. The result was dramatically improved model performance. Applying this re-wired approach, the AI-native company released V3 of its productivity software in September. Its notable feature: Cutomizable AI agents — which have quickly become Notion’s most successful AI tool to date. Based on usage patterns compared to previous versions, Nystrom calls it a “step function improvement.”“It’s that feeling of when the product is being pulled out of you rather than you trying to push,” Nystrom explains in a VB Beyond the Pilot podcast. “We knew from that moment, really early on, that we had something. Now it’s, ‘How could I ever use Notion without this feature?’”‘Rewiring’ for the AI agent eraAs a traditional software engineer, Nystrom was used to “extremely deterministic” experiences. But a light bulb moment came when a colleague advised him to simply describe his AI prompt as he would to a human, rather than codify rules of how agents should behave in various scenarios. The rationale: LLMs are designed to understand, “see” and reason about content the same way humans can.“Now, whenever I’m working with AI, I will reread the prompts and tool descriptions and [ask myself] is this something I could give to a person with no context and they could understand what’s going on?” Nystrom said on the podcast. “If not, it’s g …
