I’m skeptical about the value of generative AI when it’s slapped hastily embedded into work productivity tools used for company knowledge bases (Notion, Google Docs, MS Word, Confluence, etc.).
Functional knowledge bases represent critical thinking. For example, at Status Hero, we host product design briefs in our Notion workspace. These briefs contain the foundational context and strategy to drive implementation decisions, large and small.
The heart of the matter is that writing is the act of thinking critically. In our case, the value of a design brief is rooted in the fact that Adam wrote it. He thought through the design problem, leveraged a long career of product experience, and injected his first-hand knowledge of our customers, their needs, and our overall strategy. The result is novel and bespoke. It would be detrimental to our customers and business (and Adam!) if Adam skipped the thinking step by tabbing through the “Ask AI to write …” prompt Notion has frustratingly added to every interaction.
Put another way, what value does a knowledge base have to your business if it consists of re-hashed material written by an LLM? After all, clients like chatGPT can serve up all that generated copy on demand.
On the other hand, I’m excited about the potential of LLMs for summarization and analysis, especially when applied to things humans have written while thinking critically, like the daily intentions or goal updates that Status Hero captures.
And so that’s precisely how we’re experimenting with LLMs in Status Hero beta releases. Here’s an overview I got today based on our team’s daily check-ins, sensitive parts redacted:
The important thing here is that the source material for the summary, written check-ins, contains the thought-out intentions of my teammates rather than just a laundry list of completed tasks. That’s because, for collaborative teams, the best time to spot issues and make course corrections is before the work starts. Generally, we believe that automated summarization is a fantastic vehicle for efficiently distributing intentions and providing a jump-off point for further inquiry.
On the analysis side, we’re developing custom prompts and NLP embedding techniques to identify themes and patterns to help teams and companies plan and work smarter. It’s a much harder problem because of the hallucination issues–this is an area where accuracy matters–but the results so far are promising.
Ultimately, LLM generation may be the way forward for a performative, no-need-to-reinvent-the-wheel copy at work, like legal documents or HR rules. But its hasty incorporation into work productivity tools can undermine critical thinking, especially within knowledge bases. Beware.
However, LLMs have plenty of potential in other work areas, such as summarization and analysis of human-generated content like goals, goal updates, daily intentions, and requests for help with blockers. That’s the path we’re exploring with Status Hero.
