Original image from OpenAI, AI overlay from Emergent.

Jony Ive’s iPhone trained a generation to check a glowing rectangle 96 times a day. It supercharged the attention economy. And now, after a $6.5 billion acquisition in May of 2025 that brought Apple’s former hardware design leadership into OpenAI, Ive and team are building something designed to ask for less of your attention, not more. It is supposed to be a device that doesn’t sit in your pocket waiting to be pulled out, but perceives your environment and decides when to help. It engages you, not the other way around.

Nobody outside OpenAI has seen it yet. The first device is expected in the second half of 2026. But between official statements, court filings, and supply chain leaks, we know enough to understand what they’re attempting: a new category of computing that sits alongside your laptop and phone, what Sam Altman described to employees as a third core device, according to the Wall Street Journal.

Subscribe now

Here’s what we know, what it could be, and why it matters for anyone building products.

The Vision: Calm Over Chaos

Altman keeps returning to one metaphor. Using current devices, he told an audience at Emerson Collective’s Demo Day last November, feels like walking through Times Square with flashing lights and noise. What they’re building should feel like “sitting in the most beautiful cabin by a lake and in the mountains and sort of just enjoying the peace and calm.” The OpenAI device is designed for attention liberation instead of attention capture. Altman has described a device that can filter information and understand when something is important enough to surface, and when to stay quiet, using sensors and spatial awareness to build what he called “incredible contextual awareness of your whole life.” At the same event, Altman recalled that Ive told the team early on: “We are going to make people smile. We are going to make people feel joy. Whatever the product does, it has to do that.“ At DevDay in October, Ive put it in his own terms: he hopes the devices will “make us happy and fulfilled, and more peaceful and less anxious, and less disconnected.” It’s lofty rhetoric, so let’s explore what it might be.

What It Could Be

Based on researching all the clues available in the public domain, here is what they could be building:

• Sweetpea (more concrete leaks): A behind-the-ear wearable, closer to hearing aids in form factor than traditional earphones. The battery and processor sit behind the ear, enabling longer always-on use. It features a custom chip and ultrasonic transmitters that suggest environmental sensing and contextual awareness rather than just passive audio. According to leaked details first reported by TechRadar, the estimated release is near September 2026, with 40 to 50 million units expected to ship at a price point close to that of a smartphone.

• Gumdrop (more speculative): A pen-shaped device that could record writing and voice notes, with ChatGPT integration to make sense of memos and discussions. Reported by TechRadar as one of three ideas under consideration, with manufacturing partners already in discussions.

• Something else entirely: Every description from Altman and Ive emphasizes a new category. The device won’t be a phone. Altman told employees it isn’t glasses, adding that Ive had been skeptical about building something worn on the body. The form factor that ships may not match any current leak.

There are consistent threads across all possibilities: screen-less, proactive rather than waiting to be summoned, and designed to complement phones rather than replace them.

As Altman told Bloomberg, “In the same way that the smartphone didn’t make the laptop go away, I don’t think our first thing is going to make the smartphone go away.” Whether it’s earbuds, a pen, or a necklace, the bet is on ambient and proactive over another glowing rectangle.

No matter what it is, Altman told employees he expects to ship 100 million AI “companions,” predicting OpenAI would reach that number “faster than any company has ever shipped 100 million of something new before.“ That ambition tells us something about what they think they have, or possibly just their level of confidence!

Why This Attempt Might Be Different

AI hardware has mostly failed to win over the mass consumer market (so far). Humane’s Pin was discontinued and its assets sold to HP. Rabbit’s R1 couldn’t justify why your phone wasn’t better. The Friend pendant sparked enough public backlash that New Yorkers defaced its subway ads and organized a protest in Washington Square Park. Meta’s Ray-Ban glasses are the closest thing to a success, but even 2 million pairs sold is a long way from the 100 million promised.

The OpenAI attempt has one thing previous efforts lacked: Jony Ive, and the team he built. Ive led Apple’s industrial design team for over two decades, playing a central role in the iMac, iPod, iPhone, iPad, MacBook, and Apple Watch. Steve Jobs called him his “spiritual partner at Apple.” Tim Cook said his role in Apple’s revival “cannot be overstated.” He didn’t come to OpenAI alone. The 55-person team that joined through the acquisition includes senior Apple veterans who were responsible for shipping iPhones and Apple Watches at scale, not just designing prototypes but getting them manufactured and into hundreds of millions of hands. His team is responsible for billions of consumer device sales. That’s what $6.5 billion bought.

How We Got Here

It started in early 2023, with private dinners between Altman and Ive arranged by Airbnb CEO Brian Chesky. By September, news leaked that they were discussing AI hardware, and the Financial Times dubbed the project “the iPhone of artificial intelligence.”

In 2024, Ive assembled the team, co-founded io with Tang Tan, Evans Hankey, and Scott Cannon. He confirmed the partnership publicly in a September New York Times profile in which The Times described the project’s goal as creating “a computing experience that is less socially disruptive than the iPhone.”

In 2025, it got real. After OpenAI’s acquisition, the company almost immediately got tangled in a trademark dispute with Google-backed iyO, which was building AI-powered audio devices. The resulting court filings gave us our first clear window into the hardware work: the io team had purchased at least 30 headphone sets from various manufacturers to explore the market, and Tang Tan stated the first device would not be an in-ear or wearable device. By November, at an Emerson Collective event, Ive and Altman confirmed working prototypes exist. “I can’t believe how jaw-dropping good the work is,“ Altman said.

Then January 2026 broke the story wide open. In just three weeks, details of Sweetpea emerged from a leaker on X and were reported by TechRadar, rumors surfaced about a pen-shaped device codenamed Gumdrop via the same leaker, and OpenAI’s Chris Lehane confirmed at Davos that the first device is “on track“ for a second-half 2026 unveiling.

What This Means for Product Builders

If they succeed amid the hype, here’s what it will signal:

1. Contextual awareness is the next frontier. What OpenAI is building requires integrating structured data (calendars, contacts, preferences) with unstructured data (conversations, environmental perception, behavioral patterns). Large businesses have spent decades building unified views of structured data through CRMs, ERPs, and data warehouses. What OpenAI is describing adds real-world context on top: not just what’s in your database, but what’s happening in the user’s immediate environment. For product builders: what do you not know about your users’ environments and situations that would change everything if you did?
   

2. Rethink what you optimize for. The OpenAI device isn’t about less engagement. It’s a different kind of engagement: hands-off, ambient, proactive rather than reactive. Instead of maximizing time-in-app, you’d optimize for tasks completed without friction, moments where the device helped without being asked, cognitive load reduced. What would you build differently if “user didn’t have to think about it” was a success metric?
   

3. The disappearing interface. This vision has a name in design circles: calm technology. The idea that the best interface is one you forget you’re using. Current devices demand your attention constantly. The OpenAI device is designed to disappear: to handle things on your behalf without requiring you to look at a screen, tap a button, or even speak a command. If they pull it off, it’s not just a new product. It’s a new relationship with technology entirely.

Altman is betting $6.5 billion on a category that has failed repeatedly. That looks crazy until it works. The skeptics have history on their side. The believers have Jony Ive.

This article is brought to you by Productboard

Trusted by the world’s best product teams, Productboard turns customer needs into high-impact strategy in one end-to-end platform. With Productboard Spark, you can go from raw idea to polished product brief in seconds using AI that already understands your business.

See how Productboard Spark can help your product teams ship faster →

References

TechCrunch. Kyle Wiggers and Maxwell Zeff. “Jony Ive to Lead OpenAI’s Design Work Following $6.5B Acquisition of His Company.” May 21, 2025.

Axios. Ina Fried and Dave Lawler. “OpenAI Aims to Debut First Device in 2026, Exec Tells Axios.” January 19, 2026.

Wall Street Journal. Berber Jin. “What Sam Altman Told OpenAI About the Secret Device He’s Making With Jony Ive.” May 21, 2025.

CNBC. Kif Leswing. “Execs Say OpenAI Has First Hardware Prototypes, Plan to Reveal Device in 2 Years or Less.” November 24, 2025.

VentureBeat. Michael Nunez. “Here’s What Jony Ive and Sam Altman Revealed About Their Secretive AI Hardware Project at OpenAI’s Dev Day.” October 2025.

TechCrunch. Maxwell Zeff. “Court Filings Reveal OpenAI and io’s Early Work on an AI Device.” June 23, 2025.

TechRadar. Alex Blake. “Big OpenAI Leak Claims the ChatGPT Maker Is Developing an Earbud-Style Wearable with a Surprising Twist.” January 13, 2026.

TechRadar. David Nield. “OpenAI’s Mysterious ChatGPT Gadget Could Take the Form of an AI-Powered Pen.” December 31, 2025.

9to5Mac. Zac Hall. “Jony Ive and Sam Altman Have First Prototype of Mysterious AI Hardware.” November 24, 2025.

Bloomberg. Mark Gurman and Shirin Ghaffary. “OpenAI to Buy AI Device Startup From Apple Veteran Jony Ive in $6.5 Billion Deal.” May 21, 2025.

CNN. Clare Duffy. “How This Tiny Device Became a Symbol for the Backlash Against AI.” November 16, 2025.

The Verge. Sean Hollister. “Meta’s Ray-Bans Have Sold 2 Million Pairs.” February 14, 2025.

TechCrunch. Maxwell Zeff. “Humane’s AI Pin Is Dead, as HP Buys Startup’s Assets for $116M.” February 18, 2025.

Wikipedia. “Jony Ive.”

Apple Newsroom. “Jony Ive to Form Independent Design Company with Apple as Client.” June 27, 2019.

CNBC Make It. Tom Huddleston Jr. “Steve Jobs Once Called Designer Jony Ive His ‘Spiritual Partner’ at Apple.” May 24, 2025.

TechCrunch. Maxwell Zeff. “Meta’s Former Hardware Lead for Orion Is Joining OpenAI.” November 4, 2024.

New York Times. Tripp Mickle. “After Apple, Jony Ive Is Building an Empire of His Own.” September 21, 2024.

Reuters. “OpenAI, Jony Ive in Talks to Raise $1 Billion from SoftBank for AI Device Venture.” September 28, 2023.

Emerson Collective. “In Conversation: OpenAI’s Sam Altman and LoveFrom’s Jony Ive with Laurene Powell Jobs.” EC Demo Day, November 2025.

OpenAI. “A Conversation with Sam and Jony.” DevDay 2025, October 2025.

I'm excited to introduce Artur Grigio as the guest author of this article. Artur is a Principal Engineer at Common Sense, where he's building AI tooling to help local governments and municipalities make an impact. Artur and I work together daily navigating the realities of AI-first development—managing coding agents, vibe coding, and figuring out what actually works. With nearly a decade of experience across AWS, Amazon, and high-growth startups, Artur has built everything from fraud detection systems to horizontally scalable cloud infrastructure. He's seen firsthand how AI changes the developer workflow—and why human judgment still matters. Connect with Artur on LinkedIn to follow his work at the intersection of AI and public sector impact.

I recently joined Adam at Common Sense, a VC-backed AI company with an AI-first mentality. In other words, I am a manager of AI coding agents, and I vibe code a lot.

Every day, I go back and forth between having my mind blown by AI to wondering how it can be so thoughtless. Particularly as we hire and scale, I’ve been wondering if Zuck was right when he told Rogan “all junior and mid-level developer jobs will be gone by the end of 2025.”

It’s already 2026 and yet the anxiety behind that statement hasn’t disappeared. Models write code faster than humans, reason across your entire repository, and never get tired. Are the CEOs right?

What I’ve seen so far is not the end of junior developers—but a misunderstanding of their value. As public company CEOs increasingly point to productivity gains from GenAI as the justifications for large layoffs, we’d recommend considering these three points before you fire or stop hiring all your junior developers.

1. Cost: Humans Are Still Cheaper Than You Think

A single serious instruction to a frontier model can easily consume 30K–60K tokens. AI is incentivized to be verbose. Larger contexts, richer reasoning, more retries, more agents—token usage explodes fast.

Check out Claude’s initial set of instructions. If you think you’re saving tokens by not saying “thank you” or “please”… those are drops in the bucket.

There are two popular schools of thought about where this goes next:

• Optimistic view: Token costs will approach $0 as compute becomes abundant and we move from training-heavy to inference-heavy usage.

• Monetization view: AI companies will simply allow larger and larger contexts so they can charge for richer inputs—codebases, documents, images, video, everything.

Subscribe now

But the reality is much simpler.

OpenAI, Anthropic, and Google are locked in a race. Models are released in near lockstep. Each generation requires enormous capital expenditure—compute, energy, talent, and training time. And yet the lifecycle of each model is incredibly short. Many never come close to recouping their costs. Meanwhile, they are each pouring billions into costs that consumers don’t really see, so while it’s hard to know for sure, their pricing could be a competitive mirage.

It wouldn’t be surprising to learn that today’s frontier models are 90–95% subsidized for end users. Further, a smart pricing strategy would be to wait years for lock-in and then dramatically raise prices because the switching costs will be too high to do anything other than pay.

Eventually, that bill will come due.

When it does, token pricing won’t be $0.15 per million tokens for the best models. It could be $15+ per million tokens—or more—if you want state-of-the-art performance, reliability, and privacy guarantees. Let’s be honest, how many of us still use GPT-4.0? It’s not even a dropdown in Cursor!.

At that point, a junior developer who can reason, refactor, and maintain systems looks less like a cost—and more like a bargain. Counterintuitively, the junior developer might become the more efficient spend, or at least not the “no brainer” to outsource to AI as it is portrayed today.

2. Someone Has to Wrangle the Codebase

AI generates code aggressively.

That’s not a bug—it’s the business model. More tokens means more value captured for foundational model companies. But code generation at scale creates a new problem: entropy.

AI doesn’t feel architectural pain. It doesn’t sense when abstractions are leaking, when naming conventions drift, or when a codebase becomes conceptually incoherent. It will happily generate five different patterns to solve the same problem across five files.

Left unchecked, you don’t get a clean system. ifs are nested in for loops, which are nested in while loops… AI doesn’t care that your brain can only go 4-6 levels deep.

Junior developers are uniquely valuable here. They:

• Are lazy! They want to reuse code so much that they eventually stumble onto a good, succinct, simple solution.

• Are afraid of big PRs and won’t submit Sonnet’s rewrites of the entire component, just because you asked it to useState.

• Normalize patterns, simplify logic, and reduce duplication

• Turn AI output into maintainable human systems

In other words, they act as entropy reducers. The AI lacks even basic mental awareness of why your business exists and what might come next. It knows how to write a PaymentProcessor function, but it doesn’t know that the company is planning to pivot to subscriptions next month. A human junior developer sits in the meetings, hears the “why” and builds for a future that isn’t in the current prompt context.

AI can write a thousand lines in a minute. But someone still has to review it and make the codebase understandable again. That work does not disappear—it increases.

3. When the Product Breaks, Humans Fix It

More generated code means more surface area.

More surface area means more edge cases.

More edge cases means your product will break—in new, creative, and highly inconvenient ways.

AI is excellent at proposing fixes in isolation. It is far worse at:

• Understanding real user behavior

• Debugging production-only failures

• Reasoning about partial outages, degraded states, or cascading failures

• Making judgment calls under uncertainty

• And, it won’t press “Acknowledge” on that 2AM emergency page, which will escalate to you

This is where junior developers grow into indispensable engineers.

They triage bugs. They reproduce issues. They follow logs, metrics, and traces. They learn how the system actually behaves, not how it was designed to behave. Over time, they become the people who know where the bodies are buried. Without them, who will write your Root Cause Analysis (correctly) instead of regurgitating the same exact failure states time after time?

A system maintained only by AI becomes fragile.

Of course, we assume humans remain the primary consumers of source code for some time to come. If the next generation of LLMs can navigate ‘messy’ logic as easily as clean abstractions, the premium on code elegance will vanish, giving way to ‘dark’ codebases—systems that are mathematically functional and perfectly optimized for machine execution, but fundamentally illegible to any human mind. However, we’ve seen very limited evidence that we’ve reached this reality. AI fixes a lot of small bugs, but has poor performance on medium or high complexity issues across the code base.

Our Recommendation

Hire SOME amazing, thoughtful junior developers that use their brains AND use AI.

What is changing is that AI is replacing the illusion that software development was ever just about typing syntactically correct code.

Junior developers are how you:

• Control costs in an increasingly expensive AI ecosystem

• Keep codebases coherent instead of chaotic, and protected from serious bugs

• Build the next generation of senior engineers who actually understand the system
  

Share Emergent - Product Newsletter and Podcast

If you stop hiring juniors, you don’t eliminate the work, you just defer the pain — and make it much more expensive later.

If you’ve ever struggled to turn a half-baked AI question into a great prompt, you’re not alone. That’s why I’m releasing a custom GPT designed to rewrite your rough prompt into a high-performance, research-backed version—no prompt engineering experience required.

At its core is a meta-prompt and the world’s leading prompting research study:

We ask the AI to take whatever prompt it’s given and use the research to make a better one that is empirically superior.

This GPT builds on the findings of the Feb 2025 research paper “Improving Prompting With Prompting,” which demonstrated that well-constructed prompts can significantly enhance the quality of AI responses—especially for productivity and product design tasks. The paper showed how carefully written prompts consistently outperform ad-hoc ones across tasks like brainstorming, summarization, and code generation.

Why This Is Worth Using

🔬 Science-backed: Grounded in recent empirical research, not intuition.

🧠 Expert-level optimization: Leverages best practices like role specification, few-shot examples, task decomposition, and clarity constraints.

⚙️ Plug-and-play: You enter your draft prompt, and it returns an improved version ready for production use.

💡 Designed for builders and operators: Especially valuable for product managers, operators, engineers, and founders using LLMs in day-to-day workflows.

Try It Now

I’ve embedded the instructions and the research paper into a custom GPT so you don’t have to think twice. Just drop in your prompt preceded by the words “Improve this prompt: “ and watch the magic happen.

🔗 Launch the AI Prompt Improver »

AI agents can add superhuman capabilities to your product, but what is the intuition for when to incorporate them into your strategy?

Put simply, AI agents generate maximum value when we place them wherever modestly intelligent humans existed in workflows in the past. Agents gain power particularly when chained together, used in concert, or when forward-deployed. An agent can live on your computer to interact with your file system, on your social media account, deep inside a factory assembly line, on a satellite, or wherever that human-like quality is required for product success.

For those new to the buzz around agents, they are what they sound like: miniature GPTs that you can place inside complex architectures to perform cognitive tasks that machines have historically struggled with. They can operate autonomously, in clusters with other agents, or in a hybrid model combined with existing microservices. They leverage generative AI, so at a minimum, they can:

• Make a decision or judgment

• Summarize text or other content

• Analyze data or evaluate

• Generate text, images, or even systems

• Take actions as a real user (for example, make a purchase or send an email)

Generative AI is best for challenges requiring a modicum of judgment, whereas traditional services are superior for tasks that require certainty and precise instructions.

With this high-level heuristic in mind, let’s explore thinking models that could advance your product, moving from the least sophisticated to the most advanced. Along the way, we’ll provide examples and diagrams to illustrate how each might work across six varied use cases that we’ve named Inspectors, Conductors, Assembly Lines, Hydras, Missions, and Mobius Cycles.

Inspectors

Inspector agents excel at handling inconsistent human outputs, enforcing standards, or by providing qualitative critiques to ensure that work meets or exceeds expectations. In short, they can act as a quality-control gate. I like to think of this set of use cases as giving the agents a persona and then implanting them into a system or process where quality control or stress testing is needed. For example, a customer service system for a Fortune 500 company requires care and feeding across many dimensions:

• How technically competent were the representatives?

• Did they use appropriate soft skills in dealing with customers?

• Are they respecting company policies and processes?

A traditional human manager would have to tackle these challenges themselves, but a series of agents can operate on the manager’s behalf to spot concerning interactions (and eventually to monitor non-human agents too!). Imagine the following interaction among a hypothetical CVS pharmacy customer, a CVS customer service representative, and a Quality Control AI agent.

Share Emergent - Product Newsletter and Podcast

The above scenarios were scripted and analyzed 100% by AI agents, and you can see each taking responsibility for acting as a particular persona, providing valuable context for the customer service department in terms of how they want to train their human or AI agents to act. Just as we used to bake certain mores into a company culture, we now need to bake these into the AI to get the customer experience and personality we desire.

Stress testing is another strong use case for agents. Whenever a human or an AI comes up with a result, a stress-testing agent can act as a critical bystander trying to elevate the quality of what was produced. Take, for example, the following AI-generated summary of the Los Angeles Dodgers’ World Series win this year:

Let’s create and dispatch another agent as a sports news expert whose job it is to critique and improve on these clips. The agent correctly notices the lack of a score, no mention of the pitchers, very little sense of the actual action, and more. Here is an improved version after the writer agent stress tested and improved upon the summary. Unfortunately, our agent still needs some training, since the Dodgers did win much more recently than 1988, but the draft is substantially improved. A fact-checking agent is probably needed too!

While these agents are presented together in this example, in a live production system each might occupy different space in different or complementary systems. The choice is up to the product leaders and the system architects and will depend in large part on how often each agent might be reused in different contexts.

The inspector model excels when:

Read more

As your team grows, it’s common for each person to have a different understanding of what is implied by the words “urgent” or “high priority,” leading to costly mistakes for your product and customers. At each company, I find myself re-defining these terms during crises, so I’m sharing my definitions now so you can borrow or adapt the list and skip this step and get back to building amazing world changing products.

P0

P0 tasks require immediate attention.

• If you're not at work, you need to come in.

• If you're already at work, you should not leave until it's resolved.

• If it's the middle of the night, you must wake up to address it.

This is the highest severity level and it should be reserved for complete outages of critical production systems, situations where the software poses a severe risk to human life, a hack from an outside party, or other infrequent but high-impact events that can sink a business or destroy a customer relationship. Use this designation sparingly.

Examples:

1. A major bug is discovered in an AI-powered medical diagnosis system, and it is actively leading to misdiagnoses and patient harm.

2. An autonomous vehicle fleet management product experiences a partial outage rendering the vehicle sensors incapable of preventing an accident.

3. Hackers have gained access to your internal IT systems and are actively siphoning your IP offshore..

P1

A P1 is your top priority during normal business hours.

• Nothing else should take precedence during this time

• You don't need to go out of your way or interrupt your personal life to address this challenge.

• It is still very high priority and should be your ultimate focus, with any other items considered a distraction.

Examples:

1. A critical bug is identified in the machine learning model used for fraud detection, leading to a surge in missed detections and the costs are stacking up.

2. The virtual reality training platform experiences significant performance issues, preventing employees from completing mandatory safety training sessions, which within 36 hours will require a stop work order.

3. Search performance is degraded severely and users cannot do their work.

P2

P2s are tasks that need to be done in a timely manner but are not hair on fire. They necessarily come after P0 or P1 tasks, and the person assigned the task should determine where it fits within their schedule, within a reasonable timeframe of say a few hours to a few days at the latest.

Examples:

1. Implementing new features and enhancements to an AI-powered chatbot to incorporate a forthcoming change in rules that is imminent but not yet live.

2. Conducting performance optimization on a big data analytics platform to reduce processing time and enhance insights generation accuracy.

3. Developing additional security measures for a cloud-based enterprise resource planning (ERP) system to protect sensitive data given a known, and low probability exploit.

   Share

P3

This is something we would like to accomplish, but there's no particular urgency surrounding it. However, it’s not “someday/maybe” either.

Examples:

1. Data shows that there are some non-intuitive steps in our onboarding flow, but it’s not causing drop off or attrition, but it’s not a great look for us.

2. When users authenticate through social logins, the words displayed on the head of the box are the system names for the services, not the user-facing names.

3. We’ve changed the routing on our support tickets, but the existing system has a fallback that the team can access, but it’s a little clunky.

P4

We don't want to forget this item exists, and it would be nice to address it if possible. We're primarily tracking it so that if someone ever has a significant amount of free time, they can tackle it.

Examples:

1. The Spanish language translation of our product is using Spain Spanish but half of our users live in South America.

2. We need to update the copyright date on our website.

3. We’ve learned that there’s a good third party plugin that might save us some time, and someone ought to investigate it.

Implementation

I've never encountered team issues arising from disagreements over P3 vs P4, but teams lacking clarity between P0, P1, and P2 often face significant difficulties in a time of true need. The task creators feel that others aren't taking these issues seriously, and the task implementers believe they are proceeding based on agreed upon priorities, leading to communication breakdowns and customers left hanging.

Please feel free to use these definitions, borrow them, paste them into your own company handbook, or reference them here. You can even create your own, but don't attempt to scale your company without a clear understanding of the level of urgency required for different types of crises situations.

“PRD” is cringe, use spec instead

Let’s start by replacing this outdated term with “functional spec” or just “spec.” I detest “PRD.” Here’s why:

• The “R” for Requirements implies that Product is the master and everyone else is the order taker. False! Great outcomes come from blending disciplines to identify the best solutions.

• The label implies that the document contents and the product are static and won’t change. Both are false.

• Due to its baggage as part of the hardware waterfall movement, even the term itself makes people focus on process instead of outcomes and mission impact, which puts people in the wrong mindset.

Yeah, yeah, yeah, just take me to the template now please!

Yet they remain necessary at scale

On teams where you can skip the specs, please do so. However, most teams that are achieving some degree of success end up scaling to the point where keeping everyone aligned, and in the loop becomes a bottleneck to quick, decisive action. Use specs when:

• Alignment is required across a large group of diverse people representing different functions (ex: sales, marketing, CEO, legal, engineering, QA, etc)

• The complexity of the work and problem are poorly understood across the team

• You’re starting a new, large initiative

So here’s what must be included

Background and Goals

Explain what problem exists for the business and customers, at a high level in a few sentences, so we can all understand why it’s worth our time to read further. Don’t write an expository essay with five paragraphs, just a few sentences will do. If it won’t, you haven’t distilled the core issue.

Here are some examples:

FIRE OUTBREAKS

“Our customers track fire outbreaks across the world, but when fires are near population centers, there’s a lot of manual work required to check how close a satellite thermal signature is to a particular building or set of people. Our product gives users access to all of these data sources, but there’s no nice way to get this answer even with all of that data without layering it on top of each other. Our plan is to make a canvas that combines these insights into one map view to give rapid answers for any fire so that customers can identify proximity to populations in seconds, not hours.”

SYSTEM PERFORMANCE

“System search performance is causing customer attrition, and we need to fix it.  We lost 15% of our user base this quarter, speed is the #1 complaint coming in to customer success, and 95% of searches run an average of 15+ seconds, while industry benchmarks show than anything more than 1-3 seconds is unacceptable. Our plan is to attack the root architectural causes and add thoughtful UX flows for the most taxing queries that cannot be sped up.”

SEMICONDUCTOR DEFECT AI

“Semiconductor fabs are struggling to predict and prevent defects in their newest 3nm chip manufacturing processes. Currently, engineers spend 4+ hours per defect analyzing multi-sensor data from hundreds of process steps to identify root causes, leading to $2M+ in scrap costs per month. While our platform collects all the relevant sensor and inspection data, engineers must manually correlate across multiple dashboards and tools. Our plan is to create an AI-powered defect analysis workspace that automatically surfaces potential root causes by analyzing patterns across historical process data, reducing investigation time to minutes and catching systematic issues before they impact yield.”

That’s it, we don’t need more. Anchor us to right where the problem or opportunity exists, no more, no less.

Who’s It For?

A well-crafted spec clearly defines the target users to align marketing, sales, and product teams around who the product serves and how success is measured. This means specifying who the specific users are and what we know about them—their demographics, behaviors, work context, and motivations. This is not a place to repeat your Ideal Customer Profile for the whole product—it’s a place to call out more concretely which subset of those individuals care about this particular feature. Save space here while being precise!

Examples:

“Medicinal chemists at customers with <50 employees who cannot write code”

“Users who have created 3+ custom reports in the past six months.”

“Aerospace engineers at commercial satellite companies who manually process 50+ thermal imaging datasets per week”

Scenarios and Research

Scenarios define the real problems real customers are experiencing now that this spec is designed to solve. Share real-life stories about specific users facing clear, detailed problems. These are not generic user stories or synthetic aggregations. If you’re starting with “Users feel that” then stop and start again writing about a concrete human you’ve spoken to. If you can’t name a concrete human, stop writing the spec until you can. This level of specificity exposes the nuances of the problem and guides the product team toward meaningful solutions.

A strong spec should include at least two to three distinct scenarios, each from different customers but all tied to the same core problem. These scenarios should be varied to reflect different user contexts but stay true to the problem statement. Without multiple real scenarios, there is a risk of shifting from solving a user problem to pushing a solution. A well-designed product should address all these scenarios through a single solution, and as the feature develops, new scenarios can be added to ensure the product continues meeting evolving user needs. By the end of the PRD, the proposed solution should clearly solve these scenarios at least to an MVP level, ensuring alignment across product, design, and engineering teams.

This is also a great section to include research that helps make the case, explain the urgency, or position why this matters more than other work.

Examples:

DRONE INSPECTIONS

Dave Chen at WindAccelerators managed an inspection of 47 wind turbines last week using our autonomous drone fleet. When post-processing the imagery, he discovered that changing cloud conditions had created inconsistent lighting across the dataset, forcing his team to manually adjust contrast settings on over 8,000 images to run our detection AI. The 3-day delay meant their client missed a critical maintenance window. Dave needs a way to automatically normalize environmental variations across large inspection datasets before running computer vision analysis.

AI CODE CREATION WITH APIS

Dr. Sarah Patel at LLMsRUs was evaluating their new language model's ability to write code when she noticed it was hallucinating API endpoints that didn't exist. She spent 2 weeks manually sampling 1,000 model outputs and cross-referencing them against documentation because their evaluation tools couldn't automatically detect fictional functions. The delayed feedback meant the training team had already started their next iteration with the same issues. Sarah needs a way to automatically validate generated code against real-world API specifications at scale.

Share

Success Metrics

Next, we want to describe what outcomes we should see when the feature is live. After all, we only build to increase traction or revenue, so if we can't describe how the feature will impact one of these, we probably should not be building in the first place.

A popular model is to consider three styles of metrics in this section: business metrics, customer metrics, and technical ones. For example:

Business: 
-Earn the first pilot customer in [new market/new use case]
-MRR increases by 5%

Customer:
-25% of new users create a report with [new feature]
-Average searches per user increase from 2.2 to at least 2.5+

Technical:
-99% of API requests return in under 1.5 seconds.
-Maneuvers with less than five parameters compute in under one second 

Be as specific as possible and make sure that you add stories at the bottom of your spec for any additional metric tracking that's required to prove out these hypotheses. It’s far less important that you have lots of metrics or many categories of metrics and more important that you have at least one, measurable way to know if the feature produced the desired outcome.

Approach

Up until this point, we haven’t said much about how we’re going to solve the challenges above, or how we will achieve our goals. The approach section is all about defining this and it covers three areas.

The User Experience

This is where we want to include the user journey map, designs, mock-ups or sketches, or any descriptions of how we expect users to interact with the system. If it is a technical feature, this might be a list of the API endpoints we plan to support. It doesn't matter who is using it; this is the section that describes how they will use it and what their experience will be. Please remember that “experience” is not limited to User Interface!

This can also be a link to a design document.

The Architecture

In this section, generally in a diagram, we’ll show how the pieces of our system come together to deliver the feature in a scalable, maintainable, and efficient manner.

Software architecture is the high-level structure of a software system, defining how different parts of the system interact and work together. It includes decisions about components (like databases, interfaces, and services), how they communicate, and how data flows through the system.

This can also be a link to the Technical Spec for more complex features where more room is needed.

The Technical Approach

For shorter specs where not much space is needed to describe the technical approach, I like to include it in this section, but for more complex ones, I generally use this area to link to a Technical Specification document. There are plenty of great guides that describe what goes into a technical spec, but in short we need to address how we will technically build the system, what it’s constraints will be, which technologies we’ll use, and how they come together to deliver the feature.

Stories, Release Plan & Prioritization

This is where we add a table that summarizes the small bits of functionality that together will add up to our feature. I like to use a simple table in the following rough format:

Story | Time Estimate | Priority or Release

To avoid duplication of work, the “Story” in this context is frequently not a full user story or job story, which I put into the ticket instead. This list is more of a quick chop for prioritization purposes and for everyone to see at a high level what’s going to get included and what isn’t. For teams working in Sprints, you can assign each story a sprint number, or for other styles of prioritization you can simply mark items as P0, P1, P2, P3 to show what’s most important to get done in what tranches. This helps the spec to live beyond one sprint, although personally I don’t recommend that it live for more than 1-4, otherwise people lose track, and it’s a lot easier to copy the top half and add some new stories and make a new spec to keep everyone aligned.

If there is a separate technical spec, then I rarely include technical stories in this list, but if it’s all on one document (which I prefer if it’s smaller), then it’s fine to include user-facing or developer facing high level stories. This is NOT a replacement for a complete decomposition of the work that will live in the ticketing system.

Example:

Allow users to download a PDF report of the calculation | 1 day | P1
Track usage of the calculation feature | 4 hours | P2
Integrate Segment to receive all events | 2 days | P3

Typically I find that the sweet spot is 5-15 stories. Beyond that and you likely have something too big for one spec, and fewer than that and it’s likely you don’t need a spec at all, just a few tickets in your existing system.

Here are the nice to haves

If we have one of everything above, in many cases we’re done, but I find that complexity lives in many forms and shifts from feature to feature. Here are some nice to have sections that you should consciously decide if you need each time.

Assumptions and/or Risks

What assumptions are we making about the user? About the market? About human behavior? About adoption? About 3rd parties? What risks do we have embedded in this execution? Are there problems that have never been solved before? Are there dependencies on other teams? These are great to get out into the open so the team can solve them proactively before they block progress.

Experimentation Plan / Adoption Plan

If our uncertainty around the success of the feature is modest or high, then it frequently makes sense to define how we will get feedback on the new feature. Who will we speak to? Will we run A/B tests? How will we roll it out? In stages? To whom first? Please note that by the time we get to a spec, we should have already done a good deal of discovery work to validate the value of the feature. The mistake most teams make is to assume that once a spec is written, the work should proceed from start to finish without reference to how the customer is receiving this new feature at each increment we release into the wild. This is where we can get clear on how we’ll iterate and with whom.

For more mature features in larger companies, this can be a great place to link to a marketing and/or GTM plan in addition to an early adopter strategy.

Test / QA Plan

For more complex features, or even as a matter of good ongoing hygiene, it’s helpful to link to the QA plan, or if it’s simple, to include it in the spec. I find that great specs are about 65% content, and 35% links that other specialists can follow but that don’t need to block reading of the main value and approach content.

Non-Goals

Particularly for more ambiguous features, it’s helpful to re-assure everyone of what is NOT expected to be built, who is not going to be served by this feature, and any other work we explicitly know should not be included in this phase. This helps everyone breathe a sigh of relief and focus where it counts first.

Specs and Releases

While there is some tidiness to having one spec for each release of a feature, in practice I find this leads to teams spending far too much time writing, and not enough time on everything else. Additionally, on the strongest teams, every commit is potentially a viable externally facing release, if the team decides it is worthy. For both of these reasons, I see no reason to directly tie specs to releases. Instead, I encourage teams to use the stories section to define how specific subsets of the work will be released to the world. I highly encourage product teams to release as often as they have sufficient value that could either help a user or elicit feedback. Sometimes this is as small as deploying a new table sorting method, and other times it’s a much more meaningful piece of work. That is a judgment call for the product and engineering professionals within each organization, not something a template should specify.

Rapid Fire FAQs

How long should a spec be?

Ideally, 1 to 1.5 printed pages of text. Certainly no more than three pages. Keep it concise or no one will read it and alignment won’t happen. Link to external content and force yourself to synthesize and summarize. The spec is not a replacement for the level of detail that goes into your ticketing system. However, it can be a nice place to centralize all of the most important content and links to related content about that feature to help align functions like engineering, product, sales, marketing, and QA.

How long should the work last from one spec?

It could range from about one sprint to a few months. I recommend a scope that will last between two and eight weeks. Anything less and the writing is occupying too much process time, and anything more and the team loses the beat, alignment wanes, and urgency evaporates.

How many features go in one spec?

One spec is for one big problem statement. There can be many facets to that problem statement, but we do not mix disparate features together. For example, if the feature has billing implications, usage analytics needs, or onboarding changes, that's fine so long as they're all part of the same feature. However, if we do not yet have billing implemented at all in our app, we would use a separate spec for billing and a separate one for the current feature. In essence, the spec is for one coherent feature; it is not a summary of all the things that will go into a particular release, broader than that feature.

Here’s the template

You’ve made it all the way through. Here is my template with a quick summary of each section. Let’s go build something world changing!

Adam’s PRD/Spec Template

Strategy templates should be concise and include examples! Existing ones are failing product leaders and CEOs because they:

1. Don’t fit on a single page, making it hard to keep the context in mind

2. Fail to provide the tickler questions needed to create the content

3. Rarely take into account the business implications required for success

4. Never have examples! And…

5. Are frequently indistinguishable from mission and vision templates and roadmap ones, which are not the same as strategy!

Today I’ll improve upon what’s out there and share:

• My one-page product strategy template

• The tickler questions that help you formulate each answer

• An example pulled end to end using SpaceX’s Starlink product1

• A blank copy of the template you can download and copy to get started

So let’s start again!

We’re going to run through each topic, and we will keep our answers concise. You will probably notice that writing this document requires buy-in from across an entire business, so if you’re a product manager, don’t go it alone.

Vision … but wait, why?

We must start with the vision, because the vision is the end state our customers will experience that the strategy is designed to achieve, so without a pithy reference to our vision, the whole document loses its anchoring. The rest of the document will describe how to get there. This is descriptive and should be focused on clarity internally, not marketing power externally. Remember, this document is for alignment of effort, not for selling.

For vision, here are the tickler questions:

• What future state are we trying to create?

• How does the world change for our customer in the future?

• What will the customer actually put their hands on in that future?

Here’s what Starlink’s might have said early on:

Internet access via satellites from anywhere on Earth.

The words themselves are not sexy, but it is clear and concise. As customers, we know we will receive the ability to connect to the internet no matter where we are on Earth. We don’t know the exact features or the form factor, but we have a solid idea of what’s in it for us if we go along for the ride.

Big problem being solved, and the ancillary ones

Now we move on to what problem the vision state solves for.

For problem solved, here are the tickler questions:

• What need does this vision address?

• How is the problem currently underserved?

• Why is solving it important to the customer?

Starlink example:

People in remote locations cannot access land-based internet services, and are thus excluded from the digital economy.

Further, existing solutions are low-bandwidth, unreliable, and not self-service.

What makes our approach special?

Now we dip into what the solution looks like and why we can be best in the world at delivering it.

For our approach, here are the tickler questions:

• What are the key benefits of our product (with passing reference to the pains experienced by the customer in the status quo)?

• How is it differentiated?

• Why will we be superior as a team or company?

• What is our unique opinion about how to solve the problem?

Starlink example:

1. High-speed (100+ Mbps), low-latency (<50 ms), high-bandwidth internet
2. Easy at-home self-setup with flexible subscription terms
3. Portable terminal you can take with you
4. No one can get satellites to space more efficiently than SpaceX
5. Support from low-Earth orbit (LEO) to enable #1

Share Emergent - Product Newsletter and Podcast

Who is this for?

Now we need to define our customers. Who will our beachhead customers be that adopt the technology first?

For customer profiles, here are the tickler questions:

• Who has the problem we’re solving for most acutely?

• What behavioral or demographic attributes describe them?

• What market niche are they in?

• What unique circumstances are they struggling with?

Starlink example:

Residential customers in rural or remote areas with limited or no access to terrestrial internet options. 

Governments and NGOs providing internet access in disaster zones or developing countries.

Mobile users such as travelers, truckers, or boaters.

How will we reach them?

What marketing and go-to-market tactics will we use to reach our target customers as described above?

For GTM, here are the tickler questions:

• What methods will we use to get in touch with our future customers?

• What distribution and partnership channels might we employ?

• By what mechanism will we sell?

Starlink example:

Direct online sales.
Targeted online advertising.
Partnerships with rural communities and businesses.

What does success look like?

What measurable results are we aiming to achieve across different measures of success?

For success metrics, here are the tickler questions:

• How many customers do we need?

• What revenue will validate our approach?

• What sort of product usage traction do we expect?

• What guardrail metrics should we protect?

Starlink example:

1M subscribers within 12 months
99.5% uptime at >50 Mbps 
$500M in revenue
50 satellites launched per year

What most stands in our way?

This is where we get to our step-by-step plan. I like to conceptualize it by thinking through the obstacles to success and then imagining how to remove them.

For the obstacles and plan section, the tickler questions are:

• What are the largest obstacles to achieving these business outcomes?

• What are the largest obstacles to achieving these product outcomes?

Starlink example:

We need a large coordinated constellation of satellites in LEO
We need to build a user-friendly dish and Wi-Fi router
We need simple-to-use software for users to control access
We need a way for sales to penetrate deep into remote communities

You can take this a step further and go into more detail, but in reality, this list is the final output. Taking into account everything above, what do we need to actually get done to realize the vision? In this case, it’s building the constellation, the at-home devices, and the ground network among remote community members.

How will we support our customers?

Don’t forget your plan for how you will keep customers happy and unblocked.

Customer Success tickler question:

• How will customers interface with us?

• What will happen when there is a serious or urgent issue?

• How will customers gain knowledge about our product?

Online help center and in-app customer support, with Tier 2 live agent support and community forums

Closing thoughts

I’m not trying to pretend these one-pagers answer every single question each team needs to execute, but that isn’t their purpose. I’ve seen strategy documents in the 20+-page range, and I’ve seen ones where most of that was fluff and ones where most of it was valuable. The purpose here is to drive cross-company alignment about what we’re building, for whom, how we will reach them, why it will be hard, how we will win in spite of that, and how we’ll continue to cultivate success in the long term.

Templates

Here is what the complete

one-pager looks like for Starlink. You can see that it is easy to review and to provide feedback on, and that this model forces us to make tough tradeoff decisions instead of hiding uncertainty in long-form paragraphs of text.

Share

Here is a blank template with the tickler questions for your own use.

You can also try my Strategy One-Pager CustomGPT that takes a topic and runs it through this exercise automatically.

One of the most frequent questions I receive as a product-focused startup builder and advisor is something to the effect of “If you were our advisor/coach/product leader, what would you do in your first 100 days to help us?” It’s not an easy question to answer in a pithy way, so I’ll dedicate this post to the strategy and the model, bearing in mind that the specific implementation will vary dramatically based on what is found at each stage of the diagnostic.

Value hunting

The most concise way I can answer the question is “I will hunt for and sniff out the value.” Value can take many forms, of course, from technical intellectual property to business and customer value, but suffice it to say that the strategy is to first find what drives the business in the mind of customers, what makes the largest and most differentiated impact to their mission, and to understand in client engagements which parts of the value proposition matter most. Of course, the diagnostic is much broader than this and includes everything from who is on the team to the tools in use to the more substantive issues just mentioned.

First, listen and absorb

“Where is the roadmap?” and “What should our strategy be?” are often asked within the first two weeks in a new setting. This is well intentioned but ludicrous. The desire is benign, and one cannot blame an organization for immediately wanting clarity, but creating first-class answers to these questions takes on-the-ground information that even the best-equipped product leaders won’t have when first engaging. Crucially, when a firm early in its journey is hiring a new CPO or VP of Product, usually it means these problems are not being solved organically. An example is a killer set of technical founders who aren’t sure how to build and scale a business. It’s cheaper and easier to align with fewer cooks in the kitchen, so traditionally the CEO or another founder is playing these roles if they can, and if they are not, it generally means there is a gap, in which case the diagnostic answers will not be obvious or in plain sight, or there could be too many answers and no decision yet about which ones matter most. By contrast, a successful startup without a product-scaling challenge at its current phase likely already has these answers, even if they aren’t well documented or don’t reside within one person’s head or a specific job title.

The one thing

Before we can move into the methodical set of diagnostic challenges, we must first be clear about what one thing the product is supposed to do. Later we will determine if the one thing is the right one thing, but the first step is simply to ask founders what it is. What problem in the world is this product designed to move the needle on? Why does that problem matter? Who cares about solving that problem? Armed with a sense of the one thing, I begin to engage in a methodical review of the key factors and disciplines that will propel successful outcomes. As an aside, it is okay to have secondary objectives, but the top objective must be clear and it must be at the top; otherwise all subsequent experimentation, research, marketing, sales, and feature building may be misaligned to this one thing. A few examples of the one thing might sound like “integrating disparate datasets” (Palantir) or “helping people remember hard-to-remember details about people they meet” (NetworkNerd), or “building a self-driving car” (Waymo). Of note, the one big thing does not have to take the form of formal customer problem statements, and it also isn’t the marketing or positioning statement; it is simply a factual expression of what it is that this organization thinks it is doing right now, and it needs to be the value proposition that would sell the product alone even if there were no other features.

With the one big thing in mind, I enter the diagnostic phases. Generally I’m tackling these in parallel as schedules allow, and because early findings in each area may inform questions worth asking later in the others. I try as hard as possible to just listen and not opine on what should be, at least not yet.

Team

The team diagnostic is all about the people-startup fit. Does this organization have the right players on the team, with the right expertise and the right backgrounds and the right DNA to match the one big thing? Generally I begin by interviewing each of the founders, each of the executive team members, and the team itself. I’m asking questions like “Why did you found this company in the first place?”; “What are your strengths and weaknesses, and the company’s?”; “What’s working and not working?”; and, perhaps the most interesting, once the baseline is established, “What are you most proud to have achieved or built here in the past few months?” The goal during this phase is to identify if everybody fits, if there are any gaps in talent, or if any roles need to be adjusted, or even to surface if there are people who are simply out of sync with the strategy. The phase includes employees, contractors, and consultants, if they’re playing primary roles in the business. It may sound obvious, but I’ve frequently discovered at this phase challenges such as a startup building cutting-edge database technology without a database expert, or doing hard science work without a full-time, equity-compensated scientist, or attempting to uniformly shape the output of 100-plus engineers without a product manager (all real examples, with company names dropped to protect the well-intentioned). It’s rarely anyone’s fault per se, but startups move so fast that it’s easy to migrate into a state in which key roles that should exist are unfilled or in which a coding-academy graduate with two months of experience is doing long-term architecture. It’s not that they cannot be successful this way—the person and their passion reign supreme, not the resume or pedigree—but there must at least be a plausible story for how the talent matches the strategy.

Subscribe now

Product management

The first thing I do with the product is play with it. This may sound obvious, but it’s shockingly overlooked. My first task is to touch every single button, every single drop-down, every single setting, across every single product or product line at the company. I’m usually hit in the face pretty quickly with some sort of blocker: sometimes I just don’t understand what the feature is intended to do because of a lack of domain knowledge, but sometimes the features just don’t make sense. The first job in the product portion of the diagnostic is to figure out what’s actually there and what value each feature is believed to be delivering to the user, and of course to surface blocking bugs and issues, for example crappy performance or an inability to log in.

Next I attempt to learn to give a demo. Learning the demo is huge because it generally forces you to validate the feature set, validate the product development execution, validate a sane promotion/DevOps/release model, and even validate the messaging of the product. I frequently find that pre-Series A, giving the demo is not straightforward. It’s entirely okay if the demo is backend technology only, but if it cannot be shown in some way, it likely cannot be valued or sold to a customer, so this is essential.

At the same time, I want to consume the existing roadmap or backlog so I don’t waste time writing down features everyone already knows are missing.

Finally, I’m looking to see what sort of intelligence-gathering machine exists. How does validated customer learning make its way back into the product? Is the team using Linear/ClickUp/Jira with a backlog, is there a feedback email or Slack list, are they implementing Intercom, Productboard, etc.? Is the top customer success person tightly integrated with the product team? Same for sales and marketing. It’s key to be sure that what is learned is not lost and that it’s captured in some future referenceable manner.

Engineering

For a long time I didn’t include engineering as a phase even though I always performed it, because in more hierarchical or traditional organizations, there is a lot of pushback when a product person or with a non-computer-science background starts probing around the architecture and technology stack choices. This is a fallacy, but it’s also a delicate balance. Product must understand the opinionated decisions that engineering has made in its path toward solving the customer’s problems. Product must understand these tradeoffs and decisions (1) to be intelligent in front of the customer, (2) to have intuition for what’s hard and what’s easy to build, to avoid a small experiment taking too long or to avoid asking for an experiment thinking it’s hard when it’s actually easy, (3) because sometimes these decisions don’t meet customers’ future needs or add up to unique IP on the path to the big thing, and (4) because sometimes product can contribute directly to the tech stack with small bug fixes or getting hands dirty pulling analytics and prototyping or testing.

If the company is breaking ground and innovating on the technology end, the first basic gut check is whether the CTO or VP of Engineering is amazing. In cases where the technology itself is not hard to build, it’s not as essential, but there still must be competent engineering leadership to resolve weedy technical challenges, to spike out answers to new challenges, to be the technical voice in front of the customer, and to resolve day-to-day engineering tradeoff decisions such as how to model data or which architecture to select in any given feature’s implementation.

Market and marketing

Next I’m looking to understand the market, the messaging, and the marketing. What market does this product live in? How does it operate? Who does the market trust, and what are the incentives? A great tip I’ve learned on this one when facing a new market is to talk to a hedge-fund portfolio manager who invests in the sector, as they will often have a fantastic five-minute-or-less synthesis of how the market “works,” which will help you to back into the assumptions you need to check and test.

For example, at Slingshot, when we experimented with selling overhead imagery to consumers, we had to first understand that the existing satellite imagery market was dominated by the federal government and mostly consisted of a few players that owned the lion’s share of the satellites, and that the aerial imagery market largely was acting at the behest of insurance companies, local governments, and architecture and engineering firms. Together, these implied that orders had to be large, which informed a huge amount of follow-on decision-making. Once you understand the basic mechanics of the market, you need to understand but not obsess over the competitors. You’re not done with this phase if you cannot identify at least the top three competitors, their value propositions, and why yours might be different.

Next, how is the business and sales end of the company talking about the product, and is the customer listening? Are you using the right terminology to meet the customers in their world, not yours? At Palantir in the early days, we could have walked into government agencies and said, “We have a novel take on the graph database and numerous services that will unify records from multiple systems into it,” but instead we learned to first say, “We can take a suspected terrorist target and see how they are connected to people you can access through an understanding of their digital footprint across clandestine collection systems.” Needless to say, the latter lands much more powerfully.

Finally, what is the go-to-market strategy? I will admit, I hate this term—it encompasses far too much to far too many people, so I will scope it down. For an enterprise business, how does the business plan to earn its next dozen or so customers and, for a consumer one, the next 10,000 (if earlier-stage). Is this answer plausible, and does it match up with the roadmap, value proposition, and engineering decisions? For example, if sales thinks the path is through multinational consumer packaged goods companies and the product only caters to startup tech bros, there’s a problem.

Customer sensing

Who are the company’s existing or future planned customers, and why did each customer buy the product in the first place? Despite the “one big thing,” what is each of the customers actually using the product to do? What business value is this product generating for these customers, and if they didn’t have it, what would the alternative state of affairs be and how painful would that be for them? Usually this can be assessed through a great discussion with customer success or sales engineering, but I highly recommend also getting some firsthand customer signal even at this early phase if it’s politically possible (if it’s not, that is itself a red flag). Often the translation of the desire or need is lossy, and precision is everything in product. Precision in understanding customer needs and delivering exceptional solutions to them is the difference between Facebook and every other social media company we no longer speak of, and it’s the same in every industry. The startup ecosystem is littered with “Well, my company was Y [example: on-demand food delivery] before X [Postmates].” No, it was not, and this is harsh, but it’s the reality. If you were X, you would be X, other than issues of market timing, for example trying to be an AI-driven company 20 years ago. Generally, the other company, X, understood something about the customer just a little bit better, or at least did something about it, and that is why they won. Of course, maybe you do understand the problem perfectly but you have a co-founder explosion or run out of money, so I don’t mean to imply that Y never had a chance to be X for only this reason.

In this diagnostic I am also looking to assess traction. How many users, what churn is like, what the customer acquisition cost is (if that’s relevant, which it is not always early on), and every derivative metric that matters in product belong here. Crucially, this is where I am looking to find a ride-or-die customer, and ideally a compelling set of them. There are many names for this customer, but I like the rap-lyric version “ride or die” because it means a person who is so in love that they will ride with you even though they know you are a criminal, and would rather die or go to jail than not be with you. Of course, it’s not nearly so dramatic in startups and we obviously don’t want to be anywhere near illicit activity, but the degree of love is the same—do you have a customer who doesn’t care how much about your product is broken or failing because that one thing is so important to them and you solve it, so they will stay with you at nearly all costs?

Finally, I’m looking to understand the job stories, user stories, and/or story-mapping for the user’s journey in the product. With analytic products these are a bit more open-ended, and with transactional products they can be airtight. It’s less important how great they are at the beginning and more important that they are understood widely, regardless of what format or philosophy reigns in the culture.

Share

Strategy, vision, and roadmap

With all of this signal in tow, I begin to look critically at the existing strategy, vision, and roadmap, and I’m looking to assess if they line up, if they are credible, and if they leverage the startup’s unique DNA. Lining up is easier to assess—if the target market is commercial insurance and the persona is the claims adjuster, then I’d want to be sure the user stories reflect that adjuster and his/her specific needs, and the same for the marketing and messaging etc. The test of “credible” is more of a judgment call. Does the company have the track record, the credibility, and, as Yale/Deloitte used to call it, the eminence and “brand permission” to represent itself as a group that can credibly solve this problem. With that in mind, I begin to construct and/or revise a single document that encapsulates the vision, strategy, and roadmap and begin to solicit feedback on it. The roadmap, in my mind, is always a living document, especially at early-stage companies, because so often our assumptions do not pass muster in the market, so I aim to revise and update it no less than once a month.

Hunting for value isn’t a sprint; it’s a deliberate excavation of your company’s DNA. In those first 100 days, you’re not just building strategy artifacts but uncovering the heartbeat of your product. So before you start pontificating about strategy, listen, absorb, stand back, and assess. Then, armed with ground truth, you can start crafting a vision and structure—and team—that’s more than just pretty slides: it’s a battle plan for delivering knockout value based on the company’s unique strengths and the market’s unique gaps.

Hi! If you enjoyed these insights, please subscribe, and if you are interested in product coaching or fractional product support for your venture, please visit our website at First Principles, where we help the most ambitious founders make a difference.

The past decade has seen no shortage of companies that support building with traditional AI technologies; however, the landscape supporting generative AI builders is in its infancy. As the capabilities of foundational large language models (LLMs) like ChatGPT, Claude, Llama, and Gemini advance, and as more product teams discover novel use cases for them, the complexities of putting them into practice at production scale and quality are rapidly emerging, presenting a lucrative opportunity for founders and investors alike. 

This post explores seven product gaps we have witnessed repeatedly in startups through public companies since ChatGPT 3.5 took the world by storm in November 2022. We share insights into the nature of each challenge and why it exists, and do our best to highlight the nascent solutions offered by innovative companies seeking to address these issues. Every gap is one where we have seen developers spend considerable time and energy writing custom code to solve important problems. We cannot say with certainty when no solution exists, but we can say that strong ones have not achieved wide adoption across these seven areas, creating a massive set of opportunities.

Disclaimer: The author is an investor and shareholder in some of the companies mentioned in this article. All research and analysis presented are based on publicly available information.

Gap 1: Guaranteeing predictable outputs

When product development teams integrate LLMs into their software architecture to solve customer problems, those models must interact with non-LLM services, which expect consistent inputs. LLM outputs can vary considerably, yet the result must be consumable by subsystems that cannot interpret large bodies of unstructured text. The systems that process information immediately after the LLM can easily fail if the LLM does not provide a predictable response style. For example:

• The AI is asked to summarize a large body of emails and display those in a mobile-friendly design, with a place for the summary and some derived metadata such as “Topics.” However, because the length, formatting, and style of the summaries are inconsistent coming out of the LLM, the UI breaks, text wraps unnaturally, and expected fields like “Topics” can receive nonsensical responses.

• In cases where the model is asked programmatically to produce JSON responses for consumption by downstream services, the range and formats of the values are not always respected. For example, a field that represents a score from 0 to 100 might return a score outside of that range, or the system could generate a value that is not in an expected list of possible enumerated values, such as returning “autumn” when only “summer, fall, winter, and spring” are valid and expected. 

LLMs are designed not to return the same response every time. In fact, this non-deterministic architecture and sampling of information keeps the models feeling more “human” and “creative.” If we ask people to identify the top presidents of the United States, we will receive varied responses. This variance is fine in humans and machines alike, but we want all participants asked to provide the same number of “top” presidents and we’d like those responses to be in the same format, for example: full name, then years as president as eight-digit ranges, and country (ideally U.S.). When we receive results outside of these bounds from a machine, it breaks our interfaces, downstream services, and data models.

Fortunately, there are solutions for this gap. Most people improve their outputs through prompt engineering, or in this case more clearly specifying the desired response constraints. For example, instead of “Who are the top U.S. presidents?” we try “Who are the top U.S. presidents, list only 5, and list each one’s full name and year range as president.” This helps significantly, but it does not guarantee the response style. If your product uses OpenAI, you can take advantage of OpenAI’s “Functions,” which allow developers to specify the outputs they wish to receive. However, OpenAI includes an ominous warning in its documentation for Functions:

“We strongly recommend building in user confirmation flows before taking actions that impact the world on behalf of users (sending an email, posting something online, making a purchase, etc.).”

Palantir’s Artificial Intelligence Platform offers a no-code solution that combines this capability on top of multiple foundational LLMs. This is a powerful solution that would require a lot of custom code, especially when many models are used together, to get the best response in an ensemble setting.  

“Developing Your AI Intuition” Palantir product video showing its AI Platform (AIP)

Finally, WhyLabs.ai, a Series A startup out of Seattle, provides a full set of observability and security functions that can assist in similar applications. CrewAI can also assist here by using multiple agents to check the responses of earlier ones in the process. However, while all of these products have made some progress in enforcing strict response formats, most options appear to stop short of handling entirely incorrect, albeit well-formatted, responses without additional logic built in (See Gap 6). Ask an LLM to generate a set of latitudes and longitudes on land, and it’s possible to get a set that look real and have the proper format but that don’t respect the true constraints of latitude and longitude (for example, latitudes cannot be over 90) or that produce ones inside bodies of water instead of on land.  

Gap 2: Searching for answers in structured data sources

LLMs are predominantly trained on text data, so they inherently struggle with tabular structures and NoSQL information. The models are expecting certain words and thoughts to come before and after certain other ones, like in a paragraph of written text. This inherent structure within unstructured text data is what gives meaning to the words on the page. Although models can interpret individual rows or records in structured data stores, they struggle to understand the implied relationships that may exist between records, or in the reverse, they assume relationships between records that are physically proximate when no connection exists. Additionally, many structured data sets use schemas specific to the companies maintaining the data. These schemas may not have meaning to the models themselves because the models have not been trained on the names and meanings of the data fields used.

For example, a field for “location” can largely be understood as there are many references to locations in the text that these models are trained on. However, a field that is called “resolution” with scores from 1 to 5 is not set up in a way where the model knows if 5 is good or bad, and thus the model will struggle to answer questions that correlate strong resolution of customer support issues with the circumstances of the case without some clarification first. Additionally, there is no guarantee that the LLMs won’t make up records by sampling data from multiple records or rows.  

This is problematic because some of the most powerful problem statements in product require total accuracy and precision and come at the intersection of structured data sets and unstructured ones. Financial analysts performing an audit or preparing a financial report, health professionals reviewing historic notes and patient test measurements, and military planners executing lethal operations based on remote signals combined with narrative intelligence cannot take these sorts of risks. In these fields and many others, developers are spending significant extra time and code wrangling the LLMs for accuracy against structured data sets. 

In fact, what we have seen most commonly is that developers will use the LLM to construct and dispatch a traditional, non-LLM-powered search to a database and return only the result to the LLM for summarization, since the LLM cannot be trusted to interrogate the structured records. It’s telling that ChatGPT does not try to use its LLM on structured data either—as of this writing, if you upload a structured spreadsheet and ask a question and check the source, you will find that it is using the LLM to write Python code to do the retrieval using a Pandas data frame, not the LLM itself.

As in Gap 1, one of the best solutions so far is better prompting, but more research is required. Microsoft commissioned a study on this and shared its results last year, and its team was able to increase accuracy against structured data sets with a variety of enhanced prompting techniques.

Another option is what was mentioned previously: use the LLM to direct and structure a traditional query against the structured database. Unfortunately, this approach is also flawed, as it rapidly becomes an exercise in writing code to protect against Gap 1, where unpredictable outputs don’t match the schema of the data, or they misinterpret the schema as compared with the intent of the question. 

Pecan.ai appears to be one company that is trying to deal with at least part of this challenge in that its software allows customers to connect to structured databases as an input source that its models can interrogate, but even the company’s blog calls out that LLMs are not great at dealing with unstructured data.  The open source library LangChain and its paid enterprise companion also offer enhanced connectivity with structured data sources.

Gap 3: Understanding unusual-structure, high-value data sets

LLMs typically struggle with data types that they were not explicitly trained on, so the more difficult the data is to access and the more obscure, the less intelligently the computer can reason about it. Frequently, this would also mean that these data sources are less valuable, although this is not always the case. Consider, for example, satellite imagery, the market for which is billions of dollars in the U.S. just for the images and closer to $100 billion when the geospatial analytics products are combined with the imagery. 

LLMs are not trained on these sorts of images and they cannot, as a result, answer questions like “Does 567 Oak Street have a pool in the backyard?” based on the latest satellite or aerial imagery. To get an answer like this, the imagery must first be run through a computer vision model that detects pools, and then that detection information about the satellite image would need to be appended to the record. Then the LLM would have to query that metadata, again triggering the challenges in Gaps 1 and 2. 

The same thing is true for medical device imaging such as ultrasound, x-rays, CT scans, and MRIs. A decade or more of work has gone into analyzing these modalities but using traditional computer vision models, not LLMs. Another example is engineering drawings, such as computer-aided design (CAD) files, which contain tremendous amounts of important information that would be helpful to query without having to manually open and inspect each file.

It wasn’t long ago that still images from cameras, videos, and audio would have been on this list, so we are hopeful for rapid progress in solving these gaps, and certainly the economic incentives exist for companies to exploit them. Some companies are focused on these niche areas, though to our knowledge none are yet training LLMs to understand the pixels. 

For example, Danti.ai is making strides toward the searchability of satellite imagery and other location-based information through semantic searches, Synthetaic is working to detect the as yet undetected in those images, and Butterfly Network is making sense of ultrasound. However, we are not aware of a company that has trained an LLM to recognize the pixels in these data types so much as using traditional methods to get around this gap. Academic researchers are working on these domains, and it is likely only a matter of time before their work is more commercialized.

Share

Gap 4: Translation between LLMs and other systems

Effectively directing LLMs to interpret a question and then undertake specific tasks based on the nature of the user’s question remains a challenge. When the LLM needs only to answer the question with the data already embedded within it, it excels; however, frequently LLMs are asked to be the first line of defense in interpreting the user’s intent, where different intents should lead the software to route the query to different downstream systems. 

Take, for example, how ChatGPT knows when it was asked an image generation question that should trigger DALL-E, or when its “Analyzing” function is invoked to perform math or use formulas. Similarly, how does Gemini decide when to use a dynamic UI to display its answer? While it can look like magic in retrospect, it seems clearer over time that these innovations likely depend on strong query classification and routing logic built into the interpretation of the user’s request. Each style of question triggers different logic under the hood, and classifying the user’s question into these different realms is itself a massive undertaking, since the questions can be so entirely open-ended and dissimilar from each other. Further, if the product calls for routing the question to multiple places, a summarization function is needed to recombine the results.

We have seen no companies attempting to own this space in the abstract, but we suspect that many companies are building this capability for different use cases and different downstream systems. We believe there is room for a company to become the “Zapier for AI model routing to third-party systems” and expect that firms like Make and of course Zapier will lean in as this problem set becomes more openly discussed and clearer to all.  

Most enterprises are too early on the generative AI adoption curve to recognize that this is an issue, but once they put their data into such a system, it won’t be long before they are asking the AI to take slightly different actions based on the inputs, for example following instructions like “Issue the client a refund if we double charged them” based purely on a customer support chat conversation. Probably the closest is Adept AI, in that it has built a multi-modal model that can intelligently take action within a product UI based on the prompt.

Gap 5: Orchestration of LLMs 

Commanding and coordinating multiple LLMs or, in some cases, multiple agents to perform complex, multi-step tasks involving disparate data and objectives is ripe for investment. Increasingly, businesses and consumers are seeing the power of each atomic action an LLM can solve, but they are not seeing easy solutions for tying these actions together and effectively chaining agents into a more profitable system. 

For example, suppose you want to check your high schooler’s bank account balance and then deposit funds up to a point to help them replenish the funds, but you only want to do this if the transactions are in line with your guidance. You don’t want to add money if most of the spending is on concerts and fancy meals, but if it’s being spent on gas, school supplies, a present for Mom, and other “approved” items, then you’d like to replenish the funds commensurate with the “good” spend. This multi-step process would require multiple AI agents, each working together, passing inputs and outputs to each other and making decisions and handoffs, perhaps without a human in the loop, or maybe only at the very end to confirm the transaction. 

Each individual task could be automated, but together it’s extremely difficult to compose, with incredible amounts of custom code. The same is true for the increasingly frequent practice of taking the same question and running it across multiple foundational language models and then picking the best response or combining the responses. While these problems are omnipresent for the consumer, they are prolific for the enterprise. 

An orchestration-for-LLM-agents company would quickly become indispensable. Its intellectual property would have a lot to do with guaranteeing the inputs and outputs from each gate in the agentic system, not to mention user-friendly hooks for where human intervention is either desired (for example, logging in to banking) or required (confirming the flight plan for a commercial aircraft) and maintaining and respecting a set of rules that the user should not be allowed to override during normal prompting. 

Palantir’s AI Platform (AIP) has a “Use LLM block” functionality that allows no-code orchestration of multiple LLM actions in a series. Further, Anthropic has been working on so-called “Constitutional AI,” which gives agents a set of policies and rules to follow that allow the agents themselves to scrub and review the responses from previous LLM prompts.  CrewAI is perhaps the easiest to access solution for AI agent orchestration as it has substantial open source resources developers can build with. LangGraph, which builds upon LangChain, assists in applying saved state to orchestration.  Meanwhile, UC Berkeley has released its own project called Gorilla for connecting LLMs with various APIs, and on the earlier stage, companies like Transcend are seeking to make the connective tissue consumer-usable for everyday tasks. The prevalence of a growing set of open source options is often a harbinger for the growth of mature companies with paid offerings. 

Clearly, each of these technologies may constitute a part of the whole that is required to orchestrate a true chain of events without a lot of code or human intervention. Of course, the injection of human-in-the-loop hooks into this process or respecting policy restrictions (for example, “no withdrawals from bank account over $1,000 without a human”) will be critical for any of these agentic system products to gain widespread adoption from the public and acceptance from the third-party tools that will need to allow them to interact with their information.  

Subscribe now

Gap 6: Testing the quality and correctness of results

A major challenge in productizing LLM use cases is determining when the outputs contain inaccuracies, hallucinations, biases, or other errors. As I wrote this article, I continually asked various models to help me research companies that are working on these gaps. Interestingly, in most cases, the AI gave me names of companies I had never heard of, and I was thrilled to learn more about them. Unfortunately, almost every company it generated either doesn’t work on the problems stated or does not exist. So far as the LLM was concerned, its product “worked” and provided valid responses to my questions in the form of company names. However, the quality and correctness of the answers was unreliable. 

I find this repeatedly when researching longer-form posts like this. A lot of my prompts begin with “Are there startups or fast-moving VC-backed companies that…” and inevitably I get responses that “Google” or “Meta” do this, except neither of those companies have been startups for a long time. At least in those cases, I am receiving real companies as the outputs; frequently I receive real-sounding companies that are fake, or I get real ones but whose focus areas are not aligned with the gaps described. This is problematic when researching a blog post, for sure, but the implications are far wider and more concerning beyond writing. 

How will we know that the solution to the math question ChatGPT helps our third grader with is actually correct or that the formula used was applied in the appropriate circumstances? What happens if even more people learn the incorrect answer or learn from a model hallucination instead of learning from facts? (Perhaps adding to the problem, the LLM may already be correct more often than most human teachers when applied across the broad spectrum of human knowledge; however, it is often less correct than an expert human.) 

As mentioned above, Anthropic’s “Constitutional AI” framework could help. Additionally, the ecosystem of solutions available through Hugging Face and its associated community likely will be a strong reservoir of possible solutions, but it will take years for these technologies to mature to a point where we can intelligently confirm the accuracy of responses across a wide range of question styles. Perplexity.ai solves for this through transparent sourcing of responses, which is a great example to live up to, but it still requires the user to follow those links and have knowledge of which sources are reputable. Unfortunately, the accuracy challenge is also confounded by the proliferation of AI-generated answers in the first place.

Gap 7: Robots in, robots out

Ensuring that high-quality data feeds into our LLMs in the first place is the final gap. For now, most of the enthusiasm is focused on getting human-generated enterprise data into LLMs through fine-tuning or RAG approaches. However, the speed at which we’re adopting LLMs, combined with the incredible volume of output they can produce in seconds, suggests that perhaps exponentially sooner than we expect, most LLMs and enterprise implementations of them will be confronted with machine responses as their inputs most of the time. 

For example, we’re happy for the model to train against the successful resolution of a customer support case performed by a top human agent, but are we equally happy for that model to then enhance its training based on what it produced? It won’t take long for the AI-produced “school solutions” to dominate in these settings, particularly because the AI will continue to train on edge cases that it answered successfully and for which a human exemplar does not exist. 

It’s already difficult when models have been thoughtfully prompted and tuned to distinguish between the AI responding and a human, so we cannot expect the model itself to know the difference. This is less concerning when the model is producing correct and valid answers to concrete questions, but it becomes critically important when the models hallucinate, take on controversial opinions or biases, or attempt to be more creative than we’d like. 

It’s not hard to envision how a set of AI agents writing simple news stories on the outcomes of sporting events, for example, could start by introducing incorrect scores or readouts on how players acted, and for that content to get picked up innocuously by broader AI agents attempting to summarize events across a whole sport, and before we know it, a fan who caught a foul ball as a spectator is now being written about as if they play third base and have a $5 million a year contract with the Houston Astros. If we wait to solve this problem once everything is published, it will be too late to fully unravel the inaccurate narratives. 

For the time being, it’s straightforward to address this in business settings by explicitly not including AI responses as inputs for training or creating embeddings, but soon it will become impossible to distinguish as more primary-source writing gets done by machines instead of humans. One company that is defining solutions for this early market is Reality Defender, a venture-backed product startup focused on tools for detecting AI-written text, video, and audio. So far, the market appears to be more energized by stopping fraud and deep fakes than on data pipeline monitoring, however. 

For example, Reality Defender describes a situation in which someone records your voice without your knowledge in a regular everyday conversation, trains an audio agent to sound like you, and then passes your bank’s voice-recognition test over the phone to then take control of your finances. My hope is that we will begin to label all data sources and put them on an independently verified blockchain at least under the hood, but even that process can only produce a probability score that the information is AI-produced, not true certainty. 

Subscribe now

Concluding thoughts

Generative AI has taken the world by storm, and the wide range of its applications is impressive but nascent. For practitioners, it is clear that investment around the developer experience when using LLMs lays bare numerous serious gaps. There are plentiful others we did not cover, for example latency in getting responses from the foundational models or rapidly training on enterprise or personal data sources. Investors and founders alike will take note of plentiful opportunity, while current builders can gain camaraderie from knowing that their struggles in implementing intelligent, text-writing generative AI are not unique. For now, we recommend:

• Stronger prompting

• Custom code wrapped around model inputs and outputs

• Testing the wide variety of open source tools

• Experimentation with new functions from the foundational models

• Experimenting with the new products from growing companies supporting these gaps

Taken in concert, this is currently the best path forward, and hopefully we’ve also inspired some new founders and investors today! 

Share Emergent - Product Newsletter and Podcast

Hi! If you enjoyed these insights, please subscribe, and if you are interested in product coaching or fractional product support for your venture, please visit our website at First Principles, where we help the most ambitious founders make a difference.

The best founders and product managers spend considerable time and energy assessing whether new product concepts are valuable. However, the value of products to sets of customers is not binary; each product has what I call a Success Quotient. Through spending time launching and helping many founders go from idea stage through first revenue, I am lucky to have observed certain patterns. Namely, it’s easy to see when you have a bird’s-eye view into the process that a variety of Success Quotients exist for different products deployed in different markets. 

In this post I’ll establish some semi-quantitative thresholds to help you know if you should proceed or whether you need to revisit the drawing board. While this post can be applied to new product features, its main function is to describe the process of assessing early product-market fit for new products or new offerings that complement existing ones. 

What is the Success Quotient?

Let’s define this new term: the Success Quotient for a new product or service is the number of people, from your target market, who have the pain your product solves, and who are immediately willing to adopt the product after being exposed to it. I like to express these as ratios because it feels strange to convert actual human beings into percentages. 

Typically I use:

• X out of 10 for enterprise products

• X out of 100 for B2B SaaS products

• X out of 1,000 for consumer products 

If a product has a Success Quotient of 45 out of 1,000, this means that if we show our product to 1,000 people who represent our target market, and they have the pain our product addresses, then 45 of them will install it, ask to try it out, subscribe on the spot, or, in the case of enterprise products, request a follow-on meeting that advances the sale. 

I want to clarify that the second number in the ratio does not represent the total viewers of your Product Hunt post or recipients on your cold email list. It specifically refers to individuals experiencing the pain point your product addresses, whom you’ve engaged with directly and effectively. This engagement could be via personal emails, one-on-one conversations, or formal marketing channels. Importantly, we’re excluding those who don’t experience the described pain or who fall outside of our target markets.

Why does this number matter?

This quotient matters for two reasons. The first is that your Success Quotient gives you an idea of how big your market is, which can give you an idea of whether your price and marketing and customer acquisition costs are viable for your product. For example, if you serve a market of 5,000 specialists and have a Success Quotient of 10 out of 1,000, you can expect to max out at around 50 total paying customers. In such an example, it’s easy to see how your price point would need to be many millions or even tens of millions of dollars to justify venture-funding returns. For most products, that’s too expensive, and therefore this particular mix of product and solution are likely a bust. 

The second reason it matters—and this is arguably the more important of the two—is because Success Quotients are sticky anchors. What I mean is, whatever your quotient is when you first launch your product, don’t expect it to rise massively after you raise lots more money and improve the product. You heard that right, and this is the tough-love part! It is extremely rare in my experience that a product that launches with a quotient of, say, 5 out of 100 will become a product that achieves a 50-out-of-100 ratio. You might go from 5 to 10 out of 100, or from 315 to 375 out of 1,000, but product solutions tend to anchor hard toward their initial uptake rate. 

How can this be?

It’s quite simple: the Success Quotient is measuring the value of your holistic solution matched against the value of solving the pain (for more on this, see my post on quality of the solution vs. quality of the product). This is because we’re measuring the degree of excitement target users have for what we’re offering and demonstrating. They are not, at this phase, parsing the exact features, so evolutionary improvements in those features over the coming years won’t dramatically impact this particular quotient. 

Sure, we’re simplifying the issues of retention, viral growth loops, and much more, but I want to keep this simple because it’s easy to delude ourselves into thinking we’re on the right track when we’ve invested so much of ourselves to get to launch in the first place. For now, these other considerations can wait, as they measure challenges that come after that initial adoption moment. In a way, it’s a measure of whether would-be users believe that the initial feature set and vision are compelling ways to solve our problem, and whether they believe the problem is even worth solving. The Success Quotient captures all of these dynamics at once. (To be sure, some products make massive leaps forward merely by updating their positioning, their messaging, or their offer, and those are worthy causes, just not the focus of this article.)

What thresholds demonstrate success?

It’s easy to find companies bragging about how many users they earned when they first launched on Product Hunt or similar places, but it’s quite difficult to find publicly referenceable examples of what the actual ratios looked like, since they are frequently not launching just with their target users. I’ll do my best to share some public examples, and then I’ll need to ask for your trust based on what I have seen. Suffice it to say, the companies that post big numbers on first release are at the top of the spectrum I’m about to describe, so if you have a multi-thousand-user first day, or a giant waitlist for your B2B or enterprise product, you’re in a great, strong position. The question is what to do or think below those ranges.

For example:

• Instagram acquired 25,000 users in one day upon launching in October 2010. It had 100,000 downloads by the end of the first week, and 1 million by mid-December.1

• Slack earned 8,000 requests on its first day and 15,000 by its second week, and it grew at a rate of 5% to 10% every week for the first year.2

• GitHub earned 6,000 beta signups when it launched in April 2008.3

• And of course, the one to shame them all, ChatGPT from OpenAI won over 1 million users in its first week, 57 million by the end of its first month, and 100 million not long after.4

Again, each of these examples does not tell us the ratio, but at some level, if the absolute uptake number is in the thousands, you’re probably doing just fine. Here is how I think about each of the thresholds across each of the business contexts

Share

To make these numbers come to life, this means, for example, that if you pitch 10 prospects who have the stated pain your product solves for your enterprise product, and if only one or two of them are asking for follow-up, you’re probably not yet on the money. Whereas if you can get three or four of them to move forward, this is workable and you likely can build a business. The bottom two categories are both amazing places to be, and now you simply need to live up to the hype of your own offering. 

These are not concrete numbers, and this is not meant to be a scientific measure. It is possible to win starting from lower than these thresholds, and I’ve also seen companies that have never pitched their product because word of mouth grew them through the first few stages. This is startupland, and anything is possible, but it’s also important to know how close you are so you can decide how to fix the problems when they exist.

My numbers are dangerously low—now what?

First, you’re in incredibly strong company. You are not a failure (unless you ignore the next steps). No one writes about this, but it is in fact the most common place to be by volume of shots on goal. This is controversial advice, but my suggestion is to try a new solution, a new problem statement, an adjusted market, or some combination of these. 

The conventional wisdom is to spend more time iterating on the existing solution. I’m a huge fan of iterative software development, with customers deeply in the loop; however, that is the best plan only after you establish a strong anchor starting point. It’s not impossible (just very unlikely) to iterate your way from the danger territory into the higher uptake rates, but where I believe most founders and product people go wrong in these situations is moving straight to iteration, which incentivizes thinking small. To get from dangerous territory into “strong to quite strong,” you need to move big levers (see my post on data-driven vs. savant-driven first insights). 

A great example is how Twitch was founded. The initial traction for Justin.tv, a streaming service that followed the founder around 24/7 and allowed people to upload any content, was not seeing the strong uptake required for sustained growth. Instead of trying to make a progressively better feed of Justin or make it easier to upload more types of content, the team paid attention to the value users were deriving from the product and identified that gamers didn’t have an easy way to livestream their gameplay. This was a big shift, and one large enough that Justin.tv made the transition to Twitch, which we now know as a household name.

When I say iteration isn’t the key to solving this particular problem, I don’t mean that you should stop talking to users! Quite the contrary, in fact—talking to them is the best way to derive these big insights that you need to unstick your sticky anchor. It may be that you need to solve a different problem, or solve the problem with a superior solution or for a different market. Or, if you have nearly unlimited capital and time, you can definitely try to iterate past your sticky anchor; just don’t expect that process to yield fruit on a timeline of weeks or even months—expect it to take years.

Remember that the path to a successful product is rarely a straight line. Embrace the journey of discovery, pivot when necessary, and stay committed to solving real problems for your target audience, as this resilience and adaptability are what truly differentiate successful ventures in the long run.

Subscribe now

Hi! If you enjoyed these insights, please subscribe, and if you are interested in product coaching or fractional product support for your venture, please visit our website at First Principles, where we help the most ambitious founders make a difference.

In a perfect world, software companies would avoid critical product performance issues by using automated testing. However, few companies have that time, money, or cultural discipline until they’ve raised tens of millions in venture capital or have at least millions in annual revenue. I’d love to see more teams set up great automation frameworks early on, but the reality is that this work trades off directly against finding product-market fit. It’s hard to know what tests to write or how hard to work on them when you’re not sure which features will be in the product tomorrow. As a result, it’s somewhat inevitable that startups building novel solutions will experience not one, but many, moments like I’m describing. Here are the hallmarks:

• Customers are complaining, loudly, that the product doesn’t “work” the way it did in the demonstrations—or they aren’t talking to you anymore.

• User adoption is stalling even though you know you’re solving an important problem with a strong solution and are signing up new customers.

• Engineering knows there’s significant tech debt and feels like no one is paying attention.

• Product (and QA if it exists at this stage) is overwhelmed in attempting to clarify all the conditions where issues are occurring.

Solving these challenges is never easy, but after experiencing such moments in nearly every startup I’ve worked at or advised, it’s clear that there are some learnable patterns for making the badness go away while other approaches can cause it to linger for months or even years.

First, set user-experience latency goals

Even the best engineers I’ve ever met won’t want to do this at first, but trust me, this is where you want to start or you’ll burn two weeks just to return to this moment. Engineering will say that logging and dashboards are already in place, but frequently those metrics are not tracking the slowness, or latency, in the system as experienced by real people. It’s great that requests to the API return in 237 milliseconds, but if the user can’t load important content for 37 seconds—even if the problem is based on that API call—the issue won’t get solved, and engineering and product will talk past each other.

To be clear, great debugging requires both styles of measurements, but the first step is to set user-experience-based performance goals and then work backward. These can sound like:

• 90% of searches with five or fewer keywords return in one second or less

• 99% of analytics dashboards with three or fewer parameters load in two seconds or less

• 95% of all page loads take 1.5 seconds or less

Hopefully you’re noticing two commonalities. The first is that we never say 100%, because there’s too much entropy in complex systems to guarantee perfect performance, or at least the time needed to get there is generally not worth the tradeoffs required. Second, we’re being precise about the inputs. We’re defining conditions that we know are common, based on our usage analytics, such as total number of keywords, parameters in a given dashboard, or styles of queries etc., so that we can define the experience as the user is likely to encounter it. We’re also setting targets for how fast those actions should be or better. We’re always happy to have better, but the surge to fix the issues can come to a rest when we hit these targets. Without such concrete targets, performance debugging can be endless, and it quickly hits diminishing returns for the business and for engineering morale.

In case you’re struggling to establish benchmarks, I’ve found that in consumer products, pretty much all interactions need to be less than one or two seconds, and in emerging-technology use cases: <1 second feels snappy, <3 seconds feels solid, <6 seconds feels acceptable but not great, and <15 seconds is about the upper limit for the user not becoming totally disengaged. If you know you’ll be above these thresholds no matter how hard the team works, find a user experience solution (for example, “Email me when my dashboard is ready”).

Additionally, isolate the conditions in your experience thresholds:

• Which environment(s) does this need to work in?

• On which data sets and at what scale must this work?

• With what level of user scale? How many of the actions described will occur per user per unit of time?

• What user or customer-specific parameters are expected to be invoked in these circumstances?

Repeat this for as many core parts of the product experience as are necessary to declare victory on these critical performance issues.

Subscribe now

Create a dashboard for the chosen metric(s)

Every member of the team must have the ability to see the progress against the goal. If you have sufficient usage of your product, the dashboard itself will have enough data flowing through it to let you know how close you are to success and which approaches are or are not working. If not, you will need to build some load simulation tests at least for your core experience goals.

This dashboard step is critical so that it’s not one person out in the field saying things are slow and everyone else back home logging into a different environment with, for example, low-scale data or less resource contention, and therefore you’re not speaking the same language. When the ground truth is not aligned, the urgency and level of hard work necessary for success don’t materialize.

Generate a set of hypotheses

Once the experience goals are agreed upon, engineering needs to brainstorm a set of hypotheses for why the performance is the way it is. These need to be documented for everyone to see. This aligns engineering debugging effort and accounts for the fact that our first hypothesis is rarely correct.

It also protects against a mentality I’ve seen even among the best, where, after the current hypothesis is checked and disproven, the debugging stops due to a lack of continued creativity as to the source of the issue. When the current hypothesis is clearly articulated for all to see, teams can run one idea fully to ground, rule it out, and then move on to the next one or even parallelize debugging when it’s a team effort.

During this phase, it also makes sense to identify that next tier of performance logging that will assist in the debugging. Based on the hypotheses, what would the team need to be able to measure to form a complete stack of knowledge, end to end, of how the problem manifests? This is where system- and API-level transparency play a massive and critical role.

The only reason to delay this step until #2 is because it takes too long to instrument every portion of the backend systems (if that’s already done, kudos to you and your team). Now your team can trigger the conditions users have experienced that led to the slowness and the engineering team can analyze and inspect the hotspots. Inevitably, this leads to more hypotheses, which should also get documented and assigned so the team can make systematic progress.

Assign your best problem solver, not your best engineer

Frequently when we encounter challenges like these, we look to whomever built the piece of the system we think is the problem to identify the issues. While this occasionally pays off, the reality is that not all engineers are great at finding and tracing systemic, multi-cause issues to their sources. When you have people with both sets of talents, double their equity and increase their comp, but in all other circumstances, ask yourself: Who on the team has the best instincts for how the pieces of the system come together and doesn’t stop until they find root causes?

Frequently this is someone who doesn’t take anyone else’s ground truth for granted and views themself as something of a jack-of-all-trades but is also adept at wading through lots of code and understanding how parts make up the whole. This debugging captain will need help from team members who are more expert on certain aspects of the code base as they proceed, but it’s crucial that the engineer driving the investigation has the correct personality traits.

Set a daily sync on the issue

If it’s costing the business dearly, then it’s worth it to take 10 or 15 minutes each day, outside of other meetings, to check in together on the debugging progress. This is an engineering-led meeting, but it likely has participants from across product, engineering, support, customer success, and any other tactically relevant stakeholders. It’s not for the CEO, the customer, or anyone else who isn’t part of solving or communicating directly about the solution.

These are not rigid Agile standups. They are a time to review progress against the debugging, make requests of teammates, check alignment against the list of hypotheses being tested, or even brainstorm new options to check. 

Don’t give in to “but there’s no repro”

If the metrics from step one are clear that there’s a serious problem in terms of magnitude (for example, painfully slow) or impact (slow for a critical client in the middle of expansion negotiations), then it’s entirely unhelpful to give in to the mantra of “I can’t do anything until this issue has a set of clear issue reproduction steps.”

It’s lazy for product, customer success, or QA to fail to provide repro steps when the conditions are well-understood, but it’s lazy for engineering to dig heads into the sand when there is clear proof of an existential issue merely because the teams closest to users can’t explain the exact conditions that cause the problem.

Let me be clear—what I just said is a wildly unpopular view that will lead to product-engineering tension, but it’s tension the team must work through together, not ignore in situations of great crisis. To unlock better thinking at these impasses, I find it can be helpful to zoom out with engineering leaders or debugging captains: “What in our architecture could be contributing to this?” or “What are all the steps required to complete this action under the hood?” “Which ones are most susceptible to variation?”

Having this discussion while simultaneously providing all possible signal on user behavior can often lead to the missing epiphany. When all is lost, someone needs to figure out every single part of the architecture that is implicated from the logs and start reading code and examining hotspots under load until new hypotheses can be formulated.

Subscribe now

Dig systematically until the root cause is uncovered

Many issues can be fixed without getting to root causes, but this is shortsighted. As a matter of discipline, insist that the debugging captain write down the root cause on the same document where the active hypotheses are tracked. If the team doesn’t know the root cause, someone must keep digging.

It’s a cliché, but the deeper you can go, the higher-leverage the fix can be. You won’t necessarily fix the root cause, but you need to understand how the issue will manifest again if you don’t, and be willing to accept that those conditions are sufficiently rare.

For example, I once worked on a product where the number of security groups an enterprise account could create impacted search times. Most clients had one or a few dozen security groups. The marquee client experiencing the crippling instability had almost 100,000, and the next most was about 1,000. In this context, it did not make sense to re-architect the system to accommodate 100,000 security groups, but we did need to come up with a fix further from the root cause that would work for that specific client’s circumstances.

Resist the temptation to refactor it all

An additional benefit to having a debugging captain who is not tied to code they wrote is that once the root cause is known, the team can have a thoughtful discussion about exactly how much needs to be fixed to deal with the problem and protect the product in the future.

Unfortunately, the same trait that leads great engineers to want to constantly improve the code they see is also the trait that causes root-cause analyses to go too far. We will never have perfect software systems at this phase of debugging in a company’s journey—or if we do, it will almost certainly come at the cost of not building product features that have value to our end customers.

The symptom of this fail mode is when a small fix will do but the team undertakes a massive refactor of a critical portion of the code base without a serious discussion with engineering and product leadership about whether it’s worth it. Sometimes it is! 

However, I find that frequently what happens instead is that someone starts the rewrite, does not actually fix the bug right away, and then weeks or months go by and the issue persists, causing massive damage to both the business and the trust between product and engineering personnel. Meanwhile, the refactor breaks more critical pieces of the system because the company’s architecture and test automation weren’t evolved sufficiently at this phase to support such deep changes without starting a chain reaction of bigger problems.

In short, be deliberate about how much needs to be fixed. My rule of thumb for venture-backed companies is to make sure the fix will carry the team through about one or two orders of magnitude increase in usage/customers/complexity, as this generally means that the issue is unblocked, the team can blitz-scale, and the issue can be “fixed” again when engineering and product talent is more prolific after the next raise.

Product should be involved, a little

These tradeoff decisions are part of why product can’t be entirely hands-off. A lot of popular product practitioners will tell you that this post doesn’t even belong in a product-focused newsletter because fixing these issues is theoretically engineering’s job. 

I agree that most of product’s time should not be spent on this, or else many other important aspects of the product job will suffer; however, product serves a few key roles here:

• Identifying the problem and establishing what the user-experience performance thresholds need to be

• Communicating with customers or with customer success about the progress in resolving the challenges

• Weighing in on critical tradeoffs about how long to spend on this style of debugging, with regard to what risks are acceptable and therefore how deep of a solution is needed

• Being available to answer key questions about the benefits and pitfalls of different proposed solutions and how those will impact customers

I have rarely sought to spend significant time on performance debugging issues as a product leader or PM, and I don’t encourage product managers to wade into this zone of work if it’s not necessary. However, it’s hard to miss that the organizational and root-cause-thinking traits that lead to product success can be a true asset to engineering in these stressful all-hands-on-deck situations.

With this in mind, my recommendation is for product to be proactive and willing to dedicate at least a little time per day to supporting engineering in these crisis situations, as it helps everyone solve faster and get back to the value-generation work.

Subscribe now

Hi! If you enjoyed these insights, please subscribe, and if you are interested in product coaching or fractional product support for your venture, please visit our website at First Principles, where we help the most ambitious founders make a difference.

If you’re building a product that uses OpenAI’s APIs under the hood, and if you want to have a long-term viable business, then you need to be charging your customers at least $113/month. I’m sharing the research that informs this figure for three reasons:

1. First, in our justified exuberance for shipping fast with large language models (LLMs) like ChatGPT, it’s easy to get intoxicated to the point where we forget that part of Product’s job is to assess the business viability of a solution, and OpenAI’s API costs add up a lot faster than they seem.

2. Second, from what I’ve experienced, a lot of product managers have forgotten how to methodically determine the unit economics for a product, and OpenAI provides us with a timely and relevant example.

3. Third, the fact that the number is over $100/month teaches us that many use cases won’t be suitable for LLMs (for the time being).

For anyone not concerned with #1 and #2, scroll to the bottom. I made an easy calculator for you to use with your unique product inputs.

PLEASE CHECK: https://openai.com/api/pricing/ as PRICES CHANGE REGULARLY

Subscribe now

So how did I get to $113/month? Let’s build up the math one step at a time:

Understanding inputs and outputs to LLMs

1. We know that LLMs take a text input prompt (“input”) and they return a response answer (“output”). OpenAI charges its API customers separate prices for these inputs and outputs.

2. The price is based on the number of tokens in the input text and the number of tokens in the output text. But what the heck is a token? In short, a token is a sequence of characters loosely correlated with a word, and it is how the computer encodes meaning to the words and the text that we type. The computer, however, is parsing meaning more granularly than us, so some shorter words count as just one token, and some more complex ones are multiple tokens.

3. In essence, though, we’re getting charged based on how much text goes into the prompt submitted to the OpenAI API, and then again based on how much text the AI returns in its response.

But it gets more complicated…

1. The intuition is deceptively simple, however. For entirely user-generated requests, this would mean smaller submissions in the form of questions from the user and then longer responses back from OpenAI. Consider that “Where should I eat in New York City?” is about 9 tokens long, but the response I get is 293 tokens long, and it covers many options around the city, so there is pretty serious asymmetry between what requests will cost versus responses.

   1. It gets more complicated, though, because most apps using OpenAI are doing extensive “prompt engineering” to fully articulate all the parameters that describe how the AI should respond, including providing background, personas, and even examples. The actual prompt length likely requires a discussion with an engineer to understand. It might not be obvious. For example, the product manager might think the input cost is the number of tokens in “Good date-night ideas,” but each user prompt like this might be wrapped in an envelope of 1,000 tokens’ worth of additional context. The longer the prompt that surrounds the user’s input, the higher the cost for every single transmission going in.

2. So now we need to understand what a reasonable rule of thumb is for how long a question might be and how long a response might be. This is where I can give you a rough idea but can’t do your homework for you. For prompts with very little adornment, if you will, it feels sensible to me to assume 25 tokens going in. And for responses of modest complexity coming out, 500 tokens feels like a safe average bet.

Now let’s convert into dollars…

1. Great, for OpenAI’s GPT-4 API, that’s $0.03 per 1,000 tokens going in and $0.06 per 1,000 tokens coming back out. We’ll need to divide our total by 1,000 later to account for this. (Note: as of 7/25/24, the prices are now $5/1M input tokens and $15/1M output tokens, which roughly drop the prices quoted in this article by about 4x. The calculator at the bottom has been updated to reflect these new default values.)

2. Now we need to estimate how many requests and responses a user will catalyze each day that they are using our product. Let’s assume they use the product every day of the year. For a search-based product, 30 searches per day doesn’t feel unreasonable.

3. For a single user of the product for a year, that’s 273,750 input tokens and 5,475,000 output tokens. When we divide by 1,000 tokens and multiply by our per-1,000 token costs, we get $8.21 per year for the inputs and $328.50 per year for the outputs, or a total of $336.71 per year for OpenAI usage per user.

…and account for proper business margins

1. But this isn’t the end of the story, because we don’t build products with zero margins or we go out of business. In fact, if the cost of goods sold (COGS) is outside of the 20% to 30% range, we’ll certainly raise eyebrows, and that’s assuming that OpenAI is the only cost for what we are selling. Assuming it is, we need to sell our product for at least $1,346.85 per year.

2. In monthly SaaS terms, we’re talking about a plan that runs $112.24 a month, or for simplicity about $113/month. Anything less than this may not prove to be viable for the business.

Every team with a unique product will have different usage patterns in terms of how heavily it implicates OpenAI. For example, a tool that summarizes emails might get 5 to 10 summary requests per user per day and might have fairly short questions (“summarize this”), except that the prompt itself also includes the email, so that’s probably at least 1,000 tokens for a modest thread among colleagues for each input. You’ll have to assess who your users are, how they are interacting, and how much prompt engineering is done. Further, OpenAI has multiple flavors of its API. GPT-4-32k is double the cost I just showed, so if you need that one, double everything here, up to $226/month. Meanwhile, GPT-3.5-Turbo is 3% of the cost of GPT-4. The point is that these details matter, they determine what use cases are business-viable, and PMs should be asking “which GPT do we need to support our use case” and “how hard will we hammer it?”

To make this easy, I’ve built out a simple calculator you can play with to test your assumptions, along with a variety of prompt and response examples to help you choose the right number of tokens.

Try the calculator

Hi! If you enjoyed these insights, please subscribe, and if you are interested in tailored support for your venture, please visit our website at First Principles, where we focus on product to help the world’s most ambitious founders make a difference.

I was working with a product manager last week to generate a welcome-email experience for a new product, and, I’ll be honest—the first draft was uninspiring. I gave the PM direct feedback about each message, but I couldn’t shake the idea that maybe there was a better way to upskill the PM. I decided that what I needed to teach wasn’t how to address my concerns, but rather how to teach good product judgment. Before we dive in, I’d like to distinguish product judgment from product sense. To me, product sense is the ability to choose the right problems and the right solutions to those problems so that products can grow faster with fewer failed experiments. Meanwhile, product judgment is how we make the hundreds of smaller decisions we are required to consider when crafting something of true value to the market. Whereas the former might include questions like “Should we build a chatbot?,” the latter speaks more toward one level down: “What pre-seeded questions should we show the user to get them using that chatbot?” In the case of the welcome-email journey, the feature set for MVP launch is well settled, and it’s now time to figure out what we should say about them to onboard and excite the user. While I’m focusing on email journey content in this post, the model I’m about to share is one I use in nearly all smaller product decisions regardless of style or impact.

Draw from Experience

What do we know from our own experiences already about the problem at hand? This sounds so simple and obvious, yet we often jump straight in and forget to first reflect on what we’ve already learned. In our last product launches, for example, what sorts of welcome emails had high open rates and/or high click-through rates? Which ones did not? Our own personal experiences are far from comprehensive, but typically we already know something about what works and what doesn’t from what the market has taught us, and this should be the first reservoir of knowledge. This guidance can also be construed to mean “check your metrics,” and reflect on what we already know about our users and their needs that might inform how we tackle the problem at hand.

The voice in my head reacted as follows for this specific challenge:

• Keep the messages very short

• Focus on one or maybe two topics at most per message

• Try to make it personal

• Spend real time making the first sentence strong

• Include a graphic or animated GIF to clarify how to create value

Research What’s Out There

Most products need to be novel in only about 10% to 20% of what they do to be massively successful. For example, if your product is a new AI-driven chatbot, then its core value isn’t about innovating on payments. In this case, you don’t need to break new ground on how payments are implemented. In fact, it will increase cognitive load on your users to change patterns where you’re not trying to get noticed. With this example, when it comes time to implement payments, spending a few hours researching how the best companies handle payments and modeling your product’s experience off theirs is entirely sufficient. The mistake I see, however, is trying to divine how payments ought to work based entirely on real-time opinions and feelings. This takes precious time away from the critical and novel 10% to 20%. Frequently the best products have already run dozens of A/B tests, conducted thousands of hours of user experience tests, and hired some of the best product people and designers on the planet to get these answers. 

If the work we need to do isn’t part of breaking new ground, research and find inspiration from what’s already working well in the market and adapt it to your own needs. In my onboarding email sequence example, this would mean signing up for 10 popular and well-regarded products and analyzing in great detail how they each perform onboarding, and then assessing which aspects are worth copying for your product. How long is each message? What do they emphasize? How long do we get between messages? Do they use images or links to videos? Is it from the CEO or someone else? And so on…

Let’s look briefly at the first two emails Pipedrive sends as examples:

Here’s my internal monologue when I review the above examples:

• Very light on text

• Clear CTA to get into the app

• Reminds user of value proposition at a high level

• Gives concrete suggestions for how to start

• Not very personal, and how do I get help?

• Repetitive use of “Welcome to Pipedrive” in subject and first line of message

• Second message comes one day after first

• Second message focuses directly on training

I would take this process through the first few weeks of content across at least 5 to 10 well-run, well-loved company flows and compile all my notes before writing much myself.

Consider the First Principles

If you take only the first two steps above, you’ll get a decent outcome, but not one that perfectly fits your product. It will be polished and reasonable, but it will be mimetic and might not be great. The final big element to add to the puzzle is to consider the first principles that undergird the challenge at hand. First principles are the underlying assumptions beneath our challenge. In our example, this would sound like asking ourselves:

1. What is the purpose of a welcome email in the first place?

2. Do people even read welcome emails?

3. What should every welcome email contain in terms of content?

4. What knowledge would users of our product need to have on Day 1 to be successful?

5. Is a welcome email the best mechanism for reaching our customers when they sign up? Why or why not?

These first-principles questions will guide us toward product-specific thinking that will dramatically improve our chances of making the right first decision, tailored to our specific circumstances.

Subscribe now

The People Matter Too

Finally, we need to think about the people who have to love what we’re doing for it to be accepted and who have intelligent opinions. For the lucky few of us building in small enough companies that can focus only on the customer, fantastic, but for all the rest, our work products won’t make it out the door even if we do all the previously recommended steps but forget to consider the people who need to approve of the work. This might be the CEO, the VP of Product, someone in sales, an external consultant, or a whiny member of the team who gets a lot of respect because they are often right. It doesn’t matter who; what matters is that we spend some time thinking about their interests, the intelligent critiques they might make, and what they expect. Frequently these outside views give us tremendous signal about what we’re missing from our vantage point. Even if we don’t change our judgments based on how we expect them to react, we will at least be armed to explain why we did not and to have considered their interests. As organizations get larger, these groups of “stakeholders” get larger and more cumbersome to corral, so I’m not suggesting that we water down our judgments to the lowest common denominator. However, if we know, for example, that the CEO likes or hates a certain turn of a phrase or cares deeply about a particular value proposition, we would be wise to consider it from the start.

In this case, I knew the CEO would be thinking about:

• Making the user aware of the product’s free trial option

• Giving them an example to use in the context of the free trial to build trust

Summary Thoughts

In summary, developing strong product judgment is an essential skill for product managers who aim to make informed decisions that grow their products. It’s a process that begins with reflecting on personal experiences and analyzing metrics to draw upon past learning. Then it involves researching existing successful products to adapt proven strategies, followed by delving into first principles to ensure decisions are solving the right problems in a way that is tailored for the specific product. Lastly, it’s crucial to consider the perspectives and preferences of the reviewers within the company. By integrating these approaches—experience, research, first principles, and intelligent considerations—PMs can refine their product judgment and navigate the myriad decisions they face with confidence and strategic insight, ultimately contributing to more successful products.

Subscribe now

Hi! If you enjoyed these insights, please subscribe, and if you are interested in tailored support for your venture, please visit our website at First Principles, where we focus on product to help the world’s most ambitious founders make a difference.

The best product feature tickets do not slavishly follow a framework or checklist that makes them great. While they frequently contain common elements, what makes them powerful is how well they are understood by the engineers reading them. Determining what information to include and, perhaps more importantly, what not to include, is messy and imprecise. It’s a judgment call based on the product manager’s assessment of how much each developer needs to see included to execute successfully. Too much detail stifles creativity and innovation, and distracts attention away from value and into the process itself.  Too little information leaves engineers spinning their wheels guessing at intent and never knowing when they are done. 

Deriving a great methodology for writing a ticket, then, is not about living up to a universal rubric, but rather crafting different levels of fidelity to speak individually to whoever is consuming the content. A great ticket is for the engineer writing the code, and it drives clarity on what needs to be done and what doesn’t. It’s about communicating precisely at the exact level that builds an appropriate bridge in the developer’s mind, which then frees them to do excellent work and their own creative thinking.

Subscribe now

Let’s take this to extremes on a mundane example to make the point. Both of these tickets are perfect, depending on who is reading them.

Slim Ticket

Suppose we’ve just had a live whiteboard session in which we have jointly discussed all of these particulars, and the work is commencing right away so there is little chance of forgetting. The developer is experienced, organized, and detail-oriented and took their own notes. Then the ticket above is just fine.

At the other extreme, perhaps the developer is a brand-new engineer straight out of college, the work is being done remotely, and we have not had a chance to discuss expectations for the feature. In this case, the second option below, which is a more thorough treatment, might be right. The truth is that it’s unlikely that most product managers will choose the right level of fidelity for ticket writing out of the gate until rapport is built and everyone has had a chance to ship some features together.

Heavy Ticket

Most of the time, reality lives somewhere between the two extremes presented above. So let’s break down some best practices and why they exist.

We write to avoid confusion

The first question we have to answer is: why do we write tickets at all? Really, no one likes them. Why not just discuss everything live and avoid them altogether? After all, it’s a process burden for the product manager to maintain them and for the developer to update them.  

Some very early-stage teams with high trust and great communication can forgo them, but for most teams, the ticket serves as an atomic unit of agreement about what’s to be done, and as that list grows, taking the time to craft good ones keeps everyone on the same sheet of music, makes it easy to re-prioritize and make tradeoffs, assign and track work, capture discussions, and much more.

If we are going to write hundreds or thousands of them for even halfway mature products, then we’d better have some opinions on how to do this that are backed by experience. While some companies can avoid tickets, I don’t recommend it, because that’s how you end up with a great flow but no password reset button, or dramatically overengineering something for an extra $20K in net burn.

What tickets don’t solve

Perhaps more important than the jobs the ticket serves are those they do not. Feature tickets are not designed:

• To be a comprehensive recounting of every possible detail about how something should work that replaces live person-to-person dialogue

• To replace thoughtful functional and/or technical specifications for a complex feature

• To be a list of technical steps required to build the feature (though these can certainly be subtasks)

• To be a mechanism for providing executive visibility to “management” about what work is being done

This last point is especially salient. The most common request I see from CEOs or GMs to their CPOs or PMs is for more transparency about what is getting built right now. I’m afraid to report that every single attempt I’ve seen to economize on that request by using the ticketing system for that communication is a failure. I’m sure someone, somewhere has been able to pull this off, but generally, five minutes spent writing a Slack message or an email at the beginning of a sprint in the words the CEO expects is a lot more efficient and well-received. 

The alternative, when you consider it from the CEO’s perspective, is to be told to “follow this link, open a new system, and read our jargon-filled descriptions of what’s going on, and hopefully you’ll understand it all without the context we have … oh, and by the way, we work for you, but we expect you to go through this process like it’s a goddamn mystery adventure to find out how we’re spending your money.” Boo! CEOs deserve better, and CPOs owe them more.

Building blocks for great ticket-writing

So what should go into a great feature ticket (or the conversation, if not the ticket itself)? Ultimately, each ticket represents some unit of software development work that helps us achieve our goals. That unit exists only in your head when you start. To get it out clearly, we need to consider doing all of the following:

1. Describe the problem and the current state

What issue are we solving with the code? This doesn’t have to wax poetic like the functional spec. The goal is to remind everyone of why this ticket exists, so that it’s easy to remember when reviewing progress or moving priorities around, and of course to inspire the engineer fixing the problem to come up with even better solutions. These can take the form of a user story or a job story or can simply be a statement:

• “When on a business trip, travelers want to be reminded of who lives in the city they just landed in, so they can keep in touch.” (Job Story)

• “As an administrator, I want a way to impersonate a user’s account, so I can see what they see and more rapidly debug problems.” (User Story)

• “Searches are averaging 20 seconds to return for users, and it’s causing attrition. We need to speed these up.” (Problem Statement)

2. Describe the solution

Many novice product managers get caught up in the problem statement and neglect to acknowledge that the same problem statement or job story could have dozens of different solutions, so putting a little extra energy into this aspect is worth the time and will avoid costly miscommunications and extensive re-work.

• “We’re going to take the user’s location from their mobile device, set a radius of 5 miles, and then search their native contacts (iOS only) for any addresses. We’ll also need to geocode each address so we can compare latitudes and longitudes against other latitudes and longitudes. Whichever contacts are within the radius will get sent as in-app notifications to the user showing the name and distance of the contact.”

• “Customer support has been complaining that users are reporting errors that are hard to describe when it comes to their search results. When we impersonate users, we need to see the exact same search-results view that they see, so we can identify what looks wrong.”

• “To speed up searches, we need to create a secondary index on keywords, test, and tune it. We may also need to rebalance our search indices to reduce latency.”

3. Describe an externally visible and functional unit of value that the solution will provide

That’s a mouthful of a topic header, but each word counts. This is frequently called “acceptance” or “acceptance criteria,” but I don’t love those terms because people get confused about whose perspective we mean, and it also implies a master/servant relationship among peers. After all, a great engineer can build a solution that is superior to what was discussed, and that’s definitely not a “failure of acceptance.” Our goal here, regardless of the jargon, is to describe the result in a way where multiple observers with less context than we have could see the enhancement and all agree that it solved the problem described.

This is also a good place to specify the range of effectiveness of the solution and its observable characteristics. This applies to end-user features and backend work alike. Especially in the backend world, it’s common for engineers to assume there is no visible way to show the resulting work, and this belief is almost 100% correlated with incomplete delivery of results. Take the time to brainstorm this together among product and engineering, and it will pay off immensely.

• E-commerce example for shopping cart:

  • “Add Item: When a user clicks ‘Add to Cart,’ the selected item should appear in the cart with the correct quantity and price.

  • Remove Item: Users should be able to remove items from the cart.

  • Update Quantity: Users should be able to update the quantity of an item in the cart.

  • Calculate Total Price: The cart should automatically update the total price based on items and quantities.”

• Ridesharing app driver matching:

  • “Location Accuracy: The app should accurately determine the user’s current location within a 10-meter radius.

  • ETA Calculation: The app should provide an estimated time of arrival (ETA) for the driver that is accurate within a 5-minute window.

  • Driver Availability: The app should only show drivers who are available and within a 5-mile radius of the user.”

• Algorithmic search result ranking:

  • “50% or more of the first page of search results receive Normalized Discounted Cumulative Gain (NDCG) scores above 0.60.”

• Backend service to reset a user’s credits:

  • “A POST request sent to /admin/credits/reset with the json payload { "username": "john_doe", "email": "john.doe@example.com", "credits": 0 } sets the user’s credit balance to 0.”

4. For larger efforts, explain how the pieces relate to the whole

It’s fairly common that the skills required to focus on the trees make the forest hard to see in the moment, and I see a lot of wasted effort going into trying to build the forest, when the ticket requires only a couple of trees. Give your developers that extra bit of context so they know where the bar is and how this feature fits into the larger product vision. The easiest way to accomplish this is with great functional specs that break the work into phases, and then thoughtful decomposition of the work going into the current sprint or release, where each of the related aspects get their own ticket.

For example, a common feature is the ability to share an asset with teammates or the public. This can be a giant epic or even initiative in its own right, but if the tickets are grouped thoughtfully, then it’s easy to see the progression, and developers can focus on one challenge at a time. It also makes clear what aspects aren’t being requested within each ticket, when they are presented in proximity to the others.

• Ticket 1 Title: “Share asset with named users in my organization”

• Ticket 2 Title: “Share asset with unnamed users in my organization”

• Ticket 3 Title: “Share asset with a public link for users outside of my organization”

• Ticket 4 Title: “Shared assets respect security and visibility requirements”

• Ticket 5 Title: “Ability to select custom preview of shared asset when sharing”

• Ticket 6 Title: “Add comments to shared asset when sharing”

• Ticket 7 Title: “Email notification of recipient when asset is shared”

5. Provide links to resources that will answer common questions

This one is easy, a no-brainer, yet it’s skipped all the time. Put all the resources into the ticket so it’s easy for the developer to find. This could include any of the following:

• A link to a functional spec

• A link to a technical spec

• A link to a relevant architecture

• A link to a design or clickable prototype

• A link to research

• A link to a customer support request

• Screenshots or videos of existing behavior or future states

6. Make it atomic

Finally, and this takes a career’s worth of finesse to perfect (I’m still working on it), try your best to make the work atomic. This means that it (a) stands on its own, (b) clearly doesn’t overlap with other work, and (c) bites off the smallest meaningful slice. It’s far more satisfying for everyone if tickets can be done in a few days (and certainly less than a sprint), rather than remaining open for weeks at a time, and practically this makes it easier to shuffle work when needed, versus everything getting stuck behind some monolithic task.

• BAD: “Improve quality of LLM response.” GOOD: “Retrain LLM including this new corpus of documents.”

• BAD: “Reduce steps in checkout flow.” GOOD: “Remove the cart preview step from the checkout flow, add to summary page instead.”

• BAD: “Improve search relevance.” GOOD: “Improve relevance ranking by taking recency of content into account.”

Putting it all together

In closing, allow me to first caveat that only a small percentage of my tickets live up to these standards all of the time. You don’t have to execute flawlessly here to be great. This post is aspirational more than a must-do-every-item-every-time checklist. Conversations replace much of the need to write a lot of the nuance down, but also be cognizant of the fact that we’re all juggling many things, so if the conversation covers a lot of topics, the right thing to do is to help write it down so it’s easy to reference when your engineering counterpart is in the weeds late at night trying to remember what the heck you both discussed.

And remember, it’s better to start loose with new, innovative products because it will give experienced engineers room to express their creativity, and it’s easy to add more detail later. Once you go down the path of over-specifying, teammates will expect it, and a few months later you’ll notice that your engineering counterparts are less invested and less creative as a result.

Finally, it’s worth noting that many of these rules don’t apply, at all, one level down from the tickets the product manager writes. If a subtask for “Reduce search latency to 1 second or less” is “Deploy Elasticsearch,” that is entirely acceptable and doesn’t need to be micromanaged or overspecified. My personal preference is for product to write the external-facing tickets and then leave the decomposition of engineering tasks to the engineers. Whether you choose to create clarity through tickets or conversations or both, try making your next exchanges focused on these topics and you’ll see velocity, satisfaction among teammates, and product quality rise dramatically.

Subscribe now

Hi! If you enjoyed these insights, please subscribe, and if you are interested in tailored support for your venture, please visit our website at First Principles, where we focus on product to help the world’s most ambitious founders make a difference.

Last month I received a feature deprecation message from a company whose product I love but whose emails make me cringe. I’d like to take the 231 words they chose and use them as examples of what not to do while simultaneously deriving best practices. At the end, I’ll share my full rewrite of their message. Let’s take each issue in turn.

1. If this is your first message, you’ve already failed

The message, from Motion, an AI productivity assistant company, started as follows:

Before receiving this message, I had no context whatsoever. Motion thrust me directly into the result of what was apparently an internal debate about getting rid of Focus Time. It’s not ideal for the customer to start by learning that there is an important issue that could impact them, to not be included, and then to find out that the decision has already been made, all within the first two sentences.

The first step in writing a strong feature deprecation message is the research beforehand. We must check our usage analytics and find all the users of the feature. Then we can craft an initial message letting them know that you might deprecate the feature. Bring the customer into the process so you’re not inflicting loss aversion all at once. The customer will feel heard and, crucially, you will know if deprecation is the right move and how to conduct it. If the customer doesn’t use the feature, don’t send them a message about this at all!

2. I don’t know your feature names

Here is the subject line from the original message, and because I’m not an expert in your product, I don’t know if I should spend time reading this email.

How you name your features may seem obvious because it’s in the product’s UI writing, but your customers don’t know most of your features’ names. You have to accept that feature names are inside baseball. I know objectively what the words “focus” and “time” mean, but I have no idea if I am a user of Focus Time, the feature. I need more context on what it does to determine if this message is relevant to me.

To fix this, add a brief explanation of the term in your message to remind me. Something like “Focus Time, our feature for ensuring you have enough time outside of meetings to get your work done” would work fine. Now I know what we are talking about. It’s even better if you include a screenshot of where this button is in the product so I really know which feature we are discussing.

3. If you take, you must also give

Moments later, Motion informs me that all of my “Focus time” tasks will be removed.

To summarize through this moment: I have been told that a favorite feature is going away and that I’m on my own to deal with the consequences. The product will soon have less value for me.

We must give customers a clear and painless transition plan. Later in the message we learn that “Recurring Tasks” will take on the responsibilities that used to fall to “Focus Time,” except Motion doesn’t promise to migrate my Focus Time tasks to Recurring Tasks. This is a major miss. Now I have to work hard as a customer to retain the same value I used to receive.

4. Don’t insult the user

Motion decides to insult me in the very next sentence.

What Motion is saying, implicitly, is that the feature I love is inherently low-quality (news to me), and that I am a lover of what is apparently a bad productivity experience that no one at Motion wants to maintain. Now they’ve broken the fourth wall, so to speak, in that Motion is sharing its internal dirty laundry with me as a user. I don’t care, and neither do the other users. There are plenty of ways to build empathy with your customers in a long, hard product journey, but telling them the feature they love is poor-quality isn’t the way. Also, it begs the question, if they released this feature in poor form, what else in the product is at that same low bar?

No matter how much that feature annoys your product team, you have to avoid the “we know better than you” attitude. Even if you do know better, no one wants to read that. It alienates paying users. It’s also duplicitous. You put the feature in there in the first place, which means that at some point you thought it was a good idea. We know that and you know that, so framing the change as “we think the old workflow was dumb” just makes the user read “this product that I pay for thinks I’m dumb.”

And then they do it again. Apparently what I like isn’t “highly requested.” Facepalm!

5. Proofread mass messages for clarity

Finally we arrive at a paragraph that explains what’s going on. It’s great … until we get to the part about “we don’t think this is necessary.” Now I’m really confused. My favorite feature is going away. There’s a replacement in another feature, but the team doesn’t think I should use the replacement?

Surely Motion meant something else here, but I have no idea what that is, so now I’m sitting wondering, do I ignore this altogether or is something bad about to happen to my data? On a good day, I’ll send a message to support and ask; on a bad one, I’ve already started looking for a competitor that can solve this challenge without all the subtle insults and lack of clarity.

I’m left wondering, why didn’t Motion automate this transition? If Recurring Tasks is so much higher-quality and an approved replacement, then why do I have to do a solo transition?

6. Hold my hand

Let’s say I get over all that. How do I start using Recurring Tasks? The underline in their message, frustratingly, is not a hyperlink to a user guide or a video. It’s just a plain old underline. Boo!

We often forget that our customers don’t know the inner workings of our product. This is a painful lesson for a product team or founder, but even your strongest power users for the most part have about 1/10th the knowledge and familiarity you have.

Now I’m angry, and I don’t even know for sure if this message affects me. I know I have some specialized tasks, but are they Recurring ones or Focus Time ones? How would I know? Where in the app do I go to check if I’m impacted? If I am impacted, is there a video, tutorial, migration assistant, or some documentation to help me translate my old Focus Time tasks into Recurring Tasks?

7. Only solicit feedback if you want it

Toward the end, Motion’s message comes with a disingenuous opportunity to provide feedback. Why would anyone who loves this feature contact the company? They’ve already been insulted, and the message is so firm about Motion’s intentions that it’s clear that the team won’t care if someone does want to keep this feature.

Sure, they will help us migrate if we write them a message, but they’ve made a unilateral feature decision, didn’t solicit input, and we’ve been given no transition path. Their minds are made up. This sentence is insincere at best.

On the flip side, deprecating features is hard. It takes guts. Most teams don’t do it and they create creeping monoliths of badness. So I applaud them for taking action, but the way it’s done matters. If the decision has been made and user feedback doesn’t matter here (or if it’s already been collected in a smaller forum), don’t offer to hear my “concerns”; just say this is what you have to do, that you’ve consulted the community, and you’re here to help with the transition. Honesty really is the best policy.

8. Title your message for impact

Titles frequently are wasted real estate, yet they are the most important real estate. If I wasn’t a power user, I would not have even opened “Upcoming Changes to Focus Time,” which is the title the message begins with.

Instead of “Upcoming Changes to Focus Time,” how about “Focus Time Is Now Recurring Tasks”? With this new title I can easily discern if this is for me and the “so what” or the reason for the message. This also frees up Motion to focus the message on what’s being given and not on what’s being taken away. The headline should communicate that Recurring Tasks is now badass in new ways, not that Focus Time will be removed in the night when no one is looking.

9. Tell me WHY!!

Next, explain why your understanding of the world has changed. What has Motion learned that I should know? Why does a Recurring Task serve me better? In this message, it would have been as easy as “We’ve noticed that Recurring Tasks solves the same problems that Focus Time does, and by combining these into one, we know we can improve the reliability and usability of the experience for everyone.”  

10. Non-accountable sign-off

Not every message has to come from the founder, but please empower your PMs. Let the team that is working on this sign with someone’s name. It shows that there is a real human behind the decision, that you’re proud of your decisions, and that you care. When it’s written this way, with generic statements about support and understanding, it reads like AI wrote it to hide the ball.

Subscribe now

My Rewrite 

For comparison, here is the full original message from Motion. And here is my two-part rewrite based on the guidance above.

Part 1: Feedback Message

Subject: Protecting Your Focus Time

Hi Adam,

We see that you use Focus Time regularly and want to get in touch. As a reminder, Focus Time (screenshot below) is our feature for ensuring you have enough time outside of meetings to get your work done. We’re contemplating replacing it with a more powerful feature called Recurring Tasks and would like to get your feedback on the particulars before we make any changes.

Would you be so kind as to fill out this three-question survey for us (survey link) or, better yet, we’d be happy to speak with you on the phone (schedule here) to capture important workflows before we make any changes.

–Jane

Tasks PM @ Motion

Part 2: Deprecation Message

Subject: Focus Time Is Now Recurring Tasks

Hi Adam,

We’re pleased to share that the new version of Recurring Tasks is ready and it’s replacing Focus Time on September 1, 2023. Recurring Tasks captures all of the work you used to do with Focus Time, its performance is 10x faster, and it optimizes your schedule 25% more efficiently. It also gives you the power to do [new capability 1, new capability 2, and new capability 3], which Focus Time couldn’t support.

It’s never an easy decision to sunset a feature people rely on, so we’ve set up an automatic transition that will change all of your Focus Time tasks into Recurring Tasks. It will automatically complete on August 1, 2023, at 10 p.m. PT, giving us a full month to work out any unintended consequences together and ensure that you stay in “Motion” on what matters.

There’s no action needed from you now, but if you want to get started sooner, check out our tutorial on how to use Recurring Tasks to save an extra three hours a week, a guide on how to transition sooner if you’d like, and an article explaining more context on the research we did with users to be sure we don’t miss important needs [links].

As always, feel free to reach out with any questions about this transition to jane@usemotion.app.

Best,

Jane

Tasks PM @ Motion

Subscribe now

Hi! If you enjoyed these insights, please subscribe, and if you are interested in tailored support for your venture, please visit our website at First Principles, where we focus on product to help the world’s most ambitious founders make a difference.

Every great magic trick has three parts that build upon each other to maximize the audience’s excitement when the magician reveals the final part of the illusion. The last of the stages is called “the prestige,” a term of imprecise origin that became popularized contemporarily by Christopher Priest’s book “The Prestige” and the eponymous film released in 2006 featuring Hugh Jackman. The Hollywood re-enactment of a Victorian-era feud among magicians would have us believe that magicians spend their time dreaming up better ways to build to the climax of each illusion to create the maximum impact on the audience. Ricky Jay, the renowned magician known for his sleight of hands skills once said “You’ve got to take the observer from the ordinary, to the extraordinary, to the astounding.” If you want to have an excellent product demonstration, you need to do the exact same thing and build to your prestige.

However, a powerful prestige in a product demo is the culmination of dozens of thoughtful decisions that start long before the customer is in the room.

Subscribe now

A Story

Great demos tell stories. No matter how fantastic the features, demos filled with only features fall flat. There is no final climax in a procedural run-through of features. The product team must weave a compelling storyline into the demo so it resonates with the audience and makes the product relatable and desirable. Instead of “we have a crawler that extracts your personal data from Facebook,” try “When Suzie’s brother was tragically killed in an accident last year, Suzie needed a way to build up a permanent digital memory of her brother’s life.” With a powerful story, we instantly empathize with the emotions and aspirations of the subject. A great one not only reinforces our memory of the demo, but it also commands our attention because we want to know how it resolves. The story does not, however, have to tug on our heart strings. Umbra, a remote sensing satellite company, tells an industry-specific story that wouldn’t resonate outside of its target market and this is fine—it is the tale of surprise, sales rep-driven pricing for purchasing satellite imagery. As a result of that reality, the screenshot below from their pricing page presents both the problem and the solution all at once, creating at least a partial prestige moment. Further, Umbra lets its prospects task a satellite to take a picture during the sales meeting and then sends them the image a few hours later when it becomes available.1 Full prestige moment in action!

About a real(-ish) person

The story we tell can’t be too hyped-up or divorced from the customer’s reality. To the contrary, we need a narrative where the customer can envision themselves inside of it. Bad demos don’t reference people at all. Okay demos have made-up ones. The great ones tell a very real tale about a real person, or at least a plausibly real person experiencing an issue that occurs frequently enough for your target audience that it’s clear that your product is speaking to their needs. Instead of “suppose you need to measure the accuracy of different types of satellite location measurements,” try “In our demo we’ll follow the story of John, a Captain in the US Space Force who just learned that he needs to track the whereabouts of a new, clandestine Russian satellite, but he’s received conflicting reports about how close it is flying to an important US GPS satellite.”

Experiencing a challenge that is relatable

That real person must experience a real problem that the audience can relate to! A lot of new AI technology fails this test. “Wouldn’t it be nice if you could create an AI representation of your own voice,” is a nice start, but we need to also know “As someone writing a popular newsletter without the time to produce a podcast, wouldn’t it be nice to make episodes based on your newsletter with no additional work?” That’s how we tie the two together, by building a bridge between the person and the contextual need. Bonus points if you have a product where it’s easy to make the story your actual customer’s story in time for the demo. A fun AI marketing startup I know that helps write website copy used to start its demos with a re-write of the prospect’s website. This is fantastic because it immediately anchors the value to a situation the customer is deeply knowledgeable about. And if you’re thinking, well, that audience wouldn’t think it’s cool enough, then that’s a giant signal that it’s time to return to the drawing board and improve the product until it is compelling in the situations that matter.

Shows something seemingly impossible

Put simply, the product must actually do something incredible on behalf of a real person experiencing a real situation. This is a really high bar, and frankly most new products don’t meet this threshold. They do “something,” it’s just not something that matters or it’s not remarkable. The challenge is to find the one or two elements where the product can shine in this regard. Focusing on the previously impossible is a great way to anchor this aspect to showcase your prestige. Motion, the AI executive assistant for managing calendars, meetings, and projects, has a fantastic prestige moment showcasing this. The product deals with the challenge we all experience when we have a todo list that doesn’t match our calendar. In Motion, users can create todo tasks and the software schedules the tasks on our live calendar based on the precedence we set, and then it automatically reschedules meetings and other tasks that no longer fit, helping us achieve what we really intend instead of being slaves to our existing calendars. Before Motion, this seemed like an impossibly time consuming task without the support of an administrative assistant. This animation from their website shows the prestige in an instant.

That seems really hard to do

One crucial aspect of your product demonstration is highlighting the difficulty of the problem your audience faces and how your technology makes it possible to do something they wouldn’t even bother tackling. BoldVoice helps non-native English speakers drop their accents with personalized voice training. The reality is, most people just assume that they will always have an accent in a non-native language, but BoldVoice shows the user not only a solution but a path to doing something that seems like it wouldn’t be possible to tackle. The prestige is speaking a sentence with your own voice and then immediately getting detailed feedback that allows you to correct your accent on the very next pass and get graded with no judgement and no one watching. By contrast, demonstrations that don’t showcase the solution to a hard problem aren’t compelling, even if they connect to a real person in a real situation.

Magically becomes feasible in an instant

In the realm of product demonstrations, it is essential to create a sense of magic and wonder by showcasing how your solution makes the seemingly impossible problem suddenly feasible. The outcome should manifest instantaneously and dramatically at the finale, just like the prestige in a magic trick. We want the customer to have a breadcrumb to ground them in reality and to know that we didn’t fake our solution, but there should be sufficient mystery around how the product works and a quick enough transition into the final stage so that the customer can picture themselves getting the same outcome in seconds. If the build-up to the final demo moment takes too long or seems complex to arrive at, then the customer will assume they will also have a difficult experience, which will undermine any prestige. Laying seeds along the way and showing small prestige moments on the journey is a great path for a more complex offering, but make sure the story you choose has a final moment that puts all of the pieces together. At Palantir, our first demo centered around a terrorist suspect who was alluding authorities, and in the prestige, the system would magically find the connections between that suspect and known person’s of interest, the last piece of the puzzle needed to disrupt the next terrorist attack. It took just a few seconds and brought together all aspects of the story in one powerful moment. In the companies’ current demo videos, the narrator sets the stage of conflict in the South China Sea, and shows the below image which depicts a user’s ability to bring a satellite operating in the vicinity immediately to bear on answering a critical question. These are prestige moments, and when they hit right, your clients will stop asking you questions about what the product can do and start asking you buying questions.

Putting it all together

Capturing all aspects of this pattern is important, but it’s not necessary to envision them as ordered, distinct, or slow to build to. For most products, it will take 5-10 minutes to tell a real story and build the suspense for the final moment. However, ChatGPT is a great example of what a highly compact version looks like. For ChatGPT, the entire introduction, build, and prestige all live in about 20 seconds of demonstration. When a user types in an otherwise impossible to answer question that a real human would care about and asks it, without a human respondent in the loop, and then gets a human-quality result, that is adhering to this model. The same is true for a 20-minute Rippling demo when in the end it becomes apparent to the customer that, with the product, they can skip nearly every aspect of new employee onboarding. A sexy prestige is completely different depending on the industry and what is considered cutting edge in the audience. For tax professionals, for example, who are dramatically underserved with software solutions that ease their burdens, they don’t need to see 20K lines of code written live in front of their eyes. For them, a thoughtful capability for managing the influx of client documents and in-progress calculations delivers that moment. What matters is what the customer-base has become accustomed too. The prestige is about taking that reality and creating the maximum possible distance between that world and the new world that is empowered by your product.

Subscribe now

Hi! If you enjoyed these insights, please subscribe, and if you are interested in tailored support for your venture, please visit our website at First Principles, where we focus on product to help the world’s most ambitious Founders make a difference.

The CTO of Palantir Technologies, Shyam Sankar, in April 2023 testified before the U.S. Senate that “we must spend at least 5% of our [defense] budget on capabilities that will terrify our adversaries.”1 Shyam and CEO Alex Karp’s unwavering commitment to strengthening our country motivated me to resign from the civil service and permanently defer attending a top law school so I could join what was then Palantir’s small band of engineering mavericks. Our mission was simple: revolutionize the way the world’s largest and most important organizations answer their hardest questions. Fast-forward a decade, and Palantir is a public company whose alumni have gone on to build nine unicorns, from Anduril to Handshake to Amplitude, and over 100 additional venture-backed companies.2

At Palantir, we didn’t believe in heavy processes, because we felt that formulas and playbooks tacitly give people permission to stop thinking for themselves. Nonetheless, as I reflect on the experiences we had together and as I now partner with new startups looking to learn from Palantir’s experience, I feel compelled to compile and share my unofficial list of learnings.3 While the company has at times been a lightning rod for public speculation, the secrecy and misinformation surrounding its missions often prevent the amazing product lessons it forged from spreading the way they deserve. Not every lesson will work for every startup or innovation team, but I hope these ideas will catalyze others to consider unconventional approaches to doing the impossible and encourage the creation of more world-changing technologies.

A playbook of first principles: To disrupt an industry. . .

1. Forward-deploy your engineering

Product thought leaders tend to extol the value of talking to users, but most are barely scratching the surface. At Palantir, one of the largest teams was called Forward Deployed Engineering.4 This is the team I joined when I first got hired, and we were responsible for making our early product live up to the vision. Our main goal was creating product solutions for issues blocking customer adoption, or that could unlock new growth. The golden nugget in this philosophy is that you don’t worry about how to ask users the right questions or obsess over “interviewing” them. Instead, you are literally there, in the shit with them, getting involved in what is happening. In the early days at Palantir, this meant flying to war zones carrying a bunch of laptops to where U.S. forces actually engaged the enemy. In a commercial context, it didn’t mean jumping on the phone with the fraud specialists at a major bank. It meant going and spending the next three months in customer operations centers, working the same leads as the customer. Anduril, where many Palantir alumni have gone next, continues a similar practice, and I’ve met numerous excellent PMs who work this way even when it’s not a part of the company culture.5

It’s not just getting downrange that matters. You have to be the user to unlock this concept. I don’t mean that spiritually as in “think like the user”; I mean literally do their same job with your product as an extended member of their team and see what you learn. You don’t embed for 20 minutes, either. A lot of the real aha moments come after you’ve been in the customer’s shoes for an extended period of time. Work with your product long enough that the customer is impressed that you’ve accomplished something real for their business that they couldn’t. In the early days of one of our most successful engagements that led to an over $100 million deal, the customer had no users of our product for the first 6 to 12 months. We formed a small team that was given direct access, integrated their data into our existing product, built prototype solutions for the unique problems this engagement taught us about, and then performed the analysis and briefed it directly to a top worldwide CEO.

Subscribe now

Some financial market analysts have judged Palantir harshly for this sort of embedded dedication, attempting to cast the company as a services organization or consultancy. They completely miss the point that this process is how to learn which problems matter the most and how to build compelling solutions; it is not a permanent state but rather a tool for the vanguard as it approaches a new use case. Further, when innovating in emerging technology applications, often the customer doesn’t yet have a conceptualization of how they will work in the future, so by being one of them, you can establish the first patterns for doing that work excellently. Instead of spending years iterating based on a loose understanding of the customer, we completely understood what mattered and executed immediately, driving real value an order of magnitude faster with superior results.  

Lessons:

1. Go fully inside the customer’s environment.

2. Don’t just empathize with the user; be the user.

3. Do real customer work long enough to have full empathy and inspire.

2. Hire the best absolute greatest people who exist

Everyone says “talent is everything” and “it’s all about the people,” but many executives who I’ve encountered have never worked with the absolute best on the planet. As a result, it’s not obvious to them when they have hired someone who is decent, pretty good, or even downright average. Just because someone saves your butt doesn’t mean they are “amazing” or a “rock star.” Sure, we say that and are appreciative, but that’s not what we mean. We aren’t talking about “the top 1%.” We are talking about the literal one best person for the job who exists on Planet Earth. I cannot stress this difference enough. FAANG companies are the backup plan for this cohort.

Early in our days engaging with military customers at Palantir, we hired directly from special forces, for example ex-Navy SEALs. The SEALs are the créme de la créme in a military context, but we didn’t stop there. We hired some of the most decorated and well-known Navy SEAL commanders alive today. I distinctly recall telling customers who I was bringing to the meetings, and people who had never met either of us would instantly know that individual’s reputation and nickname, and what he was known for. Last year, a space and AI startup I was advising set a great example for this lesson. They wanted to build a first-of-its-kind satellite fabrication facility. The person they hired for the job is an engineer who has built more satellites that are in the sky today than any other human. That’s hiring the absolute best.

Hiring this way is crucial for two reasons. First, the best are, well, better, at the specific work they’ve been asked to tackle. But the less-understood reason why this is so high-leverage is because the world opens doors for people at the very top of their respective fields. The best AI engineers in the world, for example, attract media attention, top VC attention, and top founder attention. As a result, they are better connected and have disproportionately more access to the other best people in the world. This accelerates their learning and dramatically improves their ability to make the impossible happen. The difference between the #1 and the #50 in the world in terms of skills is practically indistinguishable, but as a result of being #1, that individual likely has access to an order of magnitude more knowledge, opportunity, and talent. This problem cascades down the organization through hiring generations, which is one of the reasons why the best startups are orders of magnitude better than the next tier down.

Prof. Lambeau: “You’re right, Will. I can’t do this proof. But you can, and when it comes to that, it’s only about. . . it’s just a handful of people in the world who can tell the difference between you and me. But I’m one of them.” 

—Good Will Hunting, 1997

But the top people aren’t always obvious or famous, so that cannot be our only means for identifying them. One of my favorite new startups is Quindar Space. The Quindar team is easily one of the best teams in the world for the problem they are solving. They focus on every piece of software technology required for satellite spaceflight that is not directly a part of the satellite—for example, software for determining when to maneuver, tracking onboard issues, and finding ground stations on Earth that satellites can beam data down to. The founders all built this type of software together at multiple prolific satellite companies, including OneWeb (the company with the second-most satellites in orbit after SpaceX), before they became founders at Quindar. Almost no one has more experience or skill in standing up these systems, giving them an enormous advantage. Sometimes these individuals are well known and other times they have been buried in a bureaucracy, and sometimes they don’t even know that they are the world’s best at a particular problem.

Lessons:

1. For each discipline on the critical path to success, hire the world’s #1.

2. To be sure, ask others in the community if those people are the best.

3. Look in unexpected places for the talent—it’s the experience, not the fame.

Remaining Lessons

To read the remaining five lessons, please visit the original article hosted on Lenny's Newsletter which was published on June 13, 2023.

Subscribe now

Hi! If you enjoyed these insights, please subscribe, and if you are interested in tailored support for your venture, please visit our website at First Principles, where we focus on product to help the world’s most ambitious Founders make a difference.

If a concept lives on the left (low need), teams have to iterate like crazy, get lucky, or try many potential solutions to get to success. Think social media--no one needs a virtual social network, but the solution to the problem that Facebook originally created shined because of great performance and exclusivity. The solution was worth our time.

If the concept lives on the right (high need), teams can have a pretty bad solution and still win if it even somewhat solves the need. Meanwhile, great solutions become generational champions (ex: Google search, Palantir, SpaceX). The lottery is a great example of the former--you're almost guaranteed to lose your money (bad solution!), but the human desire to make giant amounts of cash with no effort is so incredibly high that a terrible product still excels.

My point is that three out of four of these quadrants support strong businesses, but what you have to focus on and how you generate success changes dramatically.

Hi! If you enjoyed these insights, please subscribe, and if you are interested in tailored support for your venture, please visit our website at First Principles, where we focus on product to help the world’s most ambitious Founders make a difference.

Talking to users is great, but getting the valuable insights we need from these discussions is equal parts art, science, and process. Unfortunately, most people asking to do these interviews focus on the process and not the result. We quickly gather dozens of research participants, prepare a script of questions to ask, interview, and then share out a massive, unintelligible spreadsheet of notes about what was said. Nothing changes for the product except that we all feel like we checked an important box. While well intentioned, the problem is exacerbated by the increased desire across the business to “be close to users.” Marketing, product, design, engineering, and customer success all want to be a part of the process, and we’re routinely not extracting the value, and instead it’s driving Founders and executives crazy. Poor execution sours leadership belief that anything new will be surfaced, and it impacts everyone whose job depends on great access. Here’s how to turn this around.

Aim Small on Expectations, Big on Volume

First, we need to set expectations. A lot of people in a growing product company need routine access to customers to do their jobs well. But an interview performed by sales isn’t the same as one done by design or one done by product. We cannot thrive in an environment in which multiple different sets of people representing entirely different interests all show up to ask their questions in the same 30-min discussion once a month. If sales owns customer relationships in your company and it believes it is doing its part by brokering scarce access like this, your sales culture requires a reset and realignment to the value the rest of the company brings to bear in growing and building a product. This is not to say that multiple constituencies can’t join the same discussions, they can, but we need to be realistic about how much we can learn in one interview and how broad the surface area of learnings really is. In one Zoom meeting we might have time for 6-7 legitimate questions. And yet here are the different people in the room:

• Product wants to perform a value test and compare different solution options.

• Design wants to test the usability of a new or existing feature.

• Marketing wants to understand the customer’s behavioral and demographic background and process for learning about our product. It’s also looking for the voice of the customer to inform messaging.

• Customer Success is looking for validation to support bug fix requests and to gauge if the customer has enough training materials.

• Engineering is curious what simpler alternatives might still satisfy the need.

• Sales is looking to gather data on price sensitivity and the offer.

• User Research is trying to parse the complete user journey and understand deep motivations.

• And the list goes on and on…

The average “good” interview surfaces maybe 3-5 critical insights across this entire realm of questions. That’s the expectation we need to set. We also need to set an intent for the discussion and that intent should include only one of the above sampling of interests. Running a usability test for a new feature in 30-mins is great. Doing a value test and learning about the customer’s motivations is another rightsized chunk of possible observation. Mixing and matching these, while efficient in the moment, leads to each team learning so little that it makes us dangerous but not informed. We have to cultivate a culture of prolific and frequent customer contact such that we don’t have ten people from across the organization all overanalyzing the same 15-minute Zoom recording…from an early stage sales call…where we have to make a lot of guesses because someone’s Uncle’s cousin made the intro… and that’s why Sal had to the do the interview…even though Sal works in People Ops and is brilliant but has no idea how to conduct one of these.

The second expectation we need to set flows directly from the insight about how little we can learn in one chat: talking to users is ongoing and the volume of touch points must be high so we don’t have to cram everything into a few sessions with a few people. The fail mode of too high of an expectation from one meeting generally comes about because we don’t have that many users to talk to, so everyone who has access to one wants to hog them all to themselves and batch up a lot of questions for these rare, magical moments. This is a symptom of an entirely different set of problems—either bad internal culture (ex: a possessive sales force) or an external signal from the market that our product isn’t awesome enough yet to have earned other people’s time in significant numbers. Highly specialized emerging technology products can be excepted to some degree if the total user base on planet Earth is less than a few thousand people. When this is the case, we need Founders or other team members who have done the job before (and recently!) or have some access or experience that allows them to speak about the users credibly on an ongoing basis. For example, if you’re building a developer tool for quantum computing and you were a quantum computing developer, then you likely can get away with this to some degree for a period of time. And if you’re building something for a community like this and don’t have that access and can’t hire, advise, or buy your way into it, that itself is a sign that this might not be the place for you to find product-market-fit.

Now, Execute With Thought

Now that we have sufficient people to talk to and narrow expectations for how much we can learn in any one interaction, we can focus on the interview session itself. Remember, if we can learn three or four nuggets of insight in the interview, it’s a win. We can’t entirely control who those insights will benefit (product, marketing, sales, etc), but we can steer the questions toward accomplishing one narrow goal, for example a usability test, a value test, or building out a user journey map. Building an amazing product requires hundreds if not thousands of great insights across every aspect from initial awareness to core value propositions to who renews and becomes a power user, so we need to be prepared that talking to users isn’t a start to finish process but rather an ongoing activity that we never complete.

Don’t Delegate To the Least Experienced

I get it, a low signal discussion with a user can be a painful use of time, and because of the process argument I made above, it’s easy to find templates or hire an agency to handle your interviewing for you. That agency, you should expect, will put its most junior person, probably from outside of the United States on the task. For those rare moments where that person is experienced and brilliant, hooray, but for most of us the result we’re going to get is really poor. The last few issues put this into context and hopefully articulate why this job is one you shouldn’t outsource (at least not entirely) and if you do, it needs to be with someone highly credible, because the results will alter your strategy, and bad results will do so in the wrong directions.

Asking the Right Questions

I won’t belabor this point as many great books have been written on the topic, but I’m repeatedly surprised how often I hear someone request to speak to users and then I join the calls and the questions prove they haven’t read or digested anything ever learned about this specialty. You get half of the interview focused on demographics or a ton of hypotheticals and questions like “What feature do you want to see built next?” or “Do you like this?” and other questions that don’t move the needle. The questions need to explore real world circumstances, and like a scientist, probe at motivations and impact without explicitly saying what you are testing. Related is asking the right questions but not setting them up systematically to prove or disprove your value hypothesis. If we’re comparing two approaches, we need to ask questions that will reveal the participants’ preferences even if unstated. For example, you might inquire about the cost to the business of not solving a particular problem, wait ten minutes, ask the same question about a different option, and then compare those results.

Follow the Threads that Matter

I’d argue that the number one issue I notice on mediocre user interview calls is when the person running the interview stays on script after the user tells a story that might be the secret to unlocking huge value. It sounds like this:

Researcher: “How important is that aspect of your job?”

Participant: “It’s pretty low for me. But that reminds me, if I don’t get the TPS report filed with all the right inputs, we get fined $20,000 per incident; it happened three times last week.”

Researcher: “Great, and can you tell us how you found our product?”

It’s important to avoid this tight focus on a set of scripted questions that completely misses the forest for the trees. Scripts are fine to the extent that they prompt you to ask about the things you want to test or learn, but charging through them mechanically tends to destroy conversation, and conversation and stories about real situations tend to illuminate real signal.

Interpret Answers Credibly With Context

This is by far the hardest to teach and where experience matters the most. Let’s take our TPS report example for a moment. If this is a good match for our business, this should be the focus of the interview. But it may be the case that the fines are levied for legal mistakes, and our company DNA is all about building hardware, for example. Knowing what’s relevant and that you are seeing the tip of an iceberg is a big part of the challenge. The other half of the battle is recommending initial solutions that get at the core of what the customer really needs. It’s pretty easy to listen to someone say that they struggle with weight loss, for example, and it’s quite another thing to diagnose that a solution that increases the percentage of vegetables in their diet through a clever delivery mechanism and fills them up at the same time is a better solution than another calorie counter. This usually doesn’t happen real-time in the discussion with the user, but the best product people are listening for these seeds and imagining many possibilities based on the customer/industry/technology constraints.

Focus Narrowly, Often, To Be Broad

It’s very difficult to uncover what’s happening in an entire industry or for a person’s specific job in one interview. Instead of trying to hit the middle ground, it’s better to aim narrowly across a wide set of customers, but using a different focus with different subsets of them. We discover the middle by going deep across many examples, not by directly soliciting details about the middle. For example, if we are building a product for reserving space on a rocket to launch new satellites, we learn a lot more by asking five customers “How did you book your last three launches and why did you do it this way” versus “What is the most common launch software?” The latter won’t hurt you, it’s just not that helpful beyond initially identifying the existing suppliers (which you could do with Google), whereas the former will give you the answer to what you’re competing with and context about what you might need to do to be superior. We have to be deep by being narrow across a range of topics.

We should also be careful of going too narrow and wasting time. A set of 20 interviews where we ask the same exact scripted questions about the password reset flow and learn nothing new after the third one is a waste. It’s equally important to zoom out often enough to avoid going really deep on lots of little problems that don’t impact the highest value issues for the customer. As much as I encourage lots of interviews, people’s time is valuable and ultimately scarce, and if we waste it, it’s hard to re-acquire it. It’s also not okay to deeply understand the moment in time we are trying to impact but be clueless about what happens right before customers seek out our product or right after they finish.

In summary, we’re not wrong to be skeptical when we get asked so often for more access to users but:

a) We can’t let access become scarce

b) We need to respect that different constituencies need different signal to be successful

c) We should manage our own expectations so we accomplish something real with each one, and

d) We need to reject low quality, templatized interviewing.

Share

Hi! If you enjoyed these insights, please subscribe, and if you are interested in tailored support for your venture, please visit our website at First Principles, where we focus on product to help the world’s most ambitious Founders make a difference.

Founders, Chief Product Officers, and innovation teams alike need to have a philosophy about when they will build new product lines. Without a set of principles to guide these decisions, the pressure to divert resources to pay homage to the next great idea becomes overwhelming. Before long, teams have a half dozen disparate but spiritually related efforts afoot, most of which don’t deserve to exist, and they drain scarce resources away from the most important product. Based on numerous experiences operationally and as an advisor, I argue that the bar is simple and ought to be quite high for adding new products—teams shouldn’t launch new product lines until they achieve product-market-fit on their existing lines—conversely, they can and should launch complimentary products if existing product lines have product-market-fit AND if marginal dollars invested have a plausible chance of returning EQUAL OR MORE value.

If we only launch first product lines after going through rigorous external market validation, fundraising, and/or corporate budgeting cycles designed to protect against bad investments, why should the bar be any lower for a second or third product line that will cost just as much and carry the same or higher risk? It sounds easy enough to agree with, but there is something about the momentum a company has once it begins hiring or growing a team at a great clip, and once the market starts to recognize the worth of our products, that makes us feel that opportunities are everywhere. These opportunities feel easier to capture than the business in front of us. I contend that this is a psychological fallacy—we know the specifics of our current situation well, and we know what stands in the way, and conquering those challenges feels hard, but new ideas aren’t as well understood so they feel like pure upside without the trench warfare required to bring an idea to life in the market. Of course, this perception is false—the follow-on idea is JUST AS HARD; we just don’t realize it yet. The perception of ease and the excitement for what’s new distracts us, and causes us to burn cash faster while multiplexing and distracting our teammates. In short…

DON’T BRANCH INTO NEW AREAS IF…

• Your main, existing product has not achieved product-market-fit.

• Do continue testing adjustments to your product to achieve product-market-fit

• Do periodically consider if you’re on the right path strategically

DO BRANCH INTO NEW AREAS IF…

• You’ve built a quickly growing business for your first product AND

• You have strong reason to believe that lighting a new fire will grow the business as fast or faster than pouring gasoline on the existing momentum, or you have sufficient funding to justify pushing two initiatives simultaneously with autonomous teams that won’t distract each other.

• OR, if you’ve decided, per above, to abandon the first focus in favor of aligning the whole team on a new one

What we want to avoid is the painful position so many startups and innovation teams end up in. Here are the hallmarks:

• You’re actively building multiple separate products for multiple separate audiences

• There are multiple development teams with multiple sets of stakeholders, often with entangled dependencies

• Disciplines like product, marketing, and sales are splitting their time across these initiatives even though none is clearly working

• There are as many discussions of new ideas as there are for improving the existing central product line

It’s easy for us to confuse action (generally, a good thing) on a new product as good energy but what it’s really doing is numbing us as founders or executives to the lack of traction we’re experiencing on the main product line. Resolving that challenge should be the top goal of everyone who can help, not launching another product to rescue the first one. As obvious as this might sound, I’ve never encountered a startup or innovative team pre-product-market-fit that was too focused. Let’s give that a moment to sink in. To the contrary, the opposite is endemic in medium successful ventures. But the mantra of small team productivity, which is great in so many ways, isn’t true in these situations. A group of 50 people working toward one great vision is far more powerful than two teams of 20 people working toward mediocre ones while simultaneously driving 10 people common to both projects nuts and draining their energy.

A Simple Way To Manage The New Bets

Once the product has proven itself and we’re ready to launch a follow-on, how do we manage the new bet? Similar to my guidance above, I don’t see a strong reason to re-invent the wheel—pretend your new initiative is a brand new baby startup and put it through the same tests that a founder would face when approaching the investor community. A sample set of stages could look like:

1. Invest $10K-$50K to validate if there is a sufficient market and demand for the product and whether you can build the right team

2. If so, invest $100-$250K to build a working version and seek traction

3. If significant traction is achieved, invest $1-2.5M to scale and grow

4. If scaling and growth works well beyond the early adopters, invest $8-$12M to build a first class team and scale

5. Etc, etc.

Please consider the monetary values above as rough guideposts—different organizations will have different resources and different costs, but the point is to roughly 5-10X investment each time the product can prove itself ready for the next phase. For venture-backed companies, this model is obvious because the founders went through it for the first product line, so it’s simply a matter of reminding everyone who is new about these criteria. In a corporate innovation setting, the challenge generally is to get an annual budgeting cycle to line up with these needs. There’s nothing worse than generating $10M of demand six months into a budgeting cycle with only $2M invested. In these settings, I recommend pre-negotiating a set of bespoke milestones and rough investment levels so product teams don’t get stuck waiting for bureaucracy when they have succeeded in finding early indications of product-market-fit.

The other advantage to this model is that it empowers us to test more bets each time we’re ready to add a new product line. Instead of putting $5M blindly into the second line, for example, we can put smaller amounts into multiple options and see which ones advance before making a big decision that is hard to roll back structurally or financially.

But This Guidance Doesn’t Always Apply

There are always exceptions to any guiding principles, and I’ll briefly call out five:

• Organizations beyond the smallest startups should always have some people/time/budget set aside for experimentation (often labeled R&D, although I dislike this term and the structure it implies because it makes it sound like it’s supposed to be isolated from the rest of the business). Further, if a team member discovers something great on the frontlines, it should be incorporated into the existing product.

• We should always be testing our existing product and new ways to improve it or expand its reach and desirability—this article isn’t about those tests, it’s about establishing entirely new product lines altogether

• For many products, the achievement of product-market-fit for the foundational technology dramatically simplifies finding it again for the follow-on products. This was certainly true at Palantir where it was a lot less than 2X the work to serve two different markets in their similar data integration needs. In situations like these, the above framework is too rigorous and will limit growth that ought to be claimed sooner/faster.

• We have multiple well-vetted ideas showing about equal promise to be the next product line. When this happens, see if you can justify investment for both! Don’t force a false tradeoff.

• Sometimes we’re lucky enough to have an incredible team that cannot be stopped and has an amazing track record. We need to empower A+ teams like these and encourage them to skip a few stages, give them the investment they need, and trust their instincts. VCs do this all the time when they write checks for multi-exit founders with little regard for what they are going to build. We can do this internally inside a growth-stage startup or corporate innovation team too.

Share

Hi! If you enjoyed these insights, please subscribe, and if you are interested in tailored support for your venture, please visit our website at First Principles, where we focus on product to help the world’s most ambitious Founders and innovation teams make a difference.