What type of AI usage?

These past years, I have had a lot of people reaching out and wanting to talk “AI” (or LLMs) with me. Either the AI conversations have been around AI strategy for themselves, or for their company. Sometimes they have worked for a company claiming to be “AI first”.

Assuming that the conversation is about customer impact (which “AI usage” is not¹), my first question is always

“What type of AI usage are we talking about here?”

I think this question might be my biggest contribution to popping the AI bubble - making AI concrete and useful. Each type of AI usage has a different success metric, requires a different set of skills, and a different conversation.

In these talks, I have generally seen four common usages of AI:

• As a product interface
• As a product implementation detail
• As a coding tool
• As an administrative tool

Let me go through these one by one:

AI as a product interface Link to heading

AI, and specifically the breakthrough of Large Language Models (LLMs), now means we can interact with a product in new ways: Through free-flow text.

What was previously done through structured input – dropdowns, single-entry text fields, checkboxes, etc. – can now be done by simply entering sentences into a text field. This can be highly useful for flexibility; for example, a few sentences can be translated into a database query.

Less discussed but still relevant is that LLMs also allow for images, audio, and video, as input.

Compared to structured input, all this flexibility comes at the cost of correctness (misunderstandings), determinism (same input yields same output), and a loss of discoverability (what you can do).

Success metric for an AI product interface could be usage by end users.

AI as a product implementation detail Link to heading

AI allows products to be built on top of technology that can analyse large swaths of unstructured data. It can extract specific data or classify it. For example, browse the web, click around in MacOSX, analyze PDFs, etc.

AI also makes it possible for systems to learn. Input data, and you get a model that can be used to classify things.

LLMs can also be used to generate unstructured output. What in 2015 was “text templates” is now fully improved output by an LLM. “Please write a text for a report with information X, Y, and Z. I would like you to use the tone of a professional editor.” Previously, a programmer would have a template that looked like this:

This is an example of a report. Last year ({{ last_year }} we had {{ last_year_count }} number of tigers. This year, the new number is {{ this_year_count }}.

Success metric for AI product implementation details is usually something like service levels.

AI as a coding tool Link to heading

An AI can take prompts and generate and/or modify code. Tools like Claude Code, Cursor, et al. have all made this possible. It allows for generating huge amounts of code in a short amount of time - and can leverage a lot of domain-specific knowledge that an LLM might have.

Compared to manually coding something, the losses of using AI for coding can be a lack of correctness, a lack of understanding, and a lack of a well-designed UX.

A successful outcome of introducing coding as a tool can be something like “number of shipped features, while keeping service levels within bounds”.

AI as an administrative tool Link to heading

Finally, there is AI as an administrative tool. The most common product here is OpenAI’s ChatGPT or Anthropic’s Claude, but there are plenty of others.

These products are great for research, giving feedback, correcting text, or just generally reducing manual work. They can do one-off extraction of data from websites, PDFs, or other texts. They are also really useful for generating texts like e-mails, reports, or marketing material. Like a secretary of sorts.

Success metric for introducing AI as an administrative tool is probably increased productivity. A proxy metric might be usage, but AI usage is not necessarily increased productivity.

Conclusion Link to heading

I suspect that most companies handwavingly introducing “AI” in every little corner of their org. are doing so to increase company value without focusing on actual customer value.

By defining the type of AI usage in every AI conversation, our world can move away from hype to concrete value. That’s how we can make AI more than a bubble.