What AI really costs: tokens, context and hidden costs

"Will ChatGPT replace me within a year?" That was the question Tom Haberland, jazz guitar teacher and YouTuber, brought to a podcast conversation in Berlin with Jens, the founder of Selt. For an hour they talked about whether AI replaces musicians and teachers, what a token is, and why home-built AI tools sometimes give up after five minutes. The episode is in German.

The conversation is about music, but most of the answers apply just as well to a business. One above all: whether AI can take over a task is often not the deciding question. What decides it is the cost.

What is a token?

Whether you use ChatGPT, Claude or Gemini, you don't pay per question. You pay per token. A token is a chunk of text: the model splits every input and every answer into them. As a rule of thumb, a token is roughly three quarters of an English word. German, with its long compound words, needs more.

On a €20 monthly subscription you barely notice. Once AI is wired into your own processes through an API, every request has a price. Billing is per million tokens, split into input and output, and prices differ by a factor of ten or more between models.

Why every message costs more than the last

This is the first misconception. An AI model has no memory of the conversation so far. With every new message, the entire conversation is sent to the model again: the first question, the first answer, the second question, and so on.

Tom put it well on the podcast: it's like restarting the whole episode from the top just to add a single word. The longer a conversation runs, the more each new message costs. It also explains why AI-assisted programming gets expensive so quickly. Source code is long, and it gets sent again and again.

The hidden tokens: web search, PDFs, AI agents

The second cost driver is invisible. Ask a current model for the five best guitar teachers in Berlin and far more happens than an answer. The model searches the web, opens the pages it finds, reads them and summarises them. Every one of those pages lands in the context window as tokens, and from there on the invoice.

To the user it looks like an ordinary question. Behind it, tens of thousands of tokens may have been processed. The same applies to uploaded PDFs, and above all to AI agents that use tools, search files and run several steps in a row on their own. The more freedom an agent has, the harder it is to estimate the cost in advance.

When AI costs more than a person

"AI will replace jobs" has been one of the most debated claims of the past few years, and in some areas it holds. Many companies have hired fewer junior developers, for example, because AI can take on a large share of their routine work.

The conversation also touched on the counter-movement we see ourselves. At the end of the month an invoice arrives for every token used, and it is higher than expected. For some tasks a person is cheaper on balance, and often does the job better. Whether AI pays off for a task is not a question of principle. It is a calculation: how often the task comes up, how much context it needs, and what a mistake costs.

Context engineering: better answers, fewer detours

The most effective lever for quality and cost is context. A music teacher can tell within half a second that the student in front of them is a 15-year-old beginner in Europe who has never heard of a scale. An AI model can't. Ask it for "the coolest scale" and it may just as well answer with Arabic or Chinese scales.

Practitioners call this context engineering: giving the model exactly the information the task needs. That means the current level of knowledge, the goal, and the form the result should take. Tom works the same way when teaching. Before the first lesson there is a phone call to establish where the student stands and what they want to achieve. And every prompt Tom writes ends with the same line: "If anything is unclear, ask me before you start."

Good context saves twice. The answers fit the first time, and there are fewer correction rounds, each of which reprocesses the whole conversation.

When AI automation pays off

The podcast covered three examples that show where the line runs.

Pays off: the automated lesson summary. Tom teaches over Zoom and used to spend 10 to 15 minutes after every lesson writing a summary from memory. Now a small automation picks up new recordings, moves them to Google Drive and transcribes them locally on a Mac. Only the finished transcript goes to the AI through the API, which turns it into a summary. After a quick read-through, one click sends it to the student. The task comes up constantly, the token use is modest, and a person checks the result. That is what useful AI automation looks like.

Probably doesn't: the "Tom bot". Tom's idea was to feed every lesson recording into an AI that answers students' questions in Tom's style. Technically possible, but the answers need constant review, the knowledge keeps evolving and has to be kept up to date, and every student question costs tokens, including the ones asked for fun. The effort quickly outweighs answering ten questions in person.

A prototype is not a product: the AI DJ. For a party, Jens spent an afternoon building an app that connects Claude to Spotify and runs the playlist through a chat. At home everything worked. At the party it ran for five minutes, then Spotify's rate limit kicked in and the app was blocked. That is the core of vibe coding, generating software from prompts instead of writing it yourself. For a quick prototype it is excellent. For something other people depend on, you need someone who thinks about limits, security, maintenance and cost from the start.

The prototype still has value. When a client shows us a version they vibe-coded themselves, it is the best description of what they need that we could ask for.

What this means for mid-sized companies

For AI in a mid-sized company, the same holds as for a guitar teacher with a Zoom course. Not every task AI can take over should be handed to it. The tasks worth automating come up regularly, have a clearly defined context, and end with a person who checks the result. And you want to know the costs before the first monthly invoice, not after it.

This is where our AI consulting starts. We look at which processes in your company eat up time today, and we work out where AI actually pays off, including the token costs of running it. Then we build solutions that don't hit a limit after five minutes and keep running day to day.

The full episode with Tom Haberland is in the video above and on YouTube, in German. If you want to learn guitar, you'll find Tom at tomhaberland.com.