What one lecture recording costs

The feature was easy to describe: every lecture transcript gets a summary on top of it, thorough enough to revise from. The obvious next question, and the one I have watched a lot of projects skip, is what that costs the person using it.

I did the multiplication before writing any of it. It took about ten minutes and it decided the architecture.

The arithmetic

Four units in a semester. Each publishes something like two recordings a week across a twelve-week teaching period, so call it a hundred recordings a semester. A forty-five minute lecture transcribes to roughly seven thousand words, which is somewhere around nine thousand tokens going in. The summary coming back is maybe two thousand.

  100 recordings  ×  9,000 tokens in   =  ~900,000 input tokens
  100 recordings  ×  2,000 tokens out  =  ~200,000 output tokens

  at $0.10 / M input   →  $0.09
  at $0.40 / M output  →  $0.08
                          ------
                          ~$0.17

About twenty cents. For a whole semester, for everything a student is enrolled in.

The number is not “cheap.” It is negligible

That distinction matters more than it looks, because the two lead to different systems.

If the answer had been twenty dollars a semester, I would have had to build an entirely different product: usage metering, a way to cap spend, probably a paid tier, and a backend to hold my own API key so I could charge for it. All of that is real work, and all of it is work about billing rather than about the thing the tool is for.

At twenty cents, none of that is worth building. And once you know it is not worth building, a set of decisions that would otherwise be hard becomes obvious.

The arithmetic did not tell me what the feature would cost. It told me which architectures I was allowed to have.

What twenty cents bought

• 01The user brings their own key. Not mine. That means no backend, no billing, no shared rate-limit pool to manage, and no position where I am holding someone else’s spend. At a higher number this would be a hard sell; at this one it is a rounding error to whoever pays.
• 02The feature is optional and off by default. The tool does everything else — crawling, filing, transcribing — with no key at all and nothing leaving the machine. Adding a model is something the user opts into, not a dependency they inherit.
• 03Free tier first. A hundred calls spread over a semester, paced six seconds apart, sits comfortably inside the default provider’s free allowance. The paid number above is what it costs if you choose to pay. Most people never will.
• 04Eight providers behind one interface. If the user brings the key, the user picks the vendor — including a local model, where the cost is zero and nothing leaves the machine at all.

What the arithmetic did not cover

Cost per call is the easy half. The half that actually bit me was what happens when you hit a limit that is not about money.

Free tiers are rationed per minute and per day. A first sync on a fresh install has a semester of backlog to get through, so it is exactly the run most likely to hit the daily wall — and the naive behaviour is vicious. Every remaining item makes its request, gets refused, waits out its backoff, retries, gets refused again, and fails. A hundred items each politely waiting sixty seconds turns a broken run into an hour of a laptop fan.

So the allowance needs a circuit breaker. Once a quota error is final, stop asking for the rest of the run. Whatever is left gets picked up on a later sync, because a transcript that is missing its summary is retried by design. The user sees a run that finished, not a run that hung.

The bug in my own circuit breaker

I built that, shipped it, and it was still wrong — because I had assumed the only signal for “out of quota” is 429.

On a throttled free tier, the refusal often arrives as 403. And a 403 is genuinely ambiguous: it is also what you get for a key that is wrong, or a key that lacks permission. My code treated it as the second thing, raised an ordinary error, and never tripped the breaker. So the protection I had specifically built to prevent an hour of backoff did not engage in the most common case that causes one.

The fix is small and the reasoning is the interesting part. A 403 that arrives once might be a bad key. A 403 that survives every retry, with backoff, is not a key problem — a bad key fails identically forever and there is nothing to wait for either way. By the time retries are exhausted, both readings point at the same action: stop asking.

  # before — only 429 counted, so a throttled free tier
  # paid the full backoff on every remaining item
  if e.code == 429 or "RESOURCE_EXHAUSTED" in body:
      raise QuotaExhausted(message)

  # after
  if e.code in (403, 429) or "RESOURCE_EXHAUSTED" in body:
      raise QuotaExhausted(message)

The part I would tell someone starting out

Token spend is a design constraint, not a line item you discover in month two. It belongs in the same conversation as your database choice, because it constrains the architecture just as hard.

And you can nearly always work it out up front. You know roughly how much text goes in, roughly how much comes back, and roughly how often. Three numbers you can estimate in ten minutes will tell you whether you are building a feature or a business — whether you need metering, tiering, a backend and a support burden, or whether you need a text field where somebody pastes their own key.

I got to skip all of the first list. Not because I was clever about it, but because I multiplied three numbers together before I started typing.