Keeping AI free: a fair-use quota that never blocks the work

A booking product has a handful of jobs that a language model is genuinely good at. Reading a salon's paper consent form out of a PDF and laying it out as fields. Matching the columns of a messy client spreadsheet to the right places. These are small, bounded tasks where a model turns a frustrating ten minutes into a review-and-confirm. So we wanted the model available across the product, not behind a premium tier.

That runs straight into two facts. A model costs money per call, and the free tier we start from is a single shared budget across every tenant. If we were careless, one org importing a hundred files in an afternoon would spend the whole day's free budget and everyone else would get errors. And if we solved that by charging, we would have built the paywall we were trying to avoid.

The idea: AI is a boost, never a gate

The design turns on one decision. Every AI-assisted feature already has a free, deterministic version that works on its own. The PDF importer has a rule-based reader. The spreadsheet importer has a synonym matcher that maps Full Name to name and Mobile to phone. The model does not replace those. It runs after them and improves the result. If the model is missing, out of budget, or simply wrong, the deterministic result is still there. Importing never stops.

That turned the whole problem into a single function. It takes the org, an attempt that calls the model, and a fallback that returns the free result. It always returns something usable, and it only counts a call against the allowance when the model actually produced a result worth keeping.

export async function withAiQuota<T>(
  org: AiQuotaOrg,
  attempt: () => Promise<T | null>,   // the model call; null = unusable
  fallback: () => T | Promise<T>,     // the free deterministic result
): Promise<{ result: T; usedAi: boolean }> {
  // 1. No provider configured -> AI is simply off. Use the free result.
  if (!geminiConfigured()) return { result: await fallback(), usedAi: false };

  // 2. Platform circuit breaker: protect the shared free budget.
  if (await readCount(platformDayKey()) >= platformDailyCap())
    return { result: await fallback(), usedAi: false };

  // 3. This tenant's monthly allowance (the opt-in add-on lifts the ceiling).
  if (await getAiOpsUsage(org.id) >= aiOpsQuota(org))
    return { result: await fallback(), usedAi: false };

  // 4. Try the model. Only meter a real result; a null falls back.
  const improved = await attempt().catch(() => null);
  if (improved == null) return { result: await fallback(), usedAi: false };

  await Promise.all([bumpOrgMonth(org.id), bumpPlatformDay()]);
  return { result: improved, usedAi: true };
}

A feature adopts AI by wrapping its model call in this function and handing over its existing free path as the fallback. Nothing else in the feature has to know about budgets, keys or metering.

Counting without a new table

We needed a per-tenant, per-month counter that is safe when several serverless instances bump it at the same time. We already had exactly that shape for API rate limits: a tiny row with an id, a count and a reset time. So the AI meter reuses it. The key carries the tenant and the month, so a fresh month starts a fresh row on its own, and last month's row is simply never touched again.

-- key = 'aiops:<orgId>:<YYYYMM>'. One atomic upsert, so concurrent
-- instances can never lose a bump. The row resets only when the key
-- changes (next month) or its window has lapsed.
INSERT INTO rate_limits (id, count, reset_at)
VALUES (:key, 1, now() + interval '40 days')
ON CONFLICT (id) DO UPDATE SET
  count = CASE WHEN rate_limits.reset_at > now()
               THEN rate_limits.count + 1 ELSE 1 END,
  reset_at = CASE WHEN rate_limits.reset_at > now()
                  THEN rate_limits.reset_at
                  ELSE now() + interval '40 days' END;

The same trick, with a key that carries the day instead of the month, gives the platform-wide circuit breaker. When the whole platform has spent its day's free budget, the gate stops calling the model and every tenant quietly falls back until tomorrow. One noisy neighbour can spend its own allowance, but not everyone else's.

Paying for more, on purpose

Some tenants will want more than the free allowance, and that is fine, as long as it is a decision they make rather than a bill that arrives. So the paid path is an opt-in add-on that a tenant switches on themselves. When it is on, the same gate reads a higher ceiling. We never move a tenant onto a paid footing automatically, even when they run out. The default is always to fall back to the free result.

Free by default is only real if running out costs the user nothing but a little accuracy.

Everything fails soft

The last rule ties the rest together: metering must never break the task it is measuring. Reading the counter falls back to zero if the database blips. A failed bump is swallowed. A model that times out, returns nothing, or hands back unparseable output is treated as a null and falls through to the free result. The worst case for the whole system is not an error. It is an import that is a little less polished than it could have been, which is exactly the outcome we wanted when we started.