How to Reduce AI Processing Costs: Lessons From Ship Supply

Overusing AI is rampant. Across the systems we see, a surprising amount of processing cost comes from asking models to repeat work the business already understands.

A customer description that has already been matched. A unit conversion with a known pack size. A pricing calculation with an agreed rule. Each may sit inside an AI product, but none should require a fresh judgment from a language model every time.

Ship supply has an opportunity to avoid that mistake. We can borrow the useful applications other industries have already developed, then build around the operational knowledge that makes a chandlery work.

Our approach at Breakwater follows a straightforward principle: use software to remember established answers and execute known rules. Spend AI processing on the decisions that remain unresolved.

The cheapest AI processing is the processing you never do.

A cheaper model does not automatically produce a cheaper workflow.

If it needs repeated attempts, escalates most requests to another model, or creates more work for the quoter, the lower price per call can disappear into the surrounding process.

For a ship supplier, the useful measure is cost per approved quote.

Over a defined period, that means adding up model and infrastructure costs, human review, and rework, then dividing by the number of quotes completed to the required standard. Track cost per line alongside it so that changes in quote size do not disguise what is happening.

The model bill is one part of that calculation. Retrieval, document processing, repeated requests, and time spent fixing mistakes belong in it too.

This changes the engineering question. You start asking:

• How much work can we resolve without a model?
• How often does the first model attempt produce a usable result?
• Which decisions cause retries or escalation?
• How much attention does the quoter still need to give each quote?

Those questions connect AI cost optimization to something the owner can measure.

Ship supply can borrow established AI techniques for extracting information from unfamiliar documents, matching inconsistent descriptions, classifying requests, and preparing drafts for human review.

Those are useful places to apply a model’s ability to interpret language.

Much of the work around them is arithmetic, policy, or an established fact.

Consider a request for 30 litres of milk. If the verified catalogue says a case contains twelve one-litre bottles, the conversion produces 2.5 cases. Whether the business sells individual bottles, permits partial cases, or requires whole cases is a commercial rule.

A model should not have to rediscover either the multiplication or the policy.

The same applies to carrying a case cost down to a per-bottle price, applying an agreed customer discount, or calculating a selling price against a margin target. Code can implement the approved sequence, rounding, and currency precision consistently.

The important prerequisite is having the right inputs. If the supplier document says only “case,” with no pack contents, the system needs to retrieve that information or ask someone. A larger model does not make an unknown pack size become a known fact.

Even OpenAI’s own latency optimization guidanceincludes the instruction “Don’t default to an LLM.” It explicitly points developers toward conventional techniques such as caching and ordinary application logic when those methods fit the task.

Knowing where to use those techniques is part of building good AI software.

We have tested the fully agentic quoting workflow often pitched as “email to email”: an incoming request goes into an agent, and a completed customer response comes out.

In our tests, layered AI wins.

The advantage is that a defined workflow gives us control over where the system makes a judgment, what evidence it receives, and what must be checked afterward.

Agents have a useful role in exploratory work, where the next step depends on what the system discovers. A recurring quoting process also contains many steps we can specify in advance. Anthropic’s guidance on building effective agents makes that distinction: defined workflows provide consistency for known tasks, while agents provide flexibility when the task requires it.

Our spending priority is:

Memory first. Rules second. A small model third. A larger model for the difficult remainder.

Once the incoming RFQ has been structured, the work can move through layers like these:

A request should stop moving through model tiers once it meets the acceptance criteria. Rules still check the results, including results retrieved from memory.

The model’s task should also be bounded. Asking it to select from five plausible products is different from asking it to understand an entire catalogue, infer the customer’s intention, calculate quantities, and generate the price in one response.

That candidate list needs an escape route: no suitable match. If retrieval missed the correct product, forcing the model to choose one of the five produces a confident error.

Routing also needs measurement. A task that consistently defeats the smaller model should go directly to the model that handles it reliably. Paying for a failed attempt first creates overhead.

Research such as RouteLLM has demonstrated that routing between stronger and weaker models can reduce costs while maintaining response quality on evaluated benchmarks. For a production quoting system, the routing decisions need to be tested on the actual requests the business receives.

A model returning neatly structured data has completed one requirement. There are others.

A response can contain a valid product ID, quantity, and unit while still selecting the wrong product.

OpenAI’s Structured Outputs documentation explicitly notes that structured responses can still contain mistakes. Enforcing an output format makes an answer easier for software to process; the business still needs to validate what the answer means.

For quoting, those checks should include whether:

• The selected product satisfies the requested specification.
• The quantity, selling unit, and pack contents are compatible.
• The cost comes from an applicable source, in the correct currency and purchasing unit.
• The pricing calculation follows the customer’s terms.
• Missing information or policy exceptions require review.

Some checks can be enforced directly in code. Others require a judgment from the quoter. Both should have an explicit place in the workflow.

The system should also preserve enough information to explain the result: the original RFQ line, the selected item, the conversion applied, the cost source, and the checks performed.

That makes failures easier to locate. If a conversion fails, repair the conversion. If a match is uncertain, review the match. Avoid restarting the entire quote because one stage needs attention.

This is one of the practical advantages of layered AI: the boundaries make errors visible and recovery more targeted.

A customer’s confirmed product description is valuable business knowledge.

If the team has established that one customer’s “topside” means inside round, the application should retain that decision and reuse it in the appropriate context.

The scope matters. A useful record should identify which business and customer it applies to, which product was confirmed, and any relevant specification or pack conditions.

A temporary substitution also has a different meaning from a permanent correction. If the quoter chooses another brand because stock is unavailable today, that should not automatically rewrite the customer’s normal preference.

Review should make that distinction easy: use this choice on this quote, or remember it for future requests.

Stable product mappings and changing commercial facts also need different treatment. A familiar description may still identify the same item next month, while the supplier cost, available pack, or customer agreement has changed.

Reuse the approved mapping. Retrieve the applicable commercial data.

This is how the application improves through use. Its records become more complete, and familiar work requires less interpretation.

Updating those records does not require retraining the underlying model every time. Model training is a separate process. A well-maintained memory layer can improve the workflow immediately by giving the software a previously approved answer.

Over time, the business should spend less effort resolving the same questions.

A person approving every quote is part of what makes AI useful in ship supply.

One incorrect price can erase the savings from preparing the rest of the quote. Experienced quoters also understand customer expectations, unusual requests, and substitutions that are difficult to reduce to a general rule.

The discipline is to focus their attention.

If a 300-line quote has three unresolved decisions, the interface should put those decisions in front of the quoter while keeping the complete quote available for review.

“Low confidence” is rarely enough information. The reviewer needs to know what requires attention:

• Two products fit the description.
• The requested pack size is missing.
• The source cost uses a different unit.
• A substitution changes the specification.
• The proposed price falls outside the applicable margin rule.

Confidence also needs evidence behind it. A model returning confidence: 0.97 has not, by that statement alone, established a 97% likelihood of being correct.

A study by Xiong and colleaguesfound that the models it evaluated often overstated their confidence. The engineering lesson is to measure reliability on the relevant task, rather than treating a model’s confidence statement as a calibrated probability.

For a quoting application, that means looking at actual outcomes: which kinds of matches get corrected, which rules catch meaningful exceptions, and where apparently routine lines still contain errors.

Confidence in the product match should not conceal uncertainty about the pack or price. Each unresolved decision needs its own treatment.

Review a sample of routine results as well as flagged exceptions. Otherwise, the system only learns about the mistakes it already knows how to surface.

A good review process protects quality today and improves the evidence used to route tomorrow’s work.

Once the workflow is sensible, conventional LLM cost optimization becomes useful. OpenAI’s cost guidance emphasizes reducing requests, minimizing unnecessary tokens, and using smaller models where they maintain the required accuracy.

There are several practical ways to apply that discipline.

Send the evidence needed for the decision

An item-matching request needs the original description, relevant units and specifications, credible candidate products, and applicable customer context.

It rarely needs the complete catalogue or years of unrelated quote history.

Retrieve the useful material and keep it traceable. Ask for the output the application needs, such as the selected item, unresolved fields, and a concise explanation where one is useful.

Reuse work that remains valid

Store stable catalogue representations and refresh them when the underlying records change. Recognize repeated processing of the same input so a retry does not automatically restart every completed stage.

For repeated model instructions, provider prompt caching can also reduce processing. OpenAI’s prompt caching documentation explains that matching prompt prefixes can reuse previously computed state. New input and response generation still require processing, and savings depend on actual reuse and the applicable cache pricing.

Keep the distinction clear: prompt caching makes a model call more efficient. An approved business lookup can eliminate the need for that call.

Give background work a different service level

Historical catalogue cleanup and bulk data enrichment often have more flexible deadlines than a live quote.

For example, OpenAI’s Batch API offers a 50% discount compared with synchronous processing and uses a 24-hour processing window. That can suit background work whose deadline allows it.

Choose the processing mode around the operational deadline. A saving that makes the quoter wait too long can be poor value.

Put a limit on repeated attempts

Retries should respond to a specific failure.

A temporary service error may justify another attempt. Missing pack information needs another source or a person. Repeating the same request with a longer prompt does not necessarily address the problem.

Set limits on retries and escalation. When the system reaches them, surface a clear exception with the work already completed.

Every additional call should have a reason.

The measures we would want on a production dashboard are straightforward:

• Cost per approved quote and per processed line.
• The proportion of matching decisions resolved through confirmed memory and rules.
• Model escalation and retry rates.
• Human review time.
• Correction rates, including errors discovered after approval.
• Time from RFQ arrival to a quote ready for review.

Read those measures together.

A rising memory hit rate is useful if the remembered answers remain correct. A falling model bill is useful if review time and error rates remain under control. A stronger model may justify its price if it materially reduces retries or difficult manual work.

Evaluate changes against a representative set of previously reviewed RFQs, including unfamiliar items and difficult exceptions. Track performance by the types of work that matter to the business, because a model that handles routine provisions well may behave differently on technical specifications.

The objective is a workflow that becomes cheaper and more dependable as the business uses it.

For a ship supplier evaluating an AI provider, we would still start with three questions.

Where do you deliberately choose not to use AI?

The provider should be able to point to calculations, established mappings, and commercial rules handled directly in software.

What would you advise us not to automate?

The answer should show an understanding of your operation, including where an experienced person needs to make or approve a decision.

What are we actually paying for?

The provider should explain the workflow being delivered, the integrations and business knowledge that support it, and the outcomes you can measure.

A useful answer connects the technology to your operating costs.

At Breakwater, that is the standard we build around: fewer repeated decisions, controlled calculations, targeted model use, and an experienced person approving the result.

Every expensive model call should resolve a question that still needs answering. Every reusable correction should reduce the work required next time.

Breakwater builds Blue Horizon, an AI-native platform for ship suppliers. Learn more about how Breakwater applies AI to ship supply, read what we heard from suppliers at IMPA London 2026, or email will@breakwateros.com to arrange a demonstration.