Many manufacturers running AI pilots right now are about to get a result they don’t expect: the pilot will work, and it still won’t matter.

The numbers make the point. One recent survey of manufacturers found that while 86 percent reported AI being used in pockets of the business, only 15 percent said adoption was extensive, and none reported AI fundamentally transforming operations. That lack of cohesive adoption and failure to transform both lead to an inability for AI use cases to generate measurable ROI. Across the industry, pilots are running, but the returns aren’t necessarily showing up.

Some may assume this is a technology problem, but the models are capable enough for most of what manufacturers are asking them to do. The real problem is upstream of the model, in how the pilot gets scoped in the first place.

The pilot-proof paradox

To justify AI spend, most manufacturers do the responsible thing. They scope a pilot narrowly enough to prove the AI caused the result. Focusing on one plant, one function, and one task makes it easy to track clean metrics and build an airtight business case.

The trouble is that the same narrowing that makes ROI attribution clean is often what strips out the value from the project.

Manufacturing value rarely lives inside just one function. It lives in the larger cross-functional initiatives, like adjusting manufacturing schedules to accommodate emerging market dynamics or accelerating RFQ responses. A pilot built to isolate cause and effect will almost always exclude those handoffs, because those are the places where it becomes hard to attribute value cleanly. So while the pilot may prove the AI works at the task level, it doesn’t generate ROI because that individual task was never where the money was sitting.

That’s the paradox: the more rigorously a pilot is scoped to prove AI works, the less likely that proof is to translate into anything worth scaling. The AI’s ability to deliver value isn’t the problem. The method manufacturers use to test it scopes the value out before the pilot even starts.

There is a key exception: a task-level pilot can still generate real value if it relieves a genuine bottleneck and produces something immediately usable — for example, a customer support agent that resolves a ticket from start to finish. But in most manufacturing environments, the bottlenecks that respond to that kind of fix have already been addressed by more conventional means. What’s left is value that only becomes visible when you look across functions, not inside one of them.

Scoping isn’t the only reason AI returns in manufacturing have disappointed, but it’s the one hiding in plain sight. And one manufacturers can fix without waiting on better models or bigger budgets.

How to scope an effective pilot

The fix is to build a thinner pilot that redesigns a full workflow from end to end. A useful pilot should be narrow on population (focus on one plant, product family, customer segment, or order stream) but complete on process, running from the triggering event all the way through to an outcome you can measure.

A pilot with too limited a scope may look something like implementing a better scheduling algorithm inside the scheduling function. To make that example into a more effective test, instead focus on the full workflow for a single product family: a demand change triggers a schedule recommendation, which checks against parts and materials availability, which drives production execution, which produces a fulfillment and margin outcome. This can be tested with the same rigor as a narrow pilot, yet it delivers a completely different picture of what the AI is worth.

However, addressing an end-to-end workflow doesn’t have to require touching the whole enterprise. Consider something as contained as an industrial supplier responding to a single RFQ from an OEM. Even that involves eight functions working in sequence and, frequently, in cycles:

• Sales owns the completeness of customer requirements, volumes, pricing expectations, commercial terms, and timing.
• Design engineering translates the RFQ into technical requirements and proposes a concept.
• Process engineering assesses manufacturability and proposes the process route, tooling concept, equipment, cycle time, and manufacturing location.
• Production validates plant capability, capacity, labor implications, and launch practicality.
• Procurement checks material and component availability, lines up suppliers, and evaluates lead times and supplier risk.
• Quality interprets customer-specific requirements, flags special characteristics and validation needs, and pushes back on design or supplier assumptions that create risk.
• Logistics works out packaging, freight, country of origin, tariffs, and duties.
• Finance decides whether the proposed response clears the bar on returns.

That list alone explains why RFQ response takes as long as it does. But the harder problem is that this isn’t a straight line. It loops. A design choice changes tooling and cycle time. A process limitation forces a design change. A supplier constraint on materials or lead time sends the whole package back to engineering. A quality requirement adds cost that pushes the response underwater, which sends it back again. Every loop is a place where information goes stale, priorities cross, and something important gets missed on the way through.

What the automation looks like

So how would we automate an RFQ response with AI? One option would be to use AI to automate tasks within individual functions. 

For example, sales could run an intake agent that ingests the RFQ package, flags missing or conflicting requirements, drafts clarification questions, and scores the opportunity against strategic fit. Design engineering could then run an agent focused on requirements, reuse, and concepts that converts the RFQ into a structured spec, searches prior programs and test data for comparable designs, proposes initial concepts, and performs change-impact analysis. 

Those agents may be highly useful. But both would also demonstrate value mostly through efficiency gains inside a single department — faster intake here, less time spent digging through old drawings there. That is rarely enough on its own to justify the investment, and it is exactly the kind of result that looks good in a pilot readout and then quietly disappears.

The larger value sits at the seams between functions in the iteration loops themselves. Let’s picture an alternative design for answering the same RFQ where the AI crosses functional borders. This could include:

1. A requirements-to-feasibility agent spanning sales, design, process engineering, quality, procurement, and production. This agent could build one structured requirements baseline instead of six separate checklists, route each requirement to whoever has to answer it, and surface the requirements that have no known technical precedent, manufacturing solution, supplier, or cost attached yet. 
2. A design-to-cost-to-manufacture agent spanning design engineering, process engineering, production, procurement, quality, costing, and finance. This agent could evaluate a concept against performance, manufacturability, process capability, supplier availability, validation burden, tooling and capital needs, timing, and margin, all at once, and run the trade-off scenarios that would otherwise eat a week of meetings. It would keep the bill of materials, process route, cost model, risk register, and timeline synchronized, so a change in one doesn’t quietly go stale in the other four.

Both of these agents would accelerate cumbersome processes and improve speed and accuracy more substantially than automation within only a single functional portion of the RFQ process. The conventional RFQ process runs like a relay race:

 Design proposes a concept → process engineering assesses it → procurement sources it → quality evaluates it → costing prices it → finance reviews it

And if anyone downstream finds a problem, the whole thing runs as far back as the start. An agent working across those functions turns that relay into something closer to a live negotiation among all the variables at once, where a change in one triggers an immediate check against the rest. Each step goes faster, and manufacturers need far fewer iteration cycles to land on a final answer.

However, the greatest financial case, particularly for the second agent, is margin protection. This agent secures margin by catching underquoting before a bid goes out, cutting down on tooling mistakes that show up later as write-offs, and lifting win rates because pricing is more accurate when the trade-offs are visible. 

The real test for a pilot

Manufacturers deploying AI pilots don’t need to choose between provable attribution and meaningful business value. They do need to refocus on the process instead of the task. The trick is to keep the population small and think beyond the confines of a single department.

The right question isn’t whether AI can do a given task better than a person did it before. Plenty of pilots will answer yes to that and still deliver little that shows up in a P&L. Instead, ask whether AI, working across the full path from trigger to outcome, can improve the workflow enough to impact business performance. A task-level pilot can look impressive in a demo and mean nothing on the plant floor. An end-to-end pilot, even a narrow one, can reveal whether AI is improving performance or just creating activity.

The goal was never to prove that the AI works. It was to prove that the business works better because of it. Those are two different tests, and right now, most manufacturers are only running the first one.