Defect Detection · Chris West


800% / 50% throughput gain and error reduction	Multi-year AWS agreement — a partner productized it	~15,000 hrs saved annually at Tyson Foods	No ML expertise required

Two services, one job, and a user who understood neither

A poultry plant runs product carriers continuously, at speed. When one fails it creates a safety hazard, halts production, and can compromise product. Catching failures meant a person watching a moving line all shift.

Computer vision solves this in principle. In practice the path from “we have cameras and a defect problem” to “we have a model inspecting our line” ran through a stack that assumed an ML engineer at every step. Starting with training a model, exporting it, packaging it into a container, defining camera inputs, deploying to an edge appliance, routing inference output somewhere a human can act on.

Customers didn’t have that person. They had plant engineers, operations leads, and a Solutions Architect on a call.

Meanwhile the platform’s own shape worked against them. Lookout for Vision thinks in datasets, labels and trained models. Panorama thinks in nodes, packages, manifests and devices. The customer thinks in cameras and defects. Designing outward from either service would have produced a workflow that made sense to its team and to nobody on a factory floor.

Two-part diagram. Left, 'Product structure': a cloud-based console used by system integrators, connected to an edge station console that is repeated for each edge device at the end-customer location, used by integrators and line operators. Right, 'Deploy and Run inference': three colour-coded lanes showing the system integrator deploying a model from the model details page, the line operator running inference and reviewing an image gallery at the HMI, and the process engineer validating results across the inference runs, run details and dataset details pages — Bridging hardware and applications. Two consoles, two locations.

Three roles, two surfaces, one person crossing between them

Role	AWS console	On-device portal
System Integrator	●
Process Engineer	●	●
Line Operator		●

The Line Operator only ever sees a browser interface served from the appliance next to the line. They may never encounter AWS at all. The System Integrator lives entirely in the cloud console. The Process Engineer is the only role spanning both, making them the person who feels every inconsistency between the two.

That split, not the service boundary, is the design problem users actually live on.

Persona sheet with two groups. Under 'Partner / Denali': the System integrator, job titles IT Admin and system integrator, who implements and manages the system without being deeply technical; and the Process engineer, who supports manufacturing process development and validation. Under 'End-customer': the Line operator, job titles quality inspector and production or manufacturing operator, responsible for monitoring assets. Each carries a responsibility chart rating select technology, design architecture, create proof of concept, develop and test, monitor and troubleshoot, and manage production — Three roles, but two organizations. Two of them work for the partner; the one who runs the line every day does not.

Task mapping across six numbered phases — Provision, Image capture, Label, Train, Deploy, Inference — each showing which of the three personas is active, with the inactive ones greyed out, the tasks belonging to each phase, and a band beneath marking the first five phases as online and inference as offline — Understanding who will use the product, when and how.

Stages and tasks for the system integrator at Denali, in two bands. The proof-of-concept band alternates between cloud console and edge console: create station, then configure cameras and capture images, then build dataset, train models and deploy models to the edge, then configure workflows, configure factory inputs and outputs and review results, ending with creating a customer edge console GUI. The deploy-at-scale band creates stations and deploys models from the cloud console, replicates station configuration and installs the customer edge console on the edge device, then trains and hands keys to the line operator — The integrator's path alternates between cloud console and edge console at nearly every stage. Whereas the two other personas work either on cloud or on the local app.

Stages and tasks for the line operator at the end customer, covering daily production line operations entirely within the edge console: select workflow, monitor live results, and on an incorrect result flag inaccurate results, while a correct result loops back to monitoring — The operator never leaves the edge console. Select workflows, monitor, and flag missed defects or false positives when needed.

Choosing a design system for a place the design system couldn’t go

Cloudscape, AWS’s design system, was at the time allowed only for AWS console work. This application ran on localhost, served from an edge device, a non-AWS destination.

With the front-end engineers I evaluated 10–15 design systems, defining criteria before comparing: it had to be open source, and it had to carry a broad component set. An operator portal running production-line workflows needs tables, forms, status, modals and navigation. Not a minimal starter kit.

The unfortunate outcome was that we would need to use two different design systems. The cloud based interface used by ML engineers would be using one design system and the Line Operator using another with the onsite device localhost application. Most problematic was the System Integrator who spans both sides and would have a fragmented experience. Because the System Integrator life cycle was very short in time, we conceded that we would accept this situation, continue moving forward and keep an eye out for better as we continued production.

A lucky coincidence, having stayed alert to Cloudscape updates, two months into the project Cloudscape announced an open source version of their design system. I immediately went about advocating to the team that we should pivot quickly to adopt it. The winning agrument was that for our customers there would be no relearning, no retraining, the existing GUI patterns carried over (minimal front-end cost), and it was a system we could theme for a non-console context. Overall the product was now cohesive with a better customer experience and qualitative feel.

The criteria came before the comparison. The right design system arriving mid-research was luck. Being positioned to recognize and adopt it immediately was the analysis.

And we knew the visuals would be replaced. Our primary customer Denali told us during development that they’d reapply their own GUI after delivery, so the deployed product would carry their brand. What we shipped was formally a reference implementation.

Regardless I treated getting it right as no less important because when the visual layer is going to be reskinned, what you’re actually delivering is the interaction model: the information architecture, the workflow sequence, the states and edge cases. Those survive a reskin. A partner rebranding your UI keeps your structure.

Workflow setup screens from the locally hosted Automated Inspection app, built on open-source Cloudscape. Left to right: creating a workflow on a single page with workflow details, image source and model, inputs and optional outputs; editing that workflow; defining the input trigger and outputs; an Add output modal for signal type, pin value, pulse width and role; then three workflow detail views for a REST API trigger, a digital input trigger, and an image source set to a folder — Workflow setup screens from the locally hosted Automated Inspection app

Monitor live results screens from the locally hosted app, branching from an empty state into two paths. The upper path shows a digital-input trigger: loading, then an inspected carton on a conveyor with prediction, confidence, anomaly labels, result date, model and processing time, then the same view with anomaly masks shown over the carton. The lower path shows a folder-path batch: selecting a folder, processing an image, then the result with anomaly masks and an output file link — The line operator monitors live result screens from the locally hosted app.

How I worked

Set the north star — A plant engineer with no ML background gets defect detection running on their own line. Everything difficult follows from refusing to let the org chart become the workflow.

Align the organization — Built the evaluation matrix that settled the design system question, and mapped the workflow across two services and two surfaces so the team could see where customers would fall out.

Engage the constraints and push them — The labeling workflow is where customers spend their actual hours, and it had been treated as secondary. I pushed for properly designed region-drawing tools, and for previous/next navigation inside the image editing view. The original flow bounced users back to a thumbnail gallery after every single image, in a task that is inherently sequential.

Deliver with rigor — I documented states, interactions and edge cases as the deliverable. Denali’s engineers were reading those files as a spec.

Impact

In April 2023, Denali Advanced Integration signed a multi-year global Strategic Collaboration Agreement with AWS to deliver on-premises computer-vision quality inspection as a managed, end-to-end solution, built on the first-party workflows I designed.

AWS’s General Manager of Automotive and Manufacturing was quoted in the announcement. The productized result is listed on AWS Marketplace as Automated Quality Inspection on the Edge and Assisted Line Clearance, and pharmaceutical manufacturers deploying it report 800% throughput improvement with 50% human-error reduction.

At Tyson Foods, the underlying approach automated inspection of thousands of product carriers per line with high-90s anomaly detection accuracy, saving an estimated 15,000 hours of skilled labor annually.

Adoption: A partner staking a multi-year commercial agreement on it means an organization bet its business on the workflow being good enough to deploy repeatedly, under their own brand.

What I’d change

Batch handling for the human-in-the-loop review. Anomaly detection at line speed flags more items than a person reviews one at a time, and that review is where accuracy is maintained. The productized version moved toward this, clearing multiple lines at once but it should have been the starting assumption.

More scalable from the start. Each deployment required bespoke rework. A more configurable core with customer-specific variation pushed to the edges would have made it genuinely repeatable rather than repeatedly rebuilt.

AWS ended support for Panorama on 31 May 2026. The workflows outlived the service, Denali’s offering remains listed on AWS Marketplace.

Zoomed-out view of the project's Figma file, showing hundreds of artboards grouped into labelled sections. The upper region covers tasks and GUI in the cloud — station creation and management, dataset classification and labelling, model deployment. Below a dashed divider, a smaller region covers tasks and GUI at the edge, including image capture and inference runs viewed by the line operator. Collaborator avatars and internal references are pixelated — The whole file at full zoom-out — cloud console work above the divider, the edge device below. Collaborator avatars and internal references are redacted.