Skip to content

01

AWS Panorama

Bringing computer vision to common PoE cameras that were never built for it and designing the console that made an edge ML appliance manageable.

AWS Panorama · 2020–2023 · General availability October 2021

  • Edge ML
  • Console Design
  • Information Architecture
  • Design Systems
Tier 1 launch October 202150+ screens redesigned after an architectural pivot278+ usability bugs resolved pre-launchOne framework governing every console action

Almost none of it had precedent at AWS

Most computer vision runs in the cloud. That works until the cameras are in a poultry plant, a roadside cabinet, or a warehouse mezzanine where bandwidth is limited, latency matters, and footage can’t leave the premises.

Panorama was the answer: an edge appliance that connects to existing IP cameras and runs models locally. Customers keep their cameras; the appliance provides the intelligence.

The design problem was that this was a physical device you order, ship, unbox, plug into a network and register, managed through a cloud console that also handled applications, models, camera streams, deployments and over-the-air software updates. Console patterns existed for cloud resources. They did not exist for a box in a factory that might be securely offline.

And before launch, the service went through an architectural pivot. Work that had been settled reopened: how customers would build applications, what the resource model actually was, and how any of it should be managed.

Initial personas across two axes. An IDE serves one corner and a dashboard another; the console had to hold the middle, where neither would do.
Five roles, each with a different reason to be in the console. What counts as a severe action depends on which of them is taking it.
Five stages, and who is in the room at each step.

One framework instead of a hundred decisions

The temptation on a console this large is to decide confirmation and consequence patterns screen by screen. That produces a product where deleting a device feels like deleting a tag.

Instead I ran a global audit of every action in the console and rated each for severity using a CFS matrix. One consistent model, applied everywhere. This mapped the weight of an interaction matched its actual consequence, and new screens inherited the standard rather than inventing one.

Alongside it I redesigned the resource information architecture and most list and detail views, 50+ designs establishing what a device, application, node and camera stream each were, and how customers moved between them. Then catalogued every status lifecycle for every resource type, which later became the foundation for troubleshooting.

Commonality plus frequency plus severity gives one impact number, and the number sets the priority. Arguing about a score is more productive than arguing about a ticket.

Refusing to ship without an escape hatch

The architectural change meant editing deployed applications wouldn’t make GA. Based on our research it was a big deal for customers. An ML application needs iteration to reach a working state, and redeploying from scratch for a minor change was a cost our weekly customer interview sessions confirmed was unacceptable.

I pushed back rather than accepting it as descoped. Working through the options with engineering, I landed on Replace function using existing functionality to behind the scenes delete the application on the customer’s behalf and redeploy with an updated configuration, prefilling the wizard with everything from the previous deployment. That took resolving a lot of engineering edge cases to keep the substitution invisible whether the customer used the console or the API.

Replace became the primary method engineers and Solutions Architects used to update deployed applications.

Two paths into one wizard, prefilled from the previous deployment. Delete-then-redeploy had to read as a single action whether the customer used the console or the API.

The same instinct applied to software updates. I catalogued the full matrix of update types: optional, mandatory, major version, backwards-incompatible, and the online/offline complications of an edge device. Then I pushed the team on update policy distinctions, not just interface: how often mandatory and major releases should be forced on customers at all.

That argument shipped as platform behavior. In January 2023, AWS added AllowMajorVersionUpdate to the Panorama API, making major version updates opt-in.

Optional, required, mandatory, major, and the device that was offline when it landed. Each combination costs the customer something different.

How I worked

Set the north star — A customer gets from an unboxed appliance to a running computer vision application without needing to understand the platform’s internal object model.

Align the organization — Customer interviews split across three designers; four internal research sessions with ProServe, Solutions Architects and engineers from other AWS services; and I led the studies on device setup, where you learn that the person plugging in the appliance isn’t the person who’ll use it.

Engage the constraints and push them The Replace API, and the update-frequency argument.

Deliver with rigor — Reviewed 100+ console screens ahead of UX sign-off, worked through 278+ usability bugs with engineering, and ran a full audit of every help panel in the console with the content and engineering teams.

The console design file end to end, organised by feature and flow.

Rebuilding how the team worked

At the time Figma had only been approved for use at AWS for several months. Three designers shared one Sketch file that only one could edit at a time. Parallel work meant multiple files, duplicate screens, and no global view of the console. Every engineering review meant exporting a PDF. Partners rightfully complained about multi-hundred-page documents where the changes since last time were impossible to find.

I investigated migrating to Figma against three risks: a live GA timeline mid-review, two designers who hadn’t used the tool, and the migration effort itself. With the team on board, I rebuilt 100+ screens in two weeks, built a common external component library, organized by feature and user flow, with annotations alongside the designs so engineers had intended behavior in context.

The design team stopped managing PDF exports. Collaboration with the product team was amplified. All three designers worked simultaneously in one file. Engineering implemented with more autonomy, because the intent was written next to the design.

Impact

AWS Panorama reached general availability on 20 October 2021 as a Tier 1 launch, with the appliance available for purchase and a device SDK adopted by hardware manufacturers including ADLINK, Axis, Basler, Lenovo and Vivotek.

The service went on to serve Tyson Foods, automating inspection of thousands of product carriers per line, and Deloitte’s deployment for the Victorian Department of Transport. That consisted of Panorama appliances analyzing 1,200 cameras across 23,000 km of road network in real time.

Platform work continued through 2023, including Direct Hardware Access and a redesigned application SDK.

What I’d change

I’d have built a more methodical way of collecting results from our feature experiments and customer feedback. We shipped steadily and well; we measured what happened afterwards far less consistently than we designed what should happen. That gap is the thing I’d close first.

I’d also have argued for troubleshooting as launch-critical rather than post-GA. The fuller version of that argument is in the deep dive below.

Deep dives

Remote Troubleshooting → When security removed SSH at launch, every customer lost their only debugging tool. The ask was to add troubleshooting features, the customer need turned out to be more nuanced.

Defect Detection → Computer vision quality inspection for people with no ML expertise — later productized by a partner under a multi-year AWS agreement.

AWS ended support for Panorama on 31 May 2026. The work here reflects the service as it shipped.

Sources for public claims