Fundamentals
Edge AI vs Cloud AI
Cloud AI runs models in a provider's data centre. You send data out, the platform processes it, and results come back. It's elastic, quick to start, and well suited to training large models and heavy batch workloads.
Edge AI inverts the pattern: the model moves to the data. Inference runs on local hardware, so latency drops to milliseconds, camera feeds stay on the LAN, and the system keeps operating through connectivity drops. Costs shift from recurring usage bills to owned infrastructure.
Neither model wins everywhere. A sensible architecture is usually hybrid — sensitive, latency-critical workloads at the edge, with cloud used for large-scale training, long-term analytics or anything it genuinely serves better.
Full guide: Edge AI vs Cloud AI
Edge AI Hardware
What Is an NPU (at the Edge)?
An NPU — neural processing unit — is a processor built specifically for the mathematics behind neural networks: dense matrix multiplications and convolutions. Where a CPU handles a bit of everything, an NPU is tuned to do this one class of work efficiently.
That focus shows up as performance per watt. NPUs can run vision models at low power, which is why they appear in AI cameras, fanless industrial computers, compact appliances and embedded system-on-chip designs — places where a big GPU won't fit or can't be cooled.
The catch is maturity: model support, quantisation toolchains and runtimes differ between vendors. An NPU is only as good as the software stack behind it, which is why hardware selection should start from the models you intend to run.
Edge AI Hardware
NPU vs GPU for Edge Inference
An NPU is an efficiency play: dedicated silicon that executes standard neural-network operations at low power with predictable latency. A GPU is a flexibility play: thousands of parallel cores with a mature software ecosystem, able to run almost any model architecture.
For many concurrent, lightweight camera streams, NPUs are often the better fit: high stream density per watt and fanless enclosures are possible. For heavy transformer-based vision, vision-language models or fast-changing model stacks, a GPU's raw throughput and software maturity usually justify its power and cost.
Many site deployments combine both — NPUs handling steady-state camera analytics while a GPU serves heavier models and bursty demand. Sizing that mix is a workload-mapping exercise that should happen before any hardware is bought.
Edge AI Hardware
How Much TOPS Does an Edge AI System Need?
TOPS — trillions of operations per second — measures an accelerator's theoretical peak throughput, usually quoted at low precision such as INT8. It's a useful ceiling, not a promise: real throughput depends on the model architecture, quantisation, memory bandwidth and how well the runtime uses the silicon.
Sizing starts from the workload, not the number. Count concurrent camera streams, their resolution and frame rate, and the models each requires. Each of those translates into compute and memory demand that a TOPS figure alone doesn't capture.
The practical rule: treat vendor TOPS figures as a filter, not a decision. Benchmark your actual models and stream counts on candidate hardware, then keep headroom for growth. A system sized this way will outperform one bought on a bigger number.
Computer Vision
Edge AI for CCTV Analytics
Most organisations already own their most expensive AI component: the cameras. Edge AI for CCTV connects existing IP streams to a local appliance that decodes each stream and runs vision models on it — detection, classification and tracking happen on-site, frame by frame.
Typical analytics include person and vehicle detection, intrusion and restricted-area alerts, occupancy and queue measurement, PPE compliance and retail flow. What reaches the control room is metadata and alerts — not hours of uploaded footage.
Two design defaults matter. First, video stays on the LAN; only insights move. Second, deployments should start from operational analytics — sensitive functions such as facial recognition are not enabled by default. A pilot on a handful of cameras proves value before any fleet-wide rollout.
Computer vision on the edge
Offline AI
Offline AI Systems Explained
An offline AI system runs its entire pipeline — models, runtime and data — with no internet connection at all. Inference doesn't pause when the link drops, because the link was never part of the design.
That matters wherever connectivity is a risk rather than an assumption: factory floors, warehouses on industrial estates, remote sites, and any operation where a dropped connection would otherwise mean a stopped line or a blind control room.
Design changes in predictable ways: model updates arrive on a schedule or over intermittent links, monitoring is local, and the cloud becomes an optional aggregation layer that syncs when connectivity returns. Done well, offline operation is simply normal operation.
Offline & private edge AI
Computer Vision
Choosing Hardware for Computer Vision
Computer vision hardware should be chosen from the workload backwards. The questions that actually drive the decision: how many concurrent streams, at what resolution and frame rate, running which models, inside what latency budget, in what environment.
The answers sort themselves quickly. Many efficient streams favour NPUs; heavy transformer-based models or large multi-stream pipelines favour GPUs; dusty, hot or power-constrained sites push toward fanless industrial systems. Camera counts, PoE ports and storage retention then shape the rest of the platform.
Two habits prevent expensive mistakes: leave expansion headroom for the models you'll run next year, and pilot on real footage from your own site before committing to a fleet. Vision models behave differently on your cameras than on any benchmark.
Edge AI hardware categories