How can you adapt edge AI to different contexts?

Running AI on the edge, like any workload is exponentially more complex that running on the cloud. For starters, consider the "uneven edge." If you are building for your own walled garden on your own hardware (such as Apple) you have a somewhat "even edge" - meaning you can rely on the population of edge devices to be fairly similar, or predictably different. Most of us will build solutions for an uneven edge, meaning there will be many types, sizes, and strength of devices on the edge that we must cater for. Where as, building on the cloud means you have an "even center" (if I may) and you don't care if the edge is uneven. These are examples of thinking that should go into defining your AI edge strategy, goals and constraints.

One deep nuance is whether you will use embedded models that are on the edge - many devices will have chips (CPU/GPU) that come with standard AI models, and AI accelerators "embedded" on the chips. Embedded models are much faster, lighter and reliable that models that are installed with the OS, or downloaded and installed as a library or an App. However models embedded on chips can only be upgraded when the chip is upgraded (the consumer purchases a new device) and predicting and planning for chip upgrades is not an area with best practices and industrial strength ecosystem relationships. So cloud-edge, device-edge and hybrid makes sense, consider the edge-chip as well, especially when you want lightning bare-metal performance.

One opportunity with Edge AI, is to train on the edge. The ingress/egress costs of training in the cloud can be very expensive, especially with large data sets that may sit in another cloud with a CDN in between. Hence, I have found that for the many trial and error training sessions of a model, especially in the early stages, training on the edge, with data stored on the edge, can significantly reduce the cost and time/latency of training a model. However, this does come with the overhead of ensuring that your stack on the edge matches your stack in the cloud, so that when you deploy the model back to the cloud there is very little retrofitting needed to move the model from the edge to the cloud, and cloud to edge.

Consider this, at some point, we will replace chips on edge devices they way we can replace a SIM card on a mobile device. Or, chips will be re-printable and you can take your chip and put it into a machine that is connected to the cloud, and re-print it (not commercially available as yet). This will move the conversation from AI on the edge, to AI on a chip, or AI on an SD card, or AI on a SIM card. Either way, the arrangement on the edge will change, and the chip is about to become the new upgrade component. Hope this helps.