Challenges and Opportunities in the Rapidly Growing Edge Computing Market

What innovations will spawn from the challenges and concerns that edge computing brings?

During the past six months as Head of Product at FØCAL, I’ve spent much time researching edge computing, mostly with a focus on computer vision use cases. The pace of low-cost, low-power edge computing device announcements is staggering. Single board computers, including those with GPUs for handling machine learning and AI use-cases are quickly moving from the home-builder and hacker communities to large scale product companies. Edge-related manufacturers like Aaeon, Up, BeagleBoard, and Google’s Coral are rapidly expanding the previously somewhat limited offerings from Intel, NVIDIA, and Raspberry Pi. Moreover, there’s still a whole new wave of AI processors in the works such as Quadric and their EPU (edge processing unit) and Mythic AI speeding up AI with analog computing, both targeting edge device use-cases.

This trend is a result of the growing demand for mobile robotics, edge intelligence, autonomous vehicles of all shapes and sizes, physical security solutions, and everything IoT.

Security Concerns

There are two primary concerns with this growth spurt. The biggest is already well known but still lacks ideal solutions. The objective of most hardware vendors building edge solutions is to provide more intelligence while keeping COGS as low as possible. Quick-start hardware platforms and advances in software libraries that can tackle complex applications are making it easier for almost any engineer to rapidly prove product concepts. This can dangerously lead to rushing an immature product to market that compromises their customer’s data or network.

The typical penetration and vulnerability tests that are common in cloud software aren’t as prevalent in many of the lower cost hardware devices. Some percentage of low cost and insecure devices will continue to exist no matter what standards are in place. These devices may eventually find their way onto enterprise networks, creating potentially large scale vulnerabilities and risks. It’s still early, but there are already companies such as Oort building a security fabric to provide a layer of protection from IoT devices. Just as the bring-your-own-device trend brought plenty of fear, uncertainty, and doubt, the low-cost edge computing market will bring another wave with slightly different requirements and solutions.

Device Performance and Deployment Challenges

Device performance is the second concern that is leading to new and innovative solutions. I’ve learned from many edge-computing founders and engineers building drone and robotic products, that the existing development lifecycle includes many iterations of refactoring pipeline logic, tuning algorithms or neural networks, and hardware experiments before finding the right combination to meet the product and business requirements. And that’s if they work out a solution at all. Many concepts never make it to production because of this complexity.

While there are the typical code-based profiling tools available to these developers, they are quite limited in guiding how best to optimize the entire hardware and software layers. Some companies are targeting performance-related features, but they only focused on the hot marketing buzzword areas such as mobile phone processors and the optimizing the most commonly used neural network frameworks. I’m not arguing against these products; they are valuable. It’s just clear that these components are a small part of an overall solution. Optimizing individual components can only get you so far, which means these performance tools need to expand their scope. At FØCAL we're targeting profiling and simulation that supports the entire vision pipeline. These tools allow engineers to experiment with the impact of pipeline and hardware changes on overall performance metrics before spending the time to reengineer code.

There’s even a wave of new operating systems to keep the overhead minimized on embedded systems and edge devices. I haven’t dug into this area yet, but you can checkout Yocto and Ubunto Core.

Edge computing will continue to evolve rapidly, creating opportunities for new companies, products, and markets. Even with the introduction and deployment of 5G networks, there will always be use-cases for edge inference and intelligence. It’s not always feasible to send data to the cloud for processing.

Originally posted on Medium.