It’s always a surprise, and frequently an unpleasant one, when you arrive at the airport rental car that’s got your name on it. It’s temporary transportation, and my life and safety are not at particular risk—so no big deal. Nonetheless, it can make for a challenging environment driving around an unfamiliar city if you’ve got dangerous blind spots—broad and poorly positioned pillars. That’s the luck I have this week while in town for the Army Aviation Mission Solutions Summit in Nashville. Nashville’s a great town with courteous drivers, by the way, so I’m thankful for that. But imagine you’re flying a helicopter into hostile territory. Then blind spots are a big deal.
Army Aviation lethality means operating without limitation by weather, terrain, threats and operator perception of the surrounding environment. Operator situational awareness is a necessary cornerstone to this capability. The more one can see and comprehend, the faster and more confident the engagement.
And it’s not just human seeing, but machine seeing. At this summit, Abaco is demonstrating an image processing framework and advanced GPU-accelerated computing solution that can support many video and image processing applications—and in this specific demo, 360° frame-rate spherical situational awareness…a skybox view of the world. ImageFlex is part of Abaco’s award winning AXIS multi-processor software tool suite. These image processing capabilities are computationally intensive and so leverage Abaco’s GPU co-processing technologies.
The demo ingests six HD video cameras with different fields of view and constructs an integrated spherical view. The demonstration allows us to navigate around the image space, the summit floor venue in this case, using a hand controller as a surrogate for what a pilot would do with heads up systems as they survey their aircraft-surrounding environment. It’s pretty wild. The spherical image construction is a key part of the demo.
Creating a single 360° view
Stitching all the camera views into an optically correct image requires distortion correction and transformation of each camera’s image. ImageFlex then blends the image edges into a single 360° view. This is the typical processing required of a distributed aperture system. Distributed apertures around the aircraft open windows to unobstructed spherical views outside the aircraft. Distributed aperture systems can warn pilots of incoming aircraft and ground fire threats, provide day/night vision, enable tracking of wingmen or unmanned teaming assets and other awareness applications.
If that wasn’t cool enough, it also demonstrates some select image processing functions within that image mosaic—in this case, Harris Corner Detection followed by Optical Flow. Corners are stable image features from image to image as the view point changes. There are generally large image variations around corners providing valuable image recognition data which then enables data about your aircraft position location relative to points of interest in the images. Optical flow then provides a pattern of apparent motion caused by the relative motion between the aircraft and the imaged environment. LIDAR and high resolution radar sensing can also be integrated.
Where, who, what
It’s all about seeing where you are; where you’re going and who and what is around you. Degraded visual environment operations, manned-unmanned teaming, aircraft survivability, multi-sensor targeting are all key lethality capabilities enabled by operators’ ability to see and comprehend their environment. Creating an image mosaic, running critical image processing algorithms all at full motion is compute-intensive. ImageFlex and advanced GPU computing are just a couple tools Abaco Systems offers to advance Army Aviation overmatch capabilities.