Geospatial Intelligence

Geospatial is a term that has only found common usage this millennium.  Even the National Geospatial-Intelligence Agency (NGA) with its primary mission of collecting, analyzing, and distributing Geospatial Intelligence (GEOINT) was known as National Imagery and Mapping Agency (NIMA) until 2003.  The name change reflected the transition for photographs and paper maps from tools of common use to items of nostalgia.

Now Part of Everyone’s Daily Life

Can anyone remember writing a date and location on the back of a Polaroid?  GEOINT is now at our fingertips and can even be summoned by voice.  We are guided by GPS to the restaurants of our choice.  We all know that roads, rivers, borders, and 3-D buildings now come in layers and can be turned on and off in our phone Apps or Geographic Information Systems (GIS) at will.  If we cannot see over our neighbor’s fence, we find imagery from a satellite that can.

The Numbers are Astounding

It is very clear what changed.  Cameras became digital, computers to process and serve data became nearly omnipresent, and unmanned vehicles to host the cameras became commonplace.  But, the numbers are truly astounding.  Billions of cameras are sold each year.  Over a billion are in smart phones capable of tagging each image with time, position, and orientation.  Some cameras matrix hundreds of individual focal plane arrays, the imaging chips in cameras, to form a single camera capable of imaging tens of square miles in a single frame from an aircraft with many frames captured per second. Millions of drones are sold commercially each year.  Hundreds of Predator and Reaper drones have been deployed to militaries across the globe to provide reconnaissance and surveillance information for digestion by commanders and sometimes the viewers of television news.  A single commercial satellite can capture roughly a million square miles of imagery with resolution of less than a meter per day and revisit the same area nearly once per day.

With so much data, computers have naturally become essential tools for distilling intelligence information from it.  To meet the challenge, computers have gone massively parallel. As silicon chip makers feared the demise of Moore’s law due to anticipated physical limits on minimum transistor size, they worked on coupling multiple processors to divide and conquer problems in parallel. 

CPUs for desktop computers can now have on the order of 20 processing cores producing over 10¹² complex operations per second.  A Graphics Processing Unit (GPU) on the same computer can have thousands of cores.  Each GPU core is a bit slower, but together they can perform more than 10X the number of operations of the CPU.

Automation

Parallel processing is ideal for deriving GEOINT from imagery, because the processing of each pixel in an image is often essentially independent of the processing of all other pixels.  It can, therefore, be assigned to a single core and thousands of pixels can be processed simultaneously by the thousands of cores.  The sensor in Figure 1, for instance, produces around 2 million pixels per frame and 30 frames per second.  To geo-rectify each frame, each pixel is projected onto a model of the earth’s surface.  The process fixes ground features, such as roads and trees, in the resultant geo-rectified imagery, so the only objects moving over time are those that are actually moving relative to the ground (e.g., vehicles and pedestrians).  To geo-rectify one frame, about 2000 pixels are processed in a batch.  Therefore, about 1000 batches have to be processed in 1/30^th of a second.  GEOINT is derived by tracking human activity, such as vehicle positions as shown in the figure.  If there are potentially thousands of movers in the geo-rectified video then the task of following each can similarly be distributed to a different processing core, so all can be tracked simultaneously.  Each track can be disseminated over network lines to any place in the world for easy display in real-time on a GIS, such as Google Earth. 

Living on the Edge

It is, of course, no surprise that decision makers in the field would benefit from GEOINT like this provided from a central massive computing center.  But, now these massively parallel computers in rugged form can be carried in one hand and easily deployed on virtually any aircraft, ground vehicle, or boat to enable Edge Processing, which means processing of data at the source.  Clearly GEOINT derived locally can be transmitted home to aid master decision making, but it also can be used locally to provide a local decision maker better information for autonomous operation. 

Artificial Intelligence

The natural question these days is “With all of this processing power, why do we need a human decision maker at all?”  Algorithms based on Artificial Intelligence (AI) for voice and facial recognition have shown huge improvements over the last decade and are used widely in smart phones and social media.  What if they could be extended to label each object a processor tracks with classifiers/identifiers?  Figure 2 provides the concept and also hints at some of the challenges.  Like human intelligence, AI is fundamentally limited by the quality of the data it receives.  Therefore, classification (e.g., automobile) is easier than identification (e.g., my father’s F-150), nesting is beneficial (e.g., automobile/pickup/Ford/F-150), and confidence scores are important.  Predictions for AI development are seemingly always optimistic.  In 2015 the Guardian said, "From 2020, you will be a permanent backseat driver."Well, AI for driver-assisted vehicles has made much progress, but in 2020 it certainly has not replaced the human yet.  The same is true of AI for GEOINT.

Benefits versus Risks

The benefits of recent GEOINT advances are quite evident, but so are the risks.  The risk that health or financial records, social media information, current location, or travel history becomes associated with an address in a GIS somewhere is worrisome.  As with all advances, security technologies and procedures will have to evolve to address these types of concerns as they develop.