Until recently, big data was mined mostly for commercial purposes. Amazon, for instance, pushes real-time purchasing suggestions to customers based on products the customer recently browsed or purchased.
And data mining is now evolving to location-based marketing, which targets consumers based on their current location, notifying them via mobile devices of deals in their immediate area.
But big data also has great potential to predict crime, crime hot spots and criminal trends, said John DeCarlo, a professor of criminal justice and forensic science at the University of New Haven. “When we look at technologies like criminal path mapping and predictive policing, we begin to see the same potential in using big data that retailers have observed.”
University researchers in Israel think mining big data can be used to predict the location of terrorists.
Like everyone else, terrorists leave digital traces with much of what they do, whether using e-mail, cell phones or credit cards. This data can be mined, the researchers believe, to fight terrorism.
The Tel Aviv University team has developed context-based search algorithms to analyze digital data on the fly. The algorithm works like a computerized Sherlock Holmes, taking information from phone records, e-mails or credit purchases and reducing them to a set of variables. The output is a probability map used to predict future movements of a person of interest.
“The outcome is used in real time to track the likely location of terrorists on the move,” Tel Aviv industrial engineer Aviv Gruber told Homeland1.
Police and homeland security officials in this country are also mining big data, for the same reasons. Big-data mining will play a significant role in securing the presidential inauguration ceremony in January, where monitored data streams will include visual surveillance from cameras and license-plate readers mounted around Washington, incoming 911 calls, traffic reports, weather, news feeds, Facebook, Twitter and Instagram, all distilled into something meaningful.
“Some of this data is highly structured, but much of it isn’t, so pulling it together into a data warehouse using traditional integration technology isn’t practical even if it were possible, which it isn’t,” said John Crupi, chief technology officer at JackBe
New mining tools are emerging to provide authorities a better view of whatever might be going on.
“We now have the ability to mash these important data feeds together, directly from the source as they come in, to get a quick visual of the current situation,” Crupi said.
New York City already has a big data tool. In August, the New York Police Department, in partnership with Microsoft, launched a state-of-the-art crime prevention and counter-terrorism technology based on big-data mining called the Domain Awareness System.
The system tries to make tactical sense out of real-time data collected via everything from civic closed-circuit cameras and incoming 911 calls to license-plate readers and radiation sensors.
The NYPD can now do things like track a vehicle and quickly determine almost everywhere it’s been for the past several days or weeks, rapidly access arrest records and all 911 calls related to a particular incident, generate geospatial and chronological crime maps to expose any patterns, and verify immediately whether radiation alarms are triggered by dirty-bomb components or by harmless medical isotopes.
[Ed.: One of the accompanying feature articles this month describes a new situational awareness tool undergoing testing by the New York Fire Department.]