What is Open-Source Intelligence (OSINT)?

OSINT is the acronym for Open-Source Intelligence. It refers to the collection and analysis of open source data – i.e. data that is freely and publicly available such as news articles, open social media pages, blogs, radio and television broadcasts, etc. It differs from related fields such as political/security risk analytics in that the focus is firstly on what data you are collecting from where.

Before you can focus on answering a specific client question, you need to know what source of data will be best suited to answering that question and what are the limitations and biases contained within that source.

For example, if you are tracking the number of and intensity protest actions in a country with a closed media environment or with strong government control over the media space, you cannot simply collect reports of incidents from local newspapers and media channels. You will, at minimum, have to collect from diaspora and reputable international sources, in addition to social media sources to fill out your collection. These sources will have to be vetted for the specific question you are trying to answer, and even with your best efforts you will still have to make it clear to the client that as you move into more rural areas of the country, away from the capital where international journalists tend to congregate, the data will become more circumspect. With this absence of certainty in mind the client should adopt a higher risk stance.

Of course it is still important to have the best analyst team with area expertise and expert tradecraft provide insights for decision-makers based on the data collected. But at VoxCroft, we have decided to to do OSINT and put the data first, and we see open data as the most effective/efficient window into what is going on in the world.

And the great thing is that this field is not only the purview of governments anymore as decision-makers in governments, businesses, and humanitarian organizations need fast, reliable and accurate information on what is happening in the world to be able to make better decisions. Open-source media provides a wealth of data that can solve this need.

Open-Source Intelligence works by:

• Collecting open-source data from vetted sources
• Converting data into a usable format by methods such as metadata tagging, translation and/or transcription
• Filtering data based on relevance to a specific intelligence question
• Analyzing data to discover trends and events
• Presenting trends, alerts and events to decision makers

In the words of VoxCroft Co-CEO, Barend Lutz:

“I have spent most of my career in this industry, but I am still surprised at the depth and scope of the insights you can gather with open data if you approach it correctly. I am also surprised that more attention is not spent on modernizing the methods used in collecting, analyzing, and utilizing the insights gained from OSINT. There are vague copies of these methods in fields such as political risk analysis, but the deep and scientific tradecraft that is needed to do it right seems to be lacking in the industry at large.”

As in most industries there are calls for automation, but many have gone too far; handing over the reigns to computers too quickly. There will surely be a day that general artificial intelligence is smart enough to compete with a trained human risk analyst, but that day is not here yet. For now, we need to work together in what is called intelligence augmentation, where the machine does what it does best (process vast amounts of data quickly and find patterns and alterable events) and the human does what she does best in training the machine, compensating for its limitations, and providing expert judgement and analysis.

The best OSINT solutions therefore integrate human intelligence tradecraft with machine learning and automation technologies to enhance the strengths of both – Trained human analysts are better at making judgments and understanding context while machine intelligence can analyze millions of data points in a second.

Various government agencies, humanitarian organizations and businesses are already relying on OSINT to be their eyes and ears in the world. One VoxCroft customer was concerned about the safety of their delivery and sales staff in South Africa. The head of logistics contacted VoxCroft to build them a risk analytics solution so they could keep track of crime and protest events and natural disasters that could pose a risk to their employees.

The VoxCroft solution sources social media posts, news articles, and other open-source data that might be relevant to the customer. This specific customer was specifically interested in protecting the routes that the employees would take daily, so our data is filtered geographically based on this criteria. Human analysts work in conjunction with VoxCroft AI to analyze the data and figure out what is pertinent to the customer and the safety of the employees. The logistic manager can keep track of any events and trends on a dashboard customized to their needs on the VoxCroft Project Arrow platform. Alerts are sent via text to employees in the field and the logistic manager if an event occurs that might pose an immediate threat. (read more about this case study here.)

In humanitarian crises such as natural disasters, OSINT can help first responders and aid organisations. Following the 2010 earthquake in Haiti, students at Tufts University manually monitored tweets coming from survivors and those in need and mapped them onto the Ushahidi platform to help first responders, saving hundreds of lives. Modern platforms such as Project Arrow can be spun up within hours to automatically filter through Tweets and other data to provide live maps during crisis events.

Casey Schmidt, VoxCroft Co-CEO has a wealth of experience in understanding crisis situations. He is hopeful about the future of OSINT in helping manage these situations:

“Project Arrow’s algorithm sifts through millions of data points from hundreds of data sources and displays what is most relevant for the end-user. The information is presented in either a customized online database or text message and email alert. For example, humanitarian organizations can set up a Project Arrow instance to monitor and track security threats to aid personnel, implementing partners, and aid deliveries, and receive a text message alert when especially threatening incidents are discovered within Arrow’s data streams. Arrow also reports on dynamics and factors that pose an indirect security risk to humanitarian activities, such as roadblocks, looting, and political protests. Indeed, if time is the most precious commodity in a crisis, then Project Arrow’s processing and analytical speed is a critical force-multiplying capability to humanitarian response.” Read more here.

Open-source data is booming, and technologies and methodologies are keeping up to convert this data into valuable insights to decision makers for various applications. OSINT is fast encroaching on traditional risk analytics in providing more accurate, timely, and relevant information.

If you want to learn more about how OSINT could help you manage your business, schedule a call here.