Open Source Analytics

Extract meaning from high-velocity open source information

Historically, deriving intelligence from open-source digital content has been complex and time consuming, particularly when high-accuracy results are required. Generating meaningful signals from data sources such as web content and open-source video has required substantial labor on the part of analysts. We are seeing an increasing need to extract meaning from customers with access to high-velocity, open-source information. 

DISCOVER RELATIONSHIPS

Go beyond basic consolidated search by observing connections between extracted entities, source content, and other relevant metadata. These complex relationships naturally lend themselves to scalable, flexible graph analytics such as property graphs. 

STORE YOUR DATA

Amazon Neptune, a scalable and fully managed graph database, offers reliable, distributed, and fault-tolerant storage for highly interconnected data – a logical storage solution for analyst-friendly OSINT graph data. 

DISCOVER RELATIONSHIPS

STORE YOUR DATA

Go beyond basic consolidated search by observing connections between extracted entities, source content, and other relevant metadata. These complex relationships naturally lend themselves to scalable, flexible graph analytics such as property graphs. 

Amazon Neptune, a scalable and fully managed graph database, offers reliable, distributed, and fault-tolerant storage for highly interconnected data – a logical storage solution for analyst-friendly OSINT graph data. 

VISUALIZE YOUR RESULTS

Results from pattern match queries can be displayed in an interactive, browser-based graph visualization. Metadata from the original source can be used to construct relationships between nationality, publisher, and the source of the article. The content of the article and text from each video transcription is processed through Amazon Comprehend to identify entities mentioned in the article text.

STAY SECURE

A collection of AWS Lambda functions can be configured to interact with Amazon Neptune (via Gremlin REST API), which resides in a secure virtual private cloud (VPC). Using AWS API Gateway, secure web services can be established for a prototype web application to connect and create a graph visualization. The diagram below illustrates the end-to-end pipeline. 

VISUALIZE YOUR RESULTS

STAY SECURE

Results from pattern match queries can be displayed in an interactive, browser-based graph visualization. Metadata from the original source can be used to construct relationships between nationality, publisher, and the source of the article. The content of the article and text from each video transcription is processed through Amazon Comprehend to identify entities mentioned in the article text.

A collection of AWS Lambda functions can be configured to interact with Amazon Neptune (via Gremlin REST API), which resides in a secure virtual private cloud (VPC). Using AWS API Gateway, secure web services can be established for a prototype web application to connect and create a graph visualization. The diagram below illustrates the end-to-end pipeline. 

TECHNOLOGIES

Novetta Mission Analytics provides on-demand, pattern-based analysis that enables you to fully understand the impact of open source data on your operating environment in real time, based on the individuals and entities that matter most. Novetta Mission Analytics fuses proprietary and third-party data sources to deliver accurate, evidence-based analysis of individual, group, and organizational behavior. Our four global data centers ingest, analyze, and curate open source data around the clock. Read more about Novetta Mission Analytics here.

For tailored capabilities that complement Novetta Mission Analytics, Novetta’s Open-Source Intelligence (OSINT) Processing Pipeline is a customized platform that significantly reduces the effort required to process open-source data through a combination of machine learning, graph database technologies, and Amazon Web Services (AWS). Leveraging the latest API-driven machine learning services from AWS, the OSINT Processing Pipeline extracts metadata from open-source video and news articles based on the technologies and workflows such as the following:

• Process customer data through with AWS Elemental (Media Convert)
• Extract text from video audio track and save to AWS S3 using Amazon Transcribe
• Extract entities (e.g. location, organization) and entity types from transcribed and news article text via Amazon Comprehend
• Load high quality, high confidence results into Amazon Neptune
• Provide secure microservices that return information stored in Amazon Neptune leveraging AWS API Gateway and AWS Lambda
• Conduct graph-based analytics and named entity resolution through custom Novetta models

GET STARTED