In newspaper articles, emails, and other texts, it is natural to reference dates in relation to the publication date. For example, you will commonly see phrases like “yesterday” and “two weeks ago.” However, if you read an article a year after it was published, these timeframes are not immediately clear. When analyzing articles using natural language processing (NLP) techniques, relative dates referenced in an article will not be interpreted properly, if at all, since there is no context for the models. Thus, in order to effectively use tools such as event extraction and question-answering engines, relative references need to be…
BASELINE – March 2020
An ML Newsletter from Novetta
Welcome to the March 2020 installment of BASELINE, Novetta’s Machine Learning Newsletter, where we share thoughts on important advances in machine learning technologies likely to impact our customers. This month’s topics include: AdaptNLP, an open source framework for state-of-the-art Natural Language Processing (NLP) A new competition for geospatial AI applications corresponding with the release of a Synthetic Aperture Radar (SAR) dataset A new and improved way of training language models StanfordNLP gets a new coat of paint Research into training large ML models with CPUs A new tool designed for rapid prototyping quantum machine learning models AdaptNLP At Novetta, we…
BASELINE – February 2020
An ML Newsletter from Novetta
Welcome to the February 2020 installment of BASELINE, Novetta’s Machine Learning Newsletter, where we discuss important advances in machine learning technologies and their potential impact on our IC / Defense customers. This month’s topics include: New breakthroughs in language models, including conversational AI Concise APIs that simplify model implementation Promising new approaches in computer vision and graph neural networks Meet Google’s Meena Google’s Meena has raised the bar for open-domain chatbots, achieving near-human-like interactions. Meena is a conversational model designed to chat sensibly about nearly any topic while taking into account the context of the conversation. In addition to Meena,…
BASELINE – January 2020
An ML Newsletter from Novetta
Welcome to the January 2020 installment of BASELINE, Novetta’s Machine Learning Newsletter, where we share thoughts on important advances in machine learning technologies likely to impact our customers. This month’s topics include: New software tools that improve the process of developing machine learning models Improvements in language models, with some research showing that synthetically generated text can fool human reviewers The “What-If Tool” and Manifold While the primary goal for most machine learning models is high accuracy, we also need to understand why the model is making certain predictions. We’ve recently been working with the What-If Tool from the Google…
BASELINE – December 2019
An ML Newsletter from Novetta
Welcome to the December 2019 installment of BASELINE, Novetta’s Machine Learning Newsletter, where we share thoughts on important advances in machine learning technologies likely to impact our customers. This month we discuss the following areas: At AWS re:Invent, new services and features relevant to our data science, image processing, and IoT teams were announced. Netflix open-sourced a platform to simplify the engineering side of data science. Improvement in quality of deepfakes continues, while Facebook and Kaggle launched a competition to detect deepfakes. SageMaker Improvements and AutoML At AWS re:Invent, new Amazon SageMaker capabilities and features were announced intended to address…
BASELINE – November 2019
An ML Newsletter from Novetta
Welcome to the first Novetta Machine Learning Newsletter, where we will share thoughts on important advances in machine learning technologies likely to impact our customers. We’ll start with a few highlights that have impacted our work in 2019, including GPU-based acceleration of data tasks, the inflection point in performance on NLP tasks, speech recognition, and graph neural networks. GPU-Enabled Data Science with NVIDIA RAPIDS GPUs provide significant speed improvements when processing images, but when working with other data types developers have typically used slower CPUs. NVIDIA RAPIDS is a “suite of open source software libraries” that enables much of the…
Adversarial Attacks in Machine Learning: Defenses
Part 3 of 3
Introduction to Adversarial Attack Defense Methods In our previous blog posts, we introduced and experimented with adversarial attacks against deep learning models. Here, we explore ways to mitigate or decrease the effectiveness of some of these attacks with three defense methods: 1. Modifying the feature space of the neural network itself 2. Modifying an input to a network before it is used for inference 3. Modifying the training data to account for adversarial inputs Modifying the Feature Space of a Neural Network In our first experiment, we implemented a technique [1] that modifies the feature space of the weights for…
Adversarial Attacks in Machine Learning: Experimentation
Part 2 of 3
As machine learning (ML) becomes more prevalent, bad actors seek to take advantage of our confidence in models’ predictions. An example of this can be found in autonomous vehicle safety, wherein the application of a small sticker to a stop sign causes the vehicles’ ML models to incorrectly classify and interpret the visual cues on the sign [1]. In an exercise to better understand common vulnerabilities in model deployment, we experimented with a handful of prevalent adversarial attacks to understand their viability and demonstrate their potential. Figure 1: A human can easily discern this as a stop sign with stickers…
Adversarial Attacks in Machine Learning: Invisible Threats
Part 1 of 3
Artificial intelligence (AI) and machine learning (ML) models have become ingrained in everyday life, with applications ranging from biometric verification to virtual assistants. Reliance on ML becomes problematic when models misclassify an observation, potentially leading to adverse consequences. Thus the performance, robustness, and security of these models is critical to their success. Modern deep learning models have surpassed human performance in some specific tasks (e.g. types of computer vision), though they can be highly vulnerable to adversarial attacks. Such attacks are often crafted using methods similar to those employed by the models that perform the intended classifications. For example, an…
Deep Learning for Language Identification on Audio Signals
Introduction Speech-to-text transcription is an important tool for mission analytics and many natural language processing tasks. In most cases, a language is manually preset before speech-to-text transcription can be applied. For scenarios where data is being aggregated in large quantities in various languages, automatic language detection capabilities would greatly streamline data processing. Novetta has developed a technique, Language Identification For Audio Spectrograms (LIFAS), that uses spectrograms of raw audio signals as input to a convolutional neural network (CNN) to be used for language identification. This approach requires minimal pre-processing, as raw audio signals are input into the neural network and…
