In newspaper articles, emails, and other texts, it is natural to reference dates in relation to the publication date. For example, you will commonly see phrases like “yesterday” and “two weeks ago.” However, if you read an article a year after it was published, these timeframes are not immediately clear. When analyzing articles using natural language processing (NLP) techniques, relative dates referenced in an article will not be interpreted properly, if at all, since there is no context for the models. Thus, in order to effectively use tools such as event extraction and question-answering engines, relative references need to be…
State-of-the-art NLP made easy with AdaptNLP
Natural Language Processing (NLP) has advanced significantly since 2018, when ULMFiT and Google’s release of the BERT language model approached human-level performance on a range of use cases. Since then, several models, such as XLM, GPT-2, XLNet, and ALBERT, have been released in quick succession, each improving on its predecessors. While these state-of-the-art models can solve human-level, language-based tasks on large volumes of unstructured text for certain use cases, getting a handle on what to use, when to use it, and how to use it can be a challenge. At Novetta, we explored what it would take to streamline the…
NovettaNLP
A Powerful, Transformers-Based Framework for End-to-End Natural Language Processing Tasks
BERT’s release in November 2018 marked an inflection point for Natural Language Processing (NLP), moving us closer to human-level performance on a range of use cases. Since then, many transformer-based language models, such as XLM, GPT-2, XLNet, and roBERTa, have been released, with each subsequent model raising the performance ceiling. With the availability of these state-of-the-art pre-trained language models, we can produce NLP models that solve a variety of downstream language-based tasks, enabling our customers to quickly analyze large volumes of unstructured text. The challenge with using these models is that they each differ slightly in their application. For example,…
Introducing Metadata Enhanced ULMFiT
Introduction Since its release in 2018, Novetta’s Machine Learning Center of Excellence has evaluated the performance of Universal Language Model Fine-Tuning (ULMFiT)1 against many customer use cases. We have found that automated labeling of unstructured text helps analysts be more efficient, freeing them up to focus on higher-value tasks. We have been so impressed with its accuracy that we have deployed ULMFiT against a dataset in a production system, with plans to expand it to dozens of other datasets. While ULMFiT’s performance has been impressive on all the datasets we’ve tested, we thought it could do more. Our first use…
State of the Art Text Classification with ULMFiT
Novetta’s Matt Teschke spoke at ODSC East – Boston in the session,”State of the Art Text Classification with ULMFiT” on May 2, 2019. The rise of the internet has led to a faster flow of information, where news posted to a relatively obscure blog can be shared on social media and reach national publications within hours. The volume of information is such that humans alone cannot filter out noise, identify important new viewpoints, and determine how messaging trends are changing over time. At Novetta, we are constantly evaluating advances in deep learning to help our customers…
