BERT’s release in November 2018 marked an inflection point for Natural Language Processing (NLP), moving us closer to human-level performance on a range of use cases. Since then, many transformer-based language models, such as XLM, GPT-2, XLNet, and roBERTa, have been released, with each subsequent model raising the performance ceiling. With the availability of these state-of-the-art pre-trained language models, we can produce NLP models that solve a variety of downstream language-based tasks, enabling our customers to quickly analyze large volumes of unstructured text. The challenge with using these models is that they each differ slightly in their application. For example,…
Introducing Metadata Enhanced ULMFiT
Introduction Since its release in 2018, Novetta’s Machine Learning Center of Excellence has evaluated the performance of Universal Language Model Fine-Tuning (ULMFiT)1 against many customer use cases. We have found that automated labeling of unstructured text helps analysts be more efficient, freeing them up to focus on higher-value tasks. We have been so impressed with its accuracy that we have deployed ULMFiT against a dataset in a production system, with plans to expand it to dozens of other datasets. While ULMFiT’s performance has been impressive on all the datasets we’ve tested, we thought it could do more. Our first use…
State of the Art Text Classification with ULMFiT
Novetta’s Matt Teschke spoke at ODSC East – Boston in the session,”State of the Art Text Classification with ULMFiT” on May 2, 2019. The rise of the internet has led to a faster flow of information, where news posted to a relatively obscure blog can be shared on social media and reach national publications within hours. The volume of information is such that humans alone cannot filter out noise, identify important new viewpoints, and determine how messaging trends are changing over time. At Novetta, we are constantly evaluating advances in deep learning to help our customers…
