Pretrained Natural Language Processing Model for Intent Recognition (BERT-IR)
自然语言处理中的意图识别预训练模型(BERT-IR)
作者: Vasima Khan, Tariq Azfar Meenai
Abstract

Intent Recognition (IR) is considered a key area in Natural Language Processing (NLP). It has crucial usage in various applications. One is the Search Engine-Interpreting the context of text searched by the user improves the response time and helps the search engines give appropriate outputs. Another can be Social Media Analytics-Analysing profiles of users on different social media platforms has become a necessity in today’s applications like recommendation systems in the online world, digital marketing, and a lot more. Many researchers are using different techniques for achieving intent recognition but getting high accuracy in intent recognition is crucial. In this work, named BERT-IR, a pre-trained Natural Language Processing model called as BERT model, along with few add-ons, is applied for the task of Intent Recognition. We have achieved an accuracy of 97.67% on a widely used dataset which shows the capability and efficiency of our work. For comparison purposes, we have applied primarily used Machine Learning techniques, namely Naive Bayes, Logistic Regression, Decision Tree, Random Forest, and Gradient Boost as well as Deep Learning Techniques used for intent recognition like Recurrent Neural Network, Long Short Term Memory Network, and Bidirectional Long Short Term Memory Network on the same dataset and evaluated the accuracy. It is found out that BERT-IR’s accuracy is far better than that of the other models implemented.


Keywords: Intent recognition; intent detection; natural language processing; BERT; deep learning; deep neural network


摘要

意图识别(IR)被视为自然语言处理(NLP)中的关键领域,其在各种应用中均具有重要作用。其一是搜索引擎 - 解释用户搜索文本的上下文语境。这不仅缩短了响应时间,还有助于搜索引擎产出相应的输出结果。另一应用是社交媒体分析 — 分析不同社交媒体平台上的用户资料已成为当今网络世界的推荐系统、数字营销等多方面应用的必要条件。目前,研究人员通过使用各种技术来实现意图识别,但需注意的是,在意图识别中获得高精度至关重要。本研究中,BERT-IR模型(自然语言处理预训练模型—BERT与一些附加组件)被用于意图识别任务。现如今,在广泛使用的数据集上,BERT-IR模型的准确率已达到97.67%。这一数字表明该模型的高性能与高效率。出于比较目的,本研究还在同一数据集上应用了机器学习技术,即朴素贝叶斯、逻辑回归、决策树、随机森林和梯度提升,以及用于意图识别的深度学习技术,如递归神经网络、长-短期记忆网络与双向长-短期记忆网络,并对其准确性进行评估。研究结果表明,BERT-IR模型的精准度远优于其他模型。


关键词:意图识别;意图检测;自然语言处理;BERT;深度学习;深度神经网络

 


HCIS-1-3_4-66-g001.png

论文信息 PAPER INFORMATION
所属期刊
Human-Centric Intelligent Systems
ISSN(Online)
2667-1336
学科领域
计算机科学
发表时间
2020-11-20