AI may be able to do a lot of things well, but condensing paragraphs into sentences is not the easiest job around for artificial intelligence systems. But all that may be about to change.
The challenge arises from requiring a semantic understanding of the text.
And this is something that is beyond the capabilities of most off-the-shelf natural language processing models that are in use in the wild.
However, as researchers at Microsoft recently demonstrated, it is possible to develop an AI framework that can reason about relationships in a weakly structured text. This enables it to outperform conventional NLP models on a range of text summarization tasks.
A paper published by scientists at Microsoft Research in Cambridge, England outlines this.
Called “Structured Neural Summarization”, this technology was trained on articles from CNN and the Daily Mail, along with sentences that summarized each article. It was able to generate highly readable summaries successfully.
The researchers wrote:
“Summarization, the task of condensing a large and complex input into a smaller representation that retains the core semantics of the input, is a classical task for natural language processing systems. Automatic summarization requires a machine learning component to identify important entities and relationships between them, while ignoring redundancies and common concepts. However, while standard models theoretically have the ability to handle arbitrary long-distance relationships, in practice they often fail to correctly handle long texts and are easily distracted by simple noise.”
Their solution consists of two modules.
The first step is the creation of an extended sequence encoder, which is simply an AI model that processes an input sequence and predicts the next characters of the target sequence, keeping in view the previous characters of the target sequence.
And then there is a neural network that directly learns from graph representations of annotated natural language.
The researchers are excited about their initial progress and now seek to deeper integration of mixed sequence graph modeling in a wide range of tasks across both formal and natural languages.