Thread by @philipvollet

To build a chatbot you need data for your intent classification.

But what if you have too little training data?

Paraphrasing is one option for augmentation

But what is a good paraphrase?

Almost all conditioned text generation models are validated on 2 factors:

1. If the generated text conveys the same meaning as the original context (Adequacy)

2. If the text is fluent / grammatically correct english (Fluency)

For instance Neural Machine Translation outputs are tested for Adequacy and Fluency.

But a good paraphrase should be adequate and fluent while being as different as possible on the surface lexical form. With respect to this definition, the 3 key metrics that measures the quality of paraphrases are:

1. Adequacy: Is the meaning preserved adequately?
2. Fluency: Is the paraphrase fluent English?
3. Diversity: Lexical / Phrasal / Syntactical → how much has the paraphrase changed the original sentence?

What makes a paraphraser a good augmentor?

For training a NLU model we just don't need a lot of utterances but utterances with intents and slots/entities annotated.

Typical flow would be:

Given an input utterance + input annotations a good augmentor spits out N output paraphrases while preserving the intent and slots.

The output paraphrases are then converted into annotated data using the input annotations that we got in step 1.

The annotated data created out of the output paraphrases then makes the training dataset for your NLU model.

But in general being a generative model paraphrasers doesn't guarantee to preserve slots/entities

So the ability to generate high quality paraphrases in a constrained fashion without trading off the intents and slots for lexical dissimilarity makes a paraphraser a good augmentor

Parrot

A practical and feature-rich paraphrasing framework to augment human intents in text form to build robust NLU models for conversational engines.

github.com/PrithivirajDam…