Slomb/Ambig_Question

Model Details

Text classification model for ambiguity in questions. Classifies questions as ambiguous or clear. Based on distilbert/distilbert-base-uncased.

Example:

"Did he do it?" {'label': 'AMBIG', 'score': 0.9029870629310608}

"Did Peter win the game?" {'label': 'CLEAR', 'score': 0.8900136351585388}

Out-of-Scope Use

The model was only trained to classify single questions. Other kinds of data are not tested.

Training Data

I manually labeled a small part of the inquisitiveqg dataset mixed with a private dataset to train the model to recognize ambiguity in questions. A satisfactory model with 85.5% accuracy was created.

Metrics

"eval_accuracy": 0.8551401869158879,

"eval_loss": 0.3658725619316101,