rhdang/Yelp_Review

Model Description

This model predicts the star rating (1 - 5) of a Yelp review based on its text content. It was trained using GPT-2 and BERT, with BERT achieving the best performance at 75% validation accuracy. The model addresses class imbalance using weighted loss and optimizes hyperparameters to enhance generalization.

Training Details

• Dataset: Yelp Reviews dataset (100,000 samples used)

• Preprocessing:

  • GPT-2 Tokenizer with Byte-Pair Encoding (BPE) for rare words
  • Truncation (128 tokens) and padding for uniform input size

• Models Trained:

  • GPT-2: Fine-tuned with a custom classification head, achieving 67% validation accuracy

  • BERT: Fine-tuned with bidirectional attention, achieving 75% validation accuracy

• Loss Function: Weighted Cross-Entropy Loss to counteract class imbalance

Limitations

• Performance may degrade on highly informal or extremely short reviews

• Class imbalance still affects predictions for underrepresented ratings

• Model was trained on English-language reviews only