xtremedistil-l6-h384-go-emotion

This model is a fine-tuned version of microsoft/xtremedistil-l6-h384-uncased on the go_emotions dataset.

See notebook for how the model was trained and converted to ONNX format

This model is deployed to aiserv.cloud for live demo of the model.

See https://github.com/jobergum/browser-ml-inference for how to reproduce.

Training hyperparameters

• batch size 128
• learning_rate=3e-05
• epocs 4

  Num examples = 211225
  Num Epochs = 4
  Instantaneous batch size per device = 128
  Total train batch size (w. parallel, distributed & accumulation) = 128
  Gradient Accumulation steps = 1
  Total optimization steps = 6604
   [6604/6604 53:23, Epoch 4/4]
  Step	Training Loss
  500	0.263200
  1000	0.156900
  1500	0.152500
  2000	0.145400
  2500	0.140500
  3000	0.135900
  3500	0.132800
  4000	0.129400
  4500	0.127200
  5000	0.125700
  5500	0.124400
  6000	0.124100
  6500	0.123400