Model save

README.md

@@ -1,12 +1,7 @@
 ---
 base_model: slm-research-vn/Qwen2-7B-Instruct-SPPO-Function-call-v2.1
-datasets:
-- slm-research-vn/dpo-format-function-calling-v2
-- slm-research-vn/dpo-format-glaive-code-assistant-v3-with-mistral-large-slm-iter4
-- argilla/dpo-mix-7k
 library_name: peft
 tags:
-- alignment-handbook
 - trl
 - dpo
 - generated_from_trainer
@@ -20,17 +15,17 @@ should probably proofread and complete it, then remove this comment. -->
 
 # Qwen2-7B-Instruct-SPPO-Function-call-v2.4
 
-This model is a fine-tuned version of [slm-research-vn/Qwen2-7B-Instruct-SPPO-Function-call-v2.1](https://huggingface.co/slm-research-vn/Qwen2-7B-Instruct-SPPO-Function-call-v2.1) on the slm-research-vn/dpo-format-function-calling-v2, the slm-research-vn/dpo-format-glaive-code-assistant-v3-with-mistral-large-slm-iter4 and the argilla/dpo-mix-7k datasets.
+This model is a fine-tuned version of [slm-research-vn/Qwen2-7B-Instruct-SPPO-Function-call-v2.1](https://huggingface.co/slm-research-vn/Qwen2-7B-Instruct-SPPO-Function-call-v2.1) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.4345
-- Rewards/chosen: 1.3033
-- Rewards/rejected: 0.2776
-- Rewards/accuracies: 0.8185
-- Rewards/margins: 1.0258
-- Logps/rejected: -333.5228
-- Logps/chosen: -261.0424
-- Logits/rejected: -0.7224
-- Logits/chosen: -0.7089
+- Loss: 0.3144
+- Rewards/chosen: 1.9968
+- Rewards/rejected: 0.2178
+- Rewards/accuracies: 0.8844
+- Rewards/margins: 1.7789
+- Logps/rejected: -267.1377
+- Logps/chosen: -202.5171
+- Logits/rejected: -0.6136
+- Logits/chosen: -0.6052
 
 ## Model description
 
@@ -49,7 +44,7 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 5e-07
+- learning_rate: 1e-06
 - train_batch_size: 1
 - eval_batch_size: 1
 - seed: 42
@@ -67,13 +62,15 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch  | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
 |:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
-| 0.6782        | 0.1270 | 100  | 0.6611          | 0.1038         | 0.0272           | 0.8000             | 0.0766          | -338.5302      | -285.0340    | -0.7425         | -0.7284       |
-| 0.5811        | 0.2540 | 200  | 0.5409          | 0.5575         | 0.1395           | 0.8370             | 0.4180          | -336.2845      | -275.9589    | -0.7306         | -0.6945       |
-| 0.5484        | 0.3811 | 300  | 0.4777          | 0.9393         | 0.2286           | 0.8000             | 0.7107          | -334.5019      | -268.3231    | -0.7283         | -0.7031       |
-| 0.4531        | 0.5081 | 400  | 0.4535          | 1.1283         | 0.2592           | 0.8296             | 0.8690          | -333.8891      | -264.5439    | -0.7170         | -0.6879       |
-| 0.4577        | 0.6351 | 500  | 0.4415          | 1.2504         | 0.2849           | 0.8148             | 0.9655          | -333.3753      | -262.1006    | -0.7146         | -0.6865       |
-| 0.4715        | 0.7621 | 600  | 0.4364          | 1.2963         | 0.2864           | 0.8148             | 1.0099          | -333.3469      | -261.1842    | -0.7175         | -0.6913       |
-| 0.4508        | 0.8892 | 700  | 0.4348          | 1.2990         | 0.2819           | 0.8222             | 1.0172          | -333.4369      | -261.1283    | -0.7185         | -0.6937       |
+| 0.6534        | 0.1020 | 100  | 0.6139          | 0.2871         | 0.0961           | 0.7572             | 0.1911          | -269.5727      | -236.7095    | -0.6762         | -0.6844       |
+| 0.4902        | 0.2041 | 200  | 0.4530          | 1.4421         | 0.5735           | 0.8064             | 0.8685          | -260.0234      | -213.6108    | -0.6513         | -0.6502       |
+| 0.391         | 0.3061 | 300  | 0.3935          | 1.9109         | 0.6931           | 0.8382             | 1.2178          | -257.6317      | -204.2344    | -0.6321         | -0.6298       |
+| 0.3497        | 0.4082 | 400  | 0.3633          | 1.9715         | 0.5740           | 0.8468             | 1.3975          | -260.0141      | -203.0221    | -0.6323         | -0.6313       |
+| 0.3378        | 0.5102 | 500  | 0.3421          | 2.0346         | 0.4602           | 0.8699             | 1.5744          | -262.2907      | -201.7610    | -0.6197         | -0.6103       |
+| 0.2904        | 0.6122 | 600  | 0.3287          | 1.9449         | 0.3083           | 0.8757             | 1.6366          | -265.3278      | -203.5543    | -0.6221         | -0.6159       |
+| 0.3053        | 0.7143 | 700  | 0.3207          | 1.9933         | 0.2606           | 0.8902             | 1.7327          | -266.2818      | -202.5857    | -0.6162         | -0.6111       |
+| 0.2655        | 0.8163 | 800  | 0.3158          | 1.9845         | 0.2262           | 0.8815             | 1.7583          | -266.9698      | -202.7614    | -0.6127         | -0.6026       |
+| 0.2943        | 0.9184 | 900  | 0.3144          | 1.9968         | 0.2178           | 0.8844             | 1.7789          | -267.1377      | -202.5171    | -0.6136         | -0.6052       |
 
 
 ### Framework versions

all_results.json

@@ -1,22 +1,9 @@
 {
-    "epoch": 0.9996824388694824,
-    "eval_logits/chosen": -0.7089246511459351,
-    "eval_logits/rejected": -0.7224333882331848,
-    "eval_logps/chosen": -261.0423889160156,
-    "eval_logps/rejected": -333.52276611328125,
-    "eval_loss": 0.4344652593135834,
-    "eval_rewards/accuracies": 0.8185185194015503,
-    "eval_rewards/chosen": 1.3033398389816284,
-    "eval_rewards/margins": 1.0257779359817505,
-    "eval_rewards/rejected": 0.27756187319755554,
-    "eval_runtime": 185.0205,
-    "eval_samples": 2155,
-    "eval_samples_per_second": 11.647,
-    "eval_steps_per_second": 1.459,
+    "epoch": 1.0,
     "total_flos": 0.0,
-    "train_loss": 0.5202696668753327,
-    "train_runtime": 5902.1361,
-    "train_samples": 25187,
-    "train_samples_per_second": 4.267,
-    "train_steps_per_second": 0.133
+    "train_loss": 0.39260031933687173,
+    "train_runtime": 7916.79,
+    "train_samples": 31353,
+    "train_samples_per_second": 3.96,
+    "train_steps_per_second": 0.124
 }

train_results.json

@@ -1,9 +1,9 @@
 {
-    "epoch": 0.9996824388694824,
+    "epoch": 1.0,
     "total_flos": 0.0,
-    "train_loss": 0.5202696668753327,
-    "train_runtime": 5902.1361,
-    "train_samples": 25187,
-    "train_samples_per_second": 4.267,
-    "train_steps_per_second": 0.133
+    "train_loss": 0.39260031933687173,
+    "train_runtime": 7916.79,
+    "train_samples": 31353,
+    "train_samples_per_second": 3.96,
+    "train_steps_per_second": 0.124
 }