See axolotl config

axolotl version: 0.4.1

base_model: Qwen/Qwen2.5-0.5B
bf16: auto
dataset_prepared_path: /training/data/prepared
datasets:
- conversation: llama3
  path: RLHFlow/Mistral-PRM-Data
  split: train
  train_on_split: train
  type: sharegpt
flash_attention: true
fp16: false
gradient_accumulation_steps: 4
gradient_checkpointing: true
hub_model_id: rawsh/MetaMath-Qwen2.5-0.5b-PRM
hub_strategy: every_save
learning_rate: 2.0e-06
load_in_4bit: false
load_in_8bit: false
logging_steps: 2
lr_scheduler: cosine
max_grad_norm: 1.0
micro_batch_size: 1
model_type: AutoModelForCausalLM
num_epochs: 1
optimizer: paged_adamw_32bit
output_dir: /training/prm
pad_to_sequence_len: true
push_to_hub: true
sample_packing: true
save_safetensors: true
save_strategy: epoch
save_total_limit: 4
sequence_len: 8192
special_tokens:
  pad_token: <|endoftext|>
strict: false
tf32: true
tokenizer_type: AutoTokenizer
train_on_inputs: false
trust_remote_code: true
val_set_size: 0.0
wandb_name: qwen2.5-0.5b-bs32_lr2e-6_prm
wandb_project: preference-models
warmup_ratio: 0.05
weight_decay: 0.0

MetaMath-Qwen2.5-0.5b-PRM

This model is a fine-tuned version of Qwen/Qwen2.5-0.5B on the None dataset.

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-06
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 4
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 214
• num_epochs: 1

Training results

Framework versions

• Transformers 4.42.3
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1