nayohan
/

polyglot-ko-12.8b-Inst-All

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

This model is a instruct-tuned poylglot-ko-5.8b model, using full [Kullm, OIG, KoAlpaca] Instruction dataset. koquality_raw.json -> 96step

Training hyperparameters

learning_rate: 5e-5
seed: 42
distributed_type: multi-GPU (A100 80G)
num_devices: 6
train_batch_size: 4
gradient_accumulation_steps: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 5.0

Framework versions

Transformers 4.35.0
Pytorch 2.0.1+cu118
Datasets 2.14.6
deepspeed 0.11.1
accelerate 0.24.1

Downloads last month: 2,289

Safetensors

Model size

12.9B params

Tensor type

F32

·

Inference Providers NEW

Text Generation

This model is not currently available via any of the supported third-party Inference Providers, and the model is not deployed on the HF Inference API.

Model tree for nayohan/polyglot-ko-12.8b-Inst-All

Base model

EleutherAI/polyglot-ko-12.8b

Finetuned

(11)

this model

Dataset used to train nayohan/polyglot-ko-12.8b-Inst-All