Update README.md

README.md

@@ -1,58 +1,76 @@
 ---
+language:
+- id
 license: other
-base_model: "bahasa-4b"
-tags:
-- llama-factory
-- full
-- generated_from_trainer
-model-index:
-- name: v3
-  results: []
+license_name: tongyi-qianwen
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
+# Bahasa-4b Model Report
 
-# v3
+## Model Name
+**Bahasa-4b**
 
-This model is a fine-tuned version of [/home/acul/data_nvme5/bahasa-4b/saves/Qwen1.5-4B/full/train_2024-03-27-23-212-110/checkpoint-210000](https://huggingface.co//home/acul/data_nvme5/bahasa-4b/saves/Qwen1.5-4B/full/train_2024-03-27-23-212-110/checkpoint-210000) on the indo_instruct_3 dataset.
+## Model Detail
+Bahasa-4b is continued training from qwen-4b using 10 billion high quality text of Indonesian. The model outperforms some 4b, and even 7b models for Indonesian tasks.
 
-## Model description
+## Model Developers
+Bahasa AI
 
-More information needed
+## Intended Use
+This model is intended for various NLP tasks that require understanding and generating Indonesian language. It is suitable for applications such as question answering, sentiment analysis, document summarization, and more.
 
-## Intended uses & limitations
+## Training Data
+Bahasa-4b was trained on a 10 billion subset data of Indonesian dataset from a collected pool of 100 billion.
 
-More information needed
+## Benchmarks
+The following table shows the performance of Bahasa-4b compared to the models Sailor_4b and Mistral-7B-v0.1 across several benchmarks:
 
-## Training and evaluation data
+| Dataset        | Version | Metric | Mode | Sailor_4b | Bahasa-4b-hf | Mistral-7B-v0.1 |
+|----------------|---------|--------|------|-----------|--------------|-----------------|
+| tydiqa-id      | 0e9309  | EM     | gen  | 53.98     | 55.04        | 63.54           |
+| tydiqa-id      | 0e9309  | F1     | gen  | 73.48     | 75.39        | 78.73           |
+| xcopa-id       | 36c11c  | EM     | ppl  | 69.2      | 73.2         | 62.40           |
+| xcopa-id       | 36c11c  | F1     | ppl  | 69.2      | 73.2         | -               |
+| m3exam-id-ppl  | ede415  | EM     | ppl  | 31.27     | 44.47        | 26.68           |
+| belebele-id-ppl| 7fe030  | EM     | ppl  | 41.33     | 42.33        | 41.33           |
 
-More information needed
 
-## Training procedure
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+device = "cuda" # the device to load the model onto
 
-### Training hyperparameters
+model = AutoModelForCausalLM.from_pretrained(
+    "Bahasalab/Bahasa-4b-chat-v2",
+    torch_dtype="auto",
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained("Bahasalab/Bahasa-4b-chat")
 
-The following hyperparameters were used during training:
-- learning_rate: 5e-05
-- train_batch_size: 2
-- eval_batch_size: 8
-- seed: 42
-- distributed_type: multi-GPU
-- num_devices: 2
-- total_train_batch_size: 4
-- total_eval_batch_size: 16
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: cosine
-- num_epochs: 4.0
+messages = [
+    {"role": "system", "content": "Kamu adalah asisten yang membantu"},
+    {"role": "user", "content": "kamu siapa"}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
 
-### Training results
+model_inputs = tokenizer([text], return_tensors="pt").to(device)
 
+generated_ids = model.generate(
+    input_ids=model_inputs.input_ids,
+    attention_mask=model_inputs.attention_mask,
+    max_new_tokens=512,
+    eos_token_id=tokenizer.eos_token_id
 
+)
+generated_ids = [
+    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+]
 
-### Framework versions
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+print(response)
+```
 
-- Transformers 4.39.1
-- Pytorch 2.1.2+cu121
-- Datasets 2.18.0
-- Tokenizers 0.15.2
+This data demonstrates that Bahasa-4b consistently outperforms the Sailor_4b model in various Indonesian language tasks, showing improvements in both EM (Exact Match) and F1 scores across different datasets, and is competitive with the Mistral-7B-v0.1 model.