Upload folder using huggingface_hub

README.md

@@ -1,3 +1,43 @@
 ---
 license: mit
 ---
+
+This is the fine-tuned MiniCPM-V-2_6 adapter for paper [**Automatic Evaluation for Text-to-Image Generation: Fine-grained Framework,
+Distilled Evaluation Model and Meta-Evaluation Benchmark**]()
+
+## Performance
+
+| Methods                              | Manual-1 ($\rho$) | Manual-1 ($\tau$) | Manual-2 ($\rho$) | Manual-2 ($\tau$) | Manual-3 ($\rho$) | Manual-3 ($\tau$) | Manual-Avg. ($\rho$) | Manual-Avg. ($\tau$) |
+| ------------------------------------ | ----------------- | ----------------- | ----------------- | ----------------- | ----------------- | ----------------- | -------------------- | -------------------- |
+| Average Score of Human Annotators    |
+| Manual-Avg.                          | 0.9511            | 0.8807            | 0.9452            | 0.8686            | 0.9513            | 0.8793            | -                    | -                    |
+| Traditional Methods                  |
+| FID                                  | -0.1183           | -0.0871           | -0.1000           | -0.0724           | -0.0897           | -0.0685           | -0.1231              | -0.0862              |
+| LPIPS                                | -0.1206           | -0.0898           | -0.0882           | -0.0644           | -0.1025           | -0.0732           | -0.1244              | -0.0856              |
+| DreamSim                             | -0.1284           | -0.0953           | -0.1230           | -0.0897           | -0.1308           | -0.0973           | -0.1382              | -0.0968              |
+| CLIPScore                            | 0.1532            | 0.1078            | 0.1725            | 0.1210            | 0.1227            | 0.0855            | 0.1505               | 0.1016               |
+| BLIPv2Score                          | 0.2278            | 0.1588            | 0.2280            | 0.1617            | 0.2134            | 0.1477            | 0.2152               | 0.1423               |
+| ImageReward                          | 0.4171            | 0.3065            | 0.3712            | 0.2690            | 0.4134            | 0.3030            | 0.4046               | 0.2839               |
+| LLM/MLLM-Based Methods               |
+| LLMScore$_{GPT-4}$                   | 0.3009            | 0.2212            | 0.2697            | 0.2012            | 0.3299            | 0.2497            | 0.3096               | 0.2228               |
+| DSG$_{Dependent}$                    | 0.4742            | 0.3790            | 0.4204            | 0.3339            | 0.4562            | 0.3652            | 0.4582               | 0.3512               |
+| DSG$_{Independent}$                  | 0.4815            | 0.3891            | 0.4382            | 0.3502            | 0.4721            | 0.3827            | 0.4704               | 0.3655               |
+| VQAScore$_{CLIP-FlanT5}$             | 0.4984            | 0.3768            | 0.4864            | 0.3619            | 0.5118            | 0.3854            | 0.5116               | 0.3712               |
+| VIEScore$_{MiniCPM-V-2.6}$           | 0.2834            | 0.2251            | 0.2814            | 0.2231            | 0.3016            | 0.2422            | 0.2941               | 0.2250               |
+| VIEScore$_{MiniCPM-V-2.6}$           | 0.4906            | 0.3878            | 0.4869            | 0.3836            | 0.4889            | 0.3899            | 0.5101               | 0.3897               |
+| VIEScore$_{GPT-4o}$                  | **0.5522**        | **0.4283**        | 0.5306            | 0.4101            | 0.5170            | 0.4024            | 0.5545               | 0.4170               |
+| Closed-Source MLLM with Our Pipeline |
+| Ours$_{GPT-4o}$                      | 0.5437            | 0.4302            | 0.5355            | 0.4214            | 0.5138            | 0.4061            | 0.5566               | 0.4285               |
+| Our Fine-tuned Open-source MLLM      |
+| Ours$_{MiniCPM-V-2.6}$               | 0.5306            | 0.4214            | **0.6067**        | **0.4769**        | **0.5744**        | **0.4563**        | **0.5938**           | **0.4566**           |
+
+
+## Quick Start
+* Merge the adapter with original MiniCPM-V-2_6 checkpoint with [SWIFT](https://github.com/modelscope/ms-swift):
+```shell
+swift export \
+    --model_type minicpm-v-v2_6-chat \
+    --model_id_or_path openbmb/MiniCPM-V-2_6 \
+    --ckpt_dir /path/to/the/downloaded/adapter \
+    --merge_lora true
+```

adapter_config.json

@@ -0,0 +1,26 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/disk/maziao/model-zoo/hf-repo/models--openbmb--MiniCPM-V-2_6",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [],
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": "^(llm|resampler)(?!.*(lm_head|output|emb|wte|shared)).*",
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46b8ddad79ad9f3cfbdfd95ff726d23c416d6be8c587c87e971e33b22b4aa850
+size 649025984

additional_config.json

@@ -0,0 +1 @@
+{"lora_dtype": null, "lorap_lr_ratio": null, "lorap_emb_lr": 1e-06}

generation_config.json

@@ -0,0 +1,10 @@
+{
+  "do_sample": true,
+  "eos_token_id": 151645,
+  "max_new_tokens": 2048,
+  "pad_token_id": 151643,
+  "temperature": 0.3,
+  "top_k": 20,
+  "top_p": 0.7,
+  "transformers_version": "4.37.2"
+}