update readme
Browse files
README.md
CHANGED
@@ -47,7 +47,7 @@ Advancing popular visual capabilites from MiniCPM-V series, MiniCPM-o 2.6 can pr
|
|
47 |
In addition to its friendly size, MiniCPM-o 2.6 also shows **state-of-the-art token density** (i.e., number of pixels encoded into each visual token). **It produces only 640 tokens when processing a 1.8M pixel image, which is 75% fewer than most models**. This directly improves the inference speed, first-token latency, memory usage, and power consumption. As a result, MiniCPM-o 2.6 can efficiently support **multimodal live streaming** on end-side devices such as iPad.
|
48 |
|
49 |
- 💫 **Easy Usage.**
|
50 |
-
MiniCPM-o 2.6 can be easily used in various ways: (1) [llama.cpp](https://github.com/OpenBMB/llama.cpp/blob/minicpm-omni/examples/llava/README-minicpmo2.6.md) support for efficient CPU inference on local devices, (2) [int4](https://huggingface.co/openbmb/MiniCPM-o-2_6-int4) and [GGUF](https://huggingface.co/openbmb/MiniCPM-o-2_6-gguf) format quantized models in 16 sizes, (3) [vLLM](#efficient-inference-with-llamacpp-ollama-vllm) support for high-throughput and memory-efficient inference, (4) fine-tuning on new domains and tasks with [LLaMA-Factory](./docs/llamafactory_train.md), (5) quick local WebUI demo setup with [Gradio](#chat-with-our-demo-on-gradio), and (6) online web demo on [
|
51 |
|
52 |
|
53 |
**Model Architecture.**
|
@@ -1357,7 +1357,7 @@ Please look at [GitHub](https://github.com/OpenBMB/MiniCPM-o) for more detail ab
|
|
1357 |
|
1358 |
|
1359 |
## Inference with llama.cpp<a id="llamacpp"></a>
|
1360 |
-
MiniCPM-o 2.6 can run with llama.cpp. See our fork of [llama.cpp](https://github.com/OpenBMB/llama.cpp/tree/minicpm-
|
1361 |
|
1362 |
|
1363 |
## Int4 quantized version
|
@@ -1392,4 +1392,4 @@ If you find our work helpful, please consider citing our papers 📝 and liking
|
|
1392 |
journal={arXiv preprint arXiv:2408.01800},
|
1393 |
year={2024}
|
1394 |
}
|
1395 |
-
```
|
|
|
47 |
In addition to its friendly size, MiniCPM-o 2.6 also shows **state-of-the-art token density** (i.e., number of pixels encoded into each visual token). **It produces only 640 tokens when processing a 1.8M pixel image, which is 75% fewer than most models**. This directly improves the inference speed, first-token latency, memory usage, and power consumption. As a result, MiniCPM-o 2.6 can efficiently support **multimodal live streaming** on end-side devices such as iPad.
|
48 |
|
49 |
- 💫 **Easy Usage.**
|
50 |
+
MiniCPM-o 2.6 can be easily used in various ways: (1) [llama.cpp](https://github.com/OpenBMB/llama.cpp/blob/minicpm-omni/examples/llava/README-minicpmo2.6.md) support for efficient CPU inference on local devices, (2) [int4](https://huggingface.co/openbmb/MiniCPM-o-2_6-int4) and [GGUF](https://huggingface.co/openbmb/MiniCPM-o-2_6-gguf) format quantized models in 16 sizes, (3) [vLLM](#efficient-inference-with-llamacpp-ollama-vllm) support for high-throughput and memory-efficient inference, (4) fine-tuning on new domains and tasks with [LLaMA-Factory](./docs/llamafactory_train.md), (5) quick local WebUI demo setup with [Gradio](#chat-with-our-demo-on-gradio), and (6) online web demo on [US](https://minicpm-omni-webdemo-us.modelbest.cn/) server.
|
51 |
|
52 |
|
53 |
**Model Architecture.**
|
|
|
1357 |
|
1358 |
|
1359 |
## Inference with llama.cpp<a id="llamacpp"></a>
|
1360 |
+
MiniCPM-o 2.6 (vision-only mode) can run with llama.cpp. See our fork of [llama.cpp](https://github.com/OpenBMB/llama.cpp/tree/minicpm-omni) and [readme](https://github.com/OpenBMB/llama.cpp/blob/minicpm-omni/examples/llava/README-minicpmo2.6.md) for more detail.
|
1361 |
|
1362 |
|
1363 |
## Int4 quantized version
|
|
|
1392 |
journal={arXiv preprint arXiv:2408.01800},
|
1393 |
year={2024}
|
1394 |
}
|
1395 |
+
```
|