Update README.md
Browse files
README.md
CHANGED
@@ -11,7 +11,8 @@ license: apache-2.0
|
|
11 |
---
|
12 |
# Model card for DePlot
|
13 |
|
14 |
-
|
|
|
15 |
|
16 |
|
17 |
# Table of Contents
|
@@ -30,7 +31,25 @@ The abstract of the paper states that:
|
|
30 |
|
31 |
# Using the model
|
32 |
|
33 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
34 |
|
35 |
You can use the [`convert_pix2struct_checkpoint_to_pytorch.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/pix2struct/convert_pix2struct_original_pytorch_to_hf.py) script as follows:
|
36 |
```bash
|
@@ -51,24 +70,6 @@ model.push_to_hub("USERNAME/MODEL_NAME")
|
|
51 |
processor.push_to_hub("USERNAME/MODEL_NAME")
|
52 |
```
|
53 |
|
54 |
-
## Run a prediction
|
55 |
-
|
56 |
-
You can run a prediction by querying an input image together with a question as follows:
|
57 |
-
```python
|
58 |
-
from transformers import Pix2StructForConditionalGeneration, Pix2StructProcessor
|
59 |
-
import requests
|
60 |
-
from PIL import Image
|
61 |
-
|
62 |
-
model = Pix2StructForConditionalGeneration.from_pretrained('google/deplot')
|
63 |
-
processor = Pix2StructProcessor.from_pretrained('google/deplot')
|
64 |
-
url = "https://raw.githubusercontent.com/vis-nlp/ChartQA/main/ChartQA%20Dataset/val/png/5090.png"
|
65 |
-
image = Image.open(requests.get(url, stream=True).raw)
|
66 |
-
|
67 |
-
inputs = processor(images=image, text="Generate underlying data table of the figure below:", return_tensors="pt")
|
68 |
-
predictions = model.generate(**inputs, max_new_tokens=512)
|
69 |
-
print(processor.decode(predictions[0], skip_special_tokens=True))
|
70 |
-
```
|
71 |
-
|
72 |
# Contribution
|
73 |
|
74 |
This model was originally contributed by Fangyu Liu, Julian Martin Eisenschlos et al. and added to the Hugging Face ecosystem by [Younes Belkada](https://huggingface.co/ybelkada).
|
|
|
11 |
---
|
12 |
# Model card for DePlot
|
13 |
|
14 |
+
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/deplot_architecture.png"
|
15 |
+
alt="drawing" width="600"/>
|
16 |
|
17 |
|
18 |
# Table of Contents
|
|
|
31 |
|
32 |
# Using the model
|
33 |
|
34 |
+
You can run a prediction by querying an input image together with a question as follows:
|
35 |
+
|
36 |
+
```python
|
37 |
+
from transformers import Pix2StructProcessor, Pix2StructForConditionalGeneration
|
38 |
+
import requests
|
39 |
+
from PIL import Image
|
40 |
+
|
41 |
+
processor = Pix2StructProcessor.from_pretrained('google/deplot')
|
42 |
+
model = Pix2StructForConditionalGeneration.from_pretrained('google/deplot')
|
43 |
+
|
44 |
+
url = "https://raw.githubusercontent.com/vis-nlp/ChartQA/main/ChartQA%20Dataset/val/png/5090.png"
|
45 |
+
image = Image.open(requests.get(url, stream=True).raw)
|
46 |
+
|
47 |
+
inputs = processor(images=image, text="Generate underlying data table of the figure below:", return_tensors="pt")
|
48 |
+
predictions = model.generate(**inputs, max_new_tokens=512)
|
49 |
+
print(processor.decode(predictions[0], skip_special_tokens=True))
|
50 |
+
```
|
51 |
+
|
52 |
+
# Converting from T5x to huggingface
|
53 |
|
54 |
You can use the [`convert_pix2struct_checkpoint_to_pytorch.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/pix2struct/convert_pix2struct_original_pytorch_to_hf.py) script as follows:
|
55 |
```bash
|
|
|
70 |
processor.push_to_hub("USERNAME/MODEL_NAME")
|
71 |
```
|
72 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
73 |
# Contribution
|
74 |
|
75 |
This model was originally contributed by Fangyu Liu, Julian Martin Eisenschlos et al. and added to the Hugging Face ecosystem by [Younes Belkada](https://huggingface.co/ybelkada).
|