nielsr HF staff commited on
Commit
7f04d6c
1 Parent(s): b46b9db

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +21 -20
README.md CHANGED
@@ -11,7 +11,8 @@ license: apache-2.0
11
  ---
12
  # Model card for DePlot
13
 
14
- ![pull_figure](https://s3.amazonaws.com/moonup/production/uploads/62441d1d9fdefb55a0b7d12c/u8rWTawSyUegF4jzwOpNO.png)
 
15
 
16
 
17
  # Table of Contents
@@ -30,7 +31,25 @@ The abstract of the paper states that:
30
 
31
  # Using the model
32
 
33
- ## Converting from T5x to huggingface
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
34
 
35
  You can use the [`convert_pix2struct_checkpoint_to_pytorch.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/pix2struct/convert_pix2struct_original_pytorch_to_hf.py) script as follows:
36
  ```bash
@@ -51,24 +70,6 @@ model.push_to_hub("USERNAME/MODEL_NAME")
51
  processor.push_to_hub("USERNAME/MODEL_NAME")
52
  ```
53
 
54
- ## Run a prediction
55
-
56
- You can run a prediction by querying an input image together with a question as follows:
57
- ```python
58
- from transformers import Pix2StructForConditionalGeneration, Pix2StructProcessor
59
- import requests
60
- from PIL import Image
61
-
62
- model = Pix2StructForConditionalGeneration.from_pretrained('google/deplot')
63
- processor = Pix2StructProcessor.from_pretrained('google/deplot')
64
- url = "https://raw.githubusercontent.com/vis-nlp/ChartQA/main/ChartQA%20Dataset/val/png/5090.png"
65
- image = Image.open(requests.get(url, stream=True).raw)
66
-
67
- inputs = processor(images=image, text="Generate underlying data table of the figure below:", return_tensors="pt")
68
- predictions = model.generate(**inputs, max_new_tokens=512)
69
- print(processor.decode(predictions[0], skip_special_tokens=True))
70
- ```
71
-
72
  # Contribution
73
 
74
  This model was originally contributed by Fangyu Liu, Julian Martin Eisenschlos et al. and added to the Hugging Face ecosystem by [Younes Belkada](https://huggingface.co/ybelkada).
 
11
  ---
12
  # Model card for DePlot
13
 
14
+ <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/deplot_architecture.png"
15
+ alt="drawing" width="600"/>
16
 
17
 
18
  # Table of Contents
 
31
 
32
  # Using the model
33
 
34
+ You can run a prediction by querying an input image together with a question as follows:
35
+
36
+ ```python
37
+ from transformers import Pix2StructProcessor, Pix2StructForConditionalGeneration
38
+ import requests
39
+ from PIL import Image
40
+
41
+ processor = Pix2StructProcessor.from_pretrained('google/deplot')
42
+ model = Pix2StructForConditionalGeneration.from_pretrained('google/deplot')
43
+
44
+ url = "https://raw.githubusercontent.com/vis-nlp/ChartQA/main/ChartQA%20Dataset/val/png/5090.png"
45
+ image = Image.open(requests.get(url, stream=True).raw)
46
+
47
+ inputs = processor(images=image, text="Generate underlying data table of the figure below:", return_tensors="pt")
48
+ predictions = model.generate(**inputs, max_new_tokens=512)
49
+ print(processor.decode(predictions[0], skip_special_tokens=True))
50
+ ```
51
+
52
+ # Converting from T5x to huggingface
53
 
54
  You can use the [`convert_pix2struct_checkpoint_to_pytorch.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/pix2struct/convert_pix2struct_original_pytorch_to_hf.py) script as follows:
55
  ```bash
 
70
  processor.push_to_hub("USERNAME/MODEL_NAME")
71
  ```
72
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
73
  # Contribution
74
 
75
  This model was originally contributed by Fangyu Liu, Julian Martin Eisenschlos et al. and added to the Hugging Face ecosystem by [Younes Belkada](https://huggingface.co/ybelkada).