Comments (1)
您好,“Textmonkey网页版运行终端不报错,但是上传图片问题后不回答,一直在加载”这个可能是由于您的网络问题。
您也可以使用下面的推理代码:
from monkey_model.modeling_textmonkey import TextMonkeyLMHeadModel
from monkey_model.tokenization_qwen import QWenTokenizer
from monkey_model.configuration_monkey import MonkeyConfig
if __name__ == "__main__":
checkpoint_path = ""
input_image = ""
input_str = "OCR with grounding:"
device_map = "cuda"
# Create model
config = MonkeyConfig.from_pretrained(
checkpoint_path,
trust_remote_code=True,
)
model = TextMonkeyLMHeadModel.from_pretrained(checkpoint_path,
config=config,
device_map=device_map, trust_remote_code=True).eval()
tokenizer = QWenTokenizer.from_pretrained(checkpoint_path,
trust_remote_code=True)
tokenizer.padding_side = 'left'
tokenizer.pad_token_id = tokenizer.eod_id
tokenizer.IMG_TOKEN_SPAN = config.visual["n_queries"]
input_str = f"<img>{input_image}</img> {input_str}"
input_ids = tokenizer(input_str, return_tensors='pt', padding='longest')
attention_mask = input_ids.attention_mask
input_ids = input_ids.input_ids
pred = model.generate(
input_ids=input_ids.cuda(),
attention_mask=attention_mask.cuda(),
do_sample=False,
num_beams=1,
max_new_tokens=2048,
min_new_tokens=1,
length_penalty=1,
num_return_sequences=1,
output_hidden_states=True,
use_cache=True,
pad_token_id=tokenizer.eod_id,
eos_token_id=tokenizer.eod_id,
)
response = tokenizer.decode(pred[0][input_ids.size(1):].cpu(), skip_special_tokens=False).strip()
print(response)
```
from monkey.
Related Issues (20)
- Pretrained weight for text monkey HOT 3
- textMonkey data release HOT 3
- TextMonkey问题 HOT 1
- A100 40G可以跑通训练吗?全参数SFT和LoRA我在A100 40G报OOM,我debug看到是self.visual.encode(images)就报OOM了 HOT 13
- Data Access HOT 2
- TextMonkey RuntimeError HOT 8
- 为什么文档理解的输入不是pdf或者doc文档,而是图片? HOT 1
- textmonkey支持多图输入吗 HOT 1
- Will Rico data be released? HOT 4
- How to finetune only one subnetwork using Deepspeed + Transformers
- How to finetune certain params via from HF's transformers, a
- vizwiz的准确率仅有37.62?表中的结果为61.2?QwenVL是35.2,请问是数据填写错误吗? HOT 8
- Get the embeddings of the image. HOT 1
- How to set gpu card for the demo project running HOT 5
- demo doesn't give OCR with grounding HOT 3
- 模型加载问题 HOT 8
- What is the various task-specific augmentations to different dataset mentioned in TextMonkey Sec3.5 ? HOT 2
- eval/eval_doc.sh中运行是的哪个py脚本呢? HOT 2
- demo_textmonkey.py 模型加载问题 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from monkey.