Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add device argument to eval_model function #27

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

feat: add device argument to eval_model function #27

wants to merge 3 commits into from

Conversation

ElvisClaros
Copy link

@ElvisClaros ElvisClaros commented Sep 12, 2024 •
edited
Loading

This commit introduces a new argument --device to the eval_model function, allowing users to specify the device to be used (e.g., 'cpu' or 'cuda'). The code has been updated to handle the device argument appropriately, ensuring compatibility with both CPU and GPU.
12_Sep_2024_18:42:41_grim

@ElvisClaros
feat: add device argument to eval_model function
39e7eec 
This commit introduces a new argument `--device` to the `eval_model`
function, allowing users to specify the device to be used (e.g., 'cpu'
or 'cuda'). The code has been updated to handle the device argument
appropriately, ensuring compatibility with both CPU and GPU.
@ElvisClaros ElvisClaros mentioned this pull request Sep 12, 2024
Open
@Ucas-HaoranWei
Copy link
Owner

Can you create a repo？ I will cite your job on the community contribution.

@ElvisClaros
Copy link
Author

Hello, sorry, I wouldn't be understanding what you mean by Can you create a repository? I already made a fork.
Do you want me to rename the fork I made? Or do you prefer that the pull requests be done from the main branch?
Greetings, I will be attentive to your response.

@Ucas-HaoranWei
Copy link
Owner

Sorry, I miss your fork repo. Thank u very much for your contribution.

@iLoveBug
Copy link

I got some problem while I install flash-attn because of lacking CUDA support.
pip install flash-attn --no-build-isolation

Did you install this package successfully? Or is it mandatory for CPU version?

@ElvisClaros
Copy link
Author

Hello @iLoveBug I did not install flash-attn.
From what I saw in their repository it is only compatible with NVIDIA GPUs.
IMAGE

@Zetaphor
Copy link

@Ucas-HaoranWei will you consider merging this feature? It would be great to be able to keep up with the main repo without having to manually patch this in

@ayttop
Copy link

ayttop commented Sep 25, 2024

!python3 /content/GOT-OCR2.0/GOT-OCR-2.0-master/GOT/demo/run_ocr_2.0.py --model-name ./content/GOT_weights --image-file /content/ANACO-MOTOS-26-3-10AM.jpg --type ocr --device cpu

2024-09-25 02:02:21.949838: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-09-25 02:02:21.982282: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-09-25 02:02:21.992093: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-09-25 02:02:23.961465: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
usage: run_ocr_2.0.py [-h] [--model-name MODEL_NAME] --image-file IMAGE_FILE --type TYPE
[--box BOX] [--color COLOR] [--render]
run_ocr_2.0.py: error: unrecognized arguments: --device cpu

@ayttop
Copy link

ayttop commented Sep 25, 2024 •
edited
Loading

it run

!python3 /content/GOT-OCR2.0/GOT-OCR-2.0-master/GOT/demo/run_ocr_2.0.py --model-name /content/GOT-OCR2.0/GOT-OCR-2.0-master/GOT/model/GOT-OCR2_0 --image-file /content/ANACO-MOTOS-26-3-10AM.jpg --type ocr --device cpu

2024-09-25 02:35:41.540578: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-09-25 02:35:41.574488: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-09-25 02:35:41.584910: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-09-25 02:35:43.233724: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
<|im_start|>system
You should follow the instructions carefully and explain your answers in detail.<|im_end|><|im_start|>user

@ayttop
Copy link

ayttop commented Sep 25, 2024

!python3 /content/GOT-OCR2.0/GOT-OCR-2.0-master/GOT/demo/run_ocr_2.0.py --model-name /content/GOT-OCR2.0/GOT-OCR-2.0-master/GOT/model/GOT-OCR2_0 --image-file /content/ANACO-MOTOS-26-3-10AM.jpg --type ocr --device cpu

2024-09-25 02:35:41.540578: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-09-25 02:35:41.574488: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-09-25 02:35:41.584910: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-09-25 02:35:43.233724: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
<|im_start|>system
You should follow the instructions carefully and explain your answers in detail.<|im_end|><|im_start|>user

@ayttop
Copy link

ayttop commented Sep 25, 2024

ANACO-MOTOS-26-3-10AM

@ayttop
Copy link

ayttop commented Sep 25, 2024

CONVOCATORIA
Anaco Motors C.A.
Rif. J-08013854-6
ANACO, 16-03-2024.
Se convoca a los señores accionistas de esta empresa, a LA
ASAMBLEA GENERAL EXTRAORDINARIA, a celebrarse en fecha 26 de
marzo de 2024, en la sede de ANACO MOTORS, C.A., Ubicada en la
Avenida José Antonio Anzoátegui, con calle Altamira, frente al
Aeropuerto de Anaco, Estado Anzoátegui. HORA: 10 a.m.
PUNTOS A TRATAR:
ACTUALIZAR LA PARTE LEGAL, OPERATIVA Y FINANCIERA DE LA
EMPRESA.
1.Ratificación en sus cargos de los miembros de la Junta
Directiva,
comisario,
suplente

@ayttop
Copy link

ayttop commented Sep 25, 2024

y
Representante
Legal,
designada en Asamblea de fecha 25/01/2024. Ampliación del
tiempo de duración en sus cargos, POR UN PERIODO DE TRES (03)
ANOS. Revisión de los deberes formales de la compañía.
2.Aprobación de los Balances y Estados Financieros años 2016 al
2023.
3.Revisión de los activos e inventarios, dejar constancia de la
propiedad de la empresa de los mismos, de acuerdo a
documentación solicitada y Balance.
LA JUNTA DIRECTIVA
Convocatoria de Asamblea de accionistas según el criterio vinculante de Sala
Constitucional TSJ de fecha 9/12/2016 exp. 16-0826.

@ayttop
Copy link

ayttop commented Sep 25, 2024

thank you ElvisClaros

@ayttop
Copy link

ayttop commented Sep 25, 2024

aaaaaaaa

@ElvisClaros
feat: add device argument for model evaluation
9975aa5 
In GOT-OCR-2.0-master/GOT/demo/run_ocr_2.0_crop.py
- Introduced `device` argument to specify the device for model evaluation.
- Updated model loading and tensor operations to use the specified device.
- Adjusted image processing and tensor casting based on the device.
- Added `--device` argument to the command-line parser with default 'cuda'.
@ayttop
Copy link

ayttop commented Sep 25, 2024

https://colab.research.google.com/drive/1ax3QJps-upRq8QTeF3Ej7Elwnx8pLcK2#scrollTo=tvTNhGbo7cAE

@ayttop
Copy link

ayttop commented Sep 25, 2024

!git clone https://github.com/Ucas-HaoranWei/GOT-OCR2.0.git

%cd /content/GOT-OCR2.0/GOT-OCR-2.0-master
!pip install -e .
!pip install ninja
#!pip install flash-attn --no-build-isolation
%cd /content

!git clone https://huggingface.co/stepfun-ai/GOT-OCR2_0

شغال

!python3 /content/GOT-OCR2.0/GOT-OCR-2.0-master/GOT/demo/run_ocr_2.0.py --model-name /content/GOT-OCR2.0/GOT-OCR-2.0-master/GOT/model/GOT-OCR2_0 --image-file /content/ANACO-MOTOS-26-3-10AM.jpg --type ocr --device cpu

!python3 /content/GOT-OCR2.0/GOT-OCR-2.0-master/GOT/demo/run_ocr_2.0.py --model-name /content/GOT-OCR2.0/GOT-OCR-2.0-master/GOT/model/GOT-OCR2_0 --image-file /content/comprobar-sorteo-bonoloto-hoy-resultado-miercoles-8_98.webp --type ocr --device cpu

@whisper-bye
Copy link

Any update?

@pythons
Copy link

pythons commented Oct 22, 2024 •
edited
Loading

Can the ability to support the CPU run on macOS x86_64?

I got some problem while I run:
python3 GOT/demo/run_ocr_2.0.py --image-file 'ocr.png' --type ocr --device cpu

Traceback (most recent call last):
File "/Users/my/Documents/code/GOT-OCR2.0/GOT-OCR-2.0-master/GOT/demo/run_ocr_2.0.py", line 252, in
eval_model(args)
File "/Users/my/Documents/code/GOT-OCR2.0/GOT-OCR-2.0-master/GOT/demo/run_ocr_2.0.py", line 52, in eval_model
model = GOTQwenForCausalLM.from_pretrained(model_name, low_cpu_mem_usage=True, device_map=device, use_safetensors=True, pad_token_id=151643).eval()
File "/opt/anaconda3/envs/py310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3594, in from_pretrained
model = cls(config, *model_args, **model_kwargs)
File "/Users/my/Documents/code/GOT-OCR2.0/GOT-OCR-2.0-master/GOT/model/GOT_ocr_2_0.py", line 214, in init
self.model = GOTQwenModel(config)
File "/Users/my/Documents/code/GOT-OCR2.0/GOT-OCR-2.0-master/GOT/model/GOT_ocr_2_0.py", line 23, in init
super(GOTQwenModel, self).init(config)
File "/opt/anaconda3/envs/py310/lib/python3.10/site-packages/transformers/models/qwen2/modeling_qwen2.py", line 930, in init
self.embed_tokens = nn.Embedding(config.vocab_size, config.hidden_size, self.padding_idx)
File "/opt/anaconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/sparse.py", line 135, in init
assert padding_idx < self.num_embeddings, 'Padding_idx must be within num_embeddings'
AssertionError: Padding_idx must be within num_embeddings

@leeey111
Copy link

@Ucas-HaoranWei hi, is there any problem that can't be merge? this feature is very useful, really hope it can be merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@ElvisClaros @Ucas-HaoranWei @iLoveBug @Zetaphor @ayttop @whisper-bye @pythons @leeey111