colab cpu

[kim90000]

!python server.py --model models/openai-community_gpt2 --listen --cpu --auto-launch --use_eager_attention --share

00:42:38-984801 INFO Starting Text generation web UI
00:42:39-001040 INFO Loading "openai-community_gpt2"
00:42:39-011734 INFO TRANSFORMERS_PARAMS=
{ 'low_cpu_mem_usage': True,
'torch_dtype': torch.float32,
'attn_implementation': 'eager'}

2024-12-08 00:42:40.906230: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-12-08 00:42:40.931061: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-12-08 00:42:40.938311: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-08 00:42:40.954756: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-08 00:42:42.328445: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/usr/local/lib/python3.10/dist-packages/transformers/generation/configuration_utils.py:600: UserWarning: do_sample is set to False. However, min_p is set to 0.0 -- this flag is only used in sample-based generation modes. You should set do_sample=True or unset min_p.
warnings.warn(
00:42:45-005625 INFO Loaded "openai-community_gpt2" in 6.00 seconds.
00:42:45-008568 INFO LOADER: "Transformers"
00:42:45-010282 INFO TRUNCATION LENGTH: 2048
00:42:45-011883 INFO INSTRUCTION TEMPLATE: "Alpaca"
IMPORTANT: You are using gradio version 4.26.0, however version 4.44.1 is available, please upgrade.

IMPORTANT: You are using gradio version 4.26.0, however version 4.44.1 is available, please upgrade.

IMPORTANT: You are using gradio version 4.26.0, however version 4.44.1 is available, please upgrade.

IMPORTANT: You are using gradio version 4.26.0, however version 4.44.1 is available, please upgrade.

IMPORTANT: You are using gradio version 4.26.0, however version 4.44.1 is available, please upgrade.

IMPORTANT: You are using gradio version 4.26.0, however version 4.44.1 is available, please upgrade.

IMPORTANT: You are using gradio version 4.26.0, however version 4.44.1 is available, please upgrade.

Running on local URL: http://0.0.0.0:7860/

Running on public URL: https://d821d49b384b770ad1.gradio.live/

This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run gradio deploy from Terminal to deploy to Spaces (https://huggingface.co/spaces)
Output generated in 1.94 seconds (2.57 tokens/s, 5 tokens, context 128, seed 976363676)
Output generated in 2.69 seconds (2.23 tokens/s, 6 tokens, context 140, seed 868658989)
00:45:59-996548 INFO Received Ctrl+C. Shutting down Text generation web UI gracefully.
Killing tunnel 0.0.0.0:7860 <> https://d821d49b384b770ad1.gradio.live/

it run on cpu on colab

git clone https://github.com/oobabooga/text-generation-webui.git

pip install -r /content/text-generation-webui/requirements_cpu_only_noavx2.txt

pip install -r /content/text-generation-webui/requirements_cpu_only.txt

!python server.py --model models/openai-community_gpt2 --listen --cpu --auto-launch --use_eager_attention --share