Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unraid app development #5

Open
Steel-skull opened this issue Jun 11, 2023 · 18 comments
Open

unraid app development #5

Steel-skull opened this issue Jun 11, 2023 · 18 comments
Labels
enhancement New feature or request

Comments

@Steel-skull
Copy link

was wanting to make sure your cool with using and forking your repo to make a unraid community app

@Atinoda
Copy link
Owner

Atinoda commented Jun 12, 2023

Definitely cool with that - the more ways for people to easily install the better! Is there anything you'd like to share back to this repo, or any minor change requests here that would help with the Unraid app?

@Steel-skull
Copy link
Author

Awesome and not yet I'm gonna look into this more soon but if I have anything I'll let you know!

@simplesisu
Copy link

@Steel-skull
Hope all is well! Really looking forward to the community app. How is it going for you?!

Br,

@Atinoda Atinoda added the enhancement New feature or request label Aug 4, 2023
@SurvivaLlama
Copy link

Any update to the community app?

@RTStump
Copy link

RTStump commented Dec 6, 2023

Definitely cool with that - the more ways for people to easily install the better! Is there anything you'd like to share back to this repo, or any minor change requests here that would help with the Unraid app?

I know this was from a while ago, but changing the volume mapping where dependencies and such are installed in order to prevent the install from blowing up the Docker image size would be great. Right now it's adding roughly 15 GB to people's docker images with default configuration.

@Atinoda
Copy link
Owner

Atinoda commented Dec 6, 2023

@RTStump Thanks for the suggestion! I have an extensive refactor intended for the near future - and I would definitely like to slim down the image, in addition to creating some more contemporary variants. Could you please provide more details on what folders are blowing up the size? For instance, are you referring to extension dependencies or other post-install actions?

@Steel-skull
Copy link
Author

Sorry for never getting back on this I've been working on family issues and haven't been able to work on this.

@Steel-skull
Copy link
Author

Steel-skull commented Mar 3, 2024

<?xml version="1.0"?>
<Container version="2">
  <Name>text-generation-webui</Name>
  <Repository>atinoda/text-generation-webui:latest</Repository>
  <Registry/>
  <Network>bridge</Network>
  <MyIP/>
  <Shell>sh</Shell>
  <Privileged>false</Privileged>
  <Support/>
  <Project/>
  <Overview/>
  <Category/>
  <WebUI>http://[IP]:[PORT:7860]</WebUI>
  <TemplateURL/>
  <Icon/>
  <ExtraParams>--runtime=nvidia</ExtraParams>
  <PostArgs/>
  <CPUset/>
  <DateInstalled>1709426164</DateInstalled>
  <DonateText/>
  <DonateLink/>
  <Requires/>
  <Config Name="WebUI" Target="7860" Default="7860" Mode="tcp" Description="" Type="Port" Display="always" Required="true" Mask="false">7860</Config>
  <Config Name="Open AI API" Target="5000" Default="5000" Mode="tcp" Description="" Type="Port" Display="always" Required="true" Mask="false">5000</Config>
  <Config Name="Characters" Target="/app/characters" Default="./config/characters" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/disk1/docker/compose/text-generation-webui-docker/config/characters/</Config>
  <Config Name="Loras" Target="/app/loras" Default="./config/loras" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/docker/compose/text-generation-webui-docker/config/loras/</Config>
  <Config Name="Models" Target="/app/models" Default="./config/models" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/docker/compose/text-generation-webui-docker/config/models/</Config>
  <Config Name="Presets" Target="/app/presets" Default="./config/presets" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/docker/compose/text-generation-webui-docker/config/presets/</Config>
  <Config Name="Prompts" Target="/app/prompts" Default="./config/prompts" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/docker/compose/text-generation-webui-docker/config/prompts/</Config>
  <Config Name="Training" Target="/app/training" Default="./config/training" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/docker/compose/text-generation-webui-docker/config/training/</Config>
  <Config Name="Extensions" Target="/app/extensions" Default="./config/extensions" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/docker/compose/text-generation-webui-docker/config/extensions/</Config>
  <Config Name="EXTRA_LAUNCH_ARGS" Target="EXTRA_LAUNCH_ARGS" Default="" Mode="" Description="" Type="Variable" Display="always" Required="false" Mask="false">"--listen --verbose --api"</Config>
  <Config Name="NVIDIA_VISIBLE_DEVICES" Target="NVIDIA_VISIBLE_DEVICES" Default="all" Mode="" Description="" Type="Variable" Display="always" Required="false" Mask="false">all</Config>
  <Config Name="NVIDIA_DRIVER_CAPABILITIES" Target="NVIDIA_DRIVER_CAPABILITIES" Default="all" Mode="" Description="" Type="Variable" Display="always" Required="false" Mask="false">all</Config>
</Container>

template for unraid to load web-ui with --api

sorry for the delay life got in the way

@Atinoda

@Atinoda
Copy link
Owner

Atinoda commented Mar 13, 2024

@Steel-skull apologies for my delayed reply - thank you for sharing your template! I have it on my list to include it in the repo somewhere, or perhaps to submit to the community repository. However, I need to spin up an Unraid VM or put it on some spare hardware to understand the OS better before I can properly support it.

@Steel-skull
Copy link
Author

@Steel-skull apologies for my delayed reply - thank you for sharing your template! I have it on my list to include it in the repo somewhere, or perhaps to submit to the community repository. However, I need to spin up an Unraid VM or put it on some spare hardware to understand the OS better before I can properly support it.

Hey man, no prob! Sorry for the delay in getting it done. I can probably help with that as I already tinker with the container's and unraid on the regular, I was looking into submitting it to the community store altho I've been putting off looking into the submission process. I'll probably do that this weekend.

@Atinoda
Copy link
Owner

Atinoda commented Mar 13, 2024

Awesome - give us a heads up if you get it published to the community store and I'll add a link to it in the README.md!

P.S. I also had my planned refactor get backburnered for a few months by family life - but I think that just means you're prioritising the right things :)

@Basicallyapenguin
Copy link

Hello any update on this? Would love for this to be in CA for unraid!

@Steel-skull
Copy link
Author

Sadly, I've stopped using and working on it as updates from webui kept breaking the container on the regular, and there was no fix in site, so I moved to Ollama for easier deployment.

Not to mention, one update crashed my server so hard I had to reinstall unraid on a separate usb drive as it corrupted the old drive.

However, I'll take a look and see if it was ever fixed.

@Basicallyapenguin
Copy link

Okay no worries, thanks for the response!

@Atinoda
Copy link
Owner

Atinoda commented Jul 12, 2024

I'm planning a big update to the container this week to bring it back in line with the upstream project and close off a few feature requests. If I have the time, I'll also look at what's involved in publishing an Unraid version.

@Steel-skull - sorry to hear that you had issues with it. I've also been experimenting with ollama as a back-end for a new project... I don't see how this (or any other container) could brick your server though - that's kind of the point of docker container sandboxes! Barring sensitive bind mounts or resource starvation, of course. Have you had specific issues with the stable versions? I would really appreciate you sharing them so I can investigate and fix for the big update.

@Steel-skull
Copy link
Author

Steel-skull commented Jul 16, 2024

I'm planning a big update to the container this week to bring it back in line with the upstream project and close off a few feature requests. If I have the time, I'll also look at what's involved in publishing an Unraid version.

@Steel-skull - sorry to hear that you had issues with it. I've also been experimenting with ollama as a back-end for a new project... I don't see how this (or any other container) could brick your server though - that's kind of the point of docker container sandboxes! Barring sensitive bind mounts or resource starvation, of course. Have you had specific issues with the stable versions? I would really appreciate you sharing them so I can investigate and fix for the big update.

Good to hear, Ive been playing with making an Unraid container again as well and have a functional container running.

Honestly I thought the same thing lol I personally think it was a complete driver failure that led to corruption but with out the logs from the failure its hard to diagnose.

id say no specific issues currently with the container but im testing it further in the next few days.

Current working Unraid docker config:

<?xml version="1.0"?>
<Container version="2">
  <Name>text-generation-webui-test</Name>
  <Repository>atinoda/text-generation-webui</Repository>
  <Registry>https://hub.docker.com/r/atinoda/text-generation-webui/</Registry>
  <Network>bridge</Network>
  <MyIP/>
  <Shell>sh</Shell>
  <Privileged>false</Privileged>
  <Support>https://hub.docker.com/r/atinoda/text-generation-webui/</Support>
  <Project/>
  <Overview>Oobabooga's Text Generation Web UI - a gradio web UI for running Large Language Models (LLMs).&#xD;
&#xD;
Container made by Atinoda&#xD;
https://hub.docker.com/r/atinoda/text-generation-webui/&#xD;
&#xD;
Cuda: "--runtime=nvidia --gpus=all"</Overview>
  <Category/>
  <WebUI>http://[IP]:[PORT:7860]</WebUI>
  <TemplateURL/>
  <Icon/>
  <ExtraParams>--runtime=nvidia --gpus=all </ExtraParams>
  <PostArgs/>
  <CPUset/>
  <DateInstalled>1721098068</DateInstalled>
  <DonateText/>
  <DonateLink/>
  <Requires/>
  <Config Name="Container Port 1" Target="5000" Default="5000" Mode="tcp" Description="" Type="Port" Display="always" Required="false" Mask="false">5000</Config>
  <Config Name="Container Port 2" Target="5005" Default="5005" Mode="tcp" Description="" Type="Port" Display="always" Required="false" Mask="false">5005</Config>
  <Config Name="Container Port 3" Target="7860" Default="7860" Mode="tcp" Description="" Type="Port" Display="always" Required="false" Mask="false">7860</Config>
  <Config Name="Container Variable 1" Target="VIRTUAL_ENV" Default="/venv" Mode="{3}" Description="" Type="Variable" Display="always" Required="false" Mask="false">/venv</Config>
  <Config Name="Container Variable 2" Target="PYTHONUNBUFFERED" Default="1" Mode="{3}" Description="" Type="Variable" Display="always" Required="false" Mask="false">1</Config>
  <Config Name="Container Variable 5" Target="EXTRA_LAUNCH_ARGS" Default="" Mode="{3}" Description="" Type="Variable" Display="always" Required="false" Mask="false">--listen --verbose --api</Config>
  <Config Name="Cashe" Target="/root/.cache" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/text-webui/.cashe</Config>
  <Config Name="Characters" Target="/app/characters" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/text-webui/characters</Config>
  <Config Name="Instruction-Templates" Target="/app/instruction-templates" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/text-webui/instruction-templates</Config>
  <Config Name="Loras" Target="/app/loras" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/text-webui/loras</Config>
  <Config Name="Models" Target="/app/models" Default="" Mode="rw" Description="I reccomend changing this to a spot on your array not cashe drive" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/text-webui/models</Config>
  <Config Name="Prompts" Target="/app/prompts" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/text-webui/prompts</Config>
  <Config Name="Training" Target="/app/training" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/text-webui/training</Config>
  <Config Name="Extensions" Target="/app/extensions" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/text-webui/extensions</Config>
</Container>

ill update again once im able to confirm its all working no issue

@Steel-skull
Copy link
Author

Steel-skull commented Jul 20, 2024

@Atinoda ok so no big issue with the template, i am however receiving this:

Traceback (most recent call last): File "/app/modules/callbacks.py", line 61, in gentask ret = self.mfunc(callback=_callback, *args, **self.kwargs) File "/app/modules/text_generation.py", line 393, in generate_with_callback shared.model.generate(**kwargs) File "/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/venv/lib/python3.10/site-packages/transformers/generation/utils.py", line 1758, in generate result = self._sample( File "/venv/lib/python3.10/site-packages/transformers/generation/utils.py", line 2437, in _sample next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1) RuntimeError: probability tensor contains either inf, nan or element < 0

when i prompt with api through sillytavern regardless of model

@nwithan8
Copy link

I'm planning a big update to the container this week to bring it back in line with the upstream project and close off a few feature requests. If I have the time, I'll also look at what's involved in publishing an Unraid version.
@Steel-skull - sorry to hear that you had issues with it. I've also been experimenting with ollama as a back-end for a new project... I don't see how this (or any other container) could brick your server though - that's kind of the point of docker container sandboxes! Barring sensitive bind mounts or resource starvation, of course. Have you had specific issues with the stable versions? I would really appreciate you sharing them so I can investigate and fix for the big update.

Good to hear, Ive been playing with making an Unraid container again as well and have a functional container running.

Honestly I thought the same thing lol I personally think it was a complete driver failure that led to corruption but with out the logs from the failure its hard to diagnose.

id say no specific issues currently with the container but im testing it further in the next few days.

Current working Unraid docker config:

<?xml version="1.0"?>
<Container version="2">
  <Name>text-generation-webui-test</Name>
  <Repository>atinoda/text-generation-webui</Repository>
  <Registry>https://hub.docker.com/r/atinoda/text-generation-webui/</Registry>
  <Network>bridge</Network>
  <MyIP/>
  <Shell>sh</Shell>
  <Privileged>false</Privileged>
  <Support>https://hub.docker.com/r/atinoda/text-generation-webui/</Support>
  <Project/>
  <Overview>Oobabooga's Text Generation Web UI - a gradio web UI for running Large Language Models (LLMs).&#xD;
&#xD;
Container made by Atinoda&#xD;
https://hub.docker.com/r/atinoda/text-generation-webui/&#xD;
&#xD;
Cuda: "--runtime=nvidia --gpus=all"</Overview>
  <Category/>
  <WebUI>http://[IP]:[PORT:7860]</WebUI>
  <TemplateURL/>
  <Icon/>
  <ExtraParams>--runtime=nvidia --gpus=all </ExtraParams>
  <PostArgs/>
  <CPUset/>
  <DateInstalled>1721098068</DateInstalled>
  <DonateText/>
  <DonateLink/>
  <Requires/>
  <Config Name="Container Port 1" Target="5000" Default="5000" Mode="tcp" Description="" Type="Port" Display="always" Required="false" Mask="false">5000</Config>
  <Config Name="Container Port 2" Target="5005" Default="5005" Mode="tcp" Description="" Type="Port" Display="always" Required="false" Mask="false">5005</Config>
  <Config Name="Container Port 3" Target="7860" Default="7860" Mode="tcp" Description="" Type="Port" Display="always" Required="false" Mask="false">7860</Config>
  <Config Name="Container Variable 1" Target="VIRTUAL_ENV" Default="/venv" Mode="{3}" Description="" Type="Variable" Display="always" Required="false" Mask="false">/venv</Config>
  <Config Name="Container Variable 2" Target="PYTHONUNBUFFERED" Default="1" Mode="{3}" Description="" Type="Variable" Display="always" Required="false" Mask="false">1</Config>
  <Config Name="Container Variable 5" Target="EXTRA_LAUNCH_ARGS" Default="" Mode="{3}" Description="" Type="Variable" Display="always" Required="false" Mask="false">--listen --verbose --api</Config>
  <Config Name="Cashe" Target="/root/.cache" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/text-webui/.cashe</Config>
  <Config Name="Characters" Target="/app/characters" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/text-webui/characters</Config>
  <Config Name="Instruction-Templates" Target="/app/instruction-templates" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/text-webui/instruction-templates</Config>
  <Config Name="Loras" Target="/app/loras" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/text-webui/loras</Config>
  <Config Name="Models" Target="/app/models" Default="" Mode="rw" Description="I reccomend changing this to a spot on your array not cashe drive" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/text-webui/models</Config>
  <Config Name="Prompts" Target="/app/prompts" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/text-webui/prompts</Config>
  <Config Name="Training" Target="/app/training" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/text-webui/training</Config>
  <Config Name="Extensions" Target="/app/extensions" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/text-webui/extensions</Config>
</Container>

ill update again once im able to confirm its all working no issue

Well bummer, that's what I get for waiting too long to get around to making an Unraid template for the project (although it looks like this has been in the works for a while). When you're ready to publish @Steel-skull , if you don't have your own repo in the CA store, feel free to submit it to the SelfHosters repo: https://github.com/selfhosters/unRAID-CA-templates

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

7 participants