Running docker: "failed to find this automation in '/config' folder" (for packages, split config) #149

ghost · 2022-08-30T08:48:57Z

Describe the bug
Shortumation picks up the automation titles that are in packages or using !include directory, but I get the error:
"failed to find this automation in '/config' folder"

To Reproduce
Steps to reproduce the behavior:

run Shortumation.

Expected behavior
I expect to not get this error

Screenshots

Desktop (please complete the following information):

OS: MacOS 12.5.1 (but HA running on a Pi-4)
Browser Safari
Version 15.6.1

Additional context
It works fine for automations created using the UI (using /config/automations.yaml)

asosnovsky · 2022-08-31T01:52:08Z

what version of Shortumation you are running? it seems like you have an old build (see bottom right of the UI)

ghost · 2022-08-31T07:23:05Z

I have v 0.6.3

asosnovsky · 2022-08-31T18:11:22Z

strange, do you mind sharing your configuratio.yaml (just the parts to do with automations/packages) or anything else that could help me replicate the issue on my end

ghost · 2022-08-31T20:07:17Z

of course, no problem:

packages: !include_dir_named JCHH/packages/
automation: !include automations.yaml
automation JCH: !include_dir_merge_list JCHH/automations/

I'm not honestly sure what else will help. I am running HA in Docker on DietPi (A Lightweight Debian OS).

asosnovsky · 2022-09-01T02:19:37Z

@jchhenderson please try to update!

also if you still encounter issues please re-post the logs :)

sorry that this issue keeps draging on...

ghost · 2022-09-01T07:43:53Z

sorry that this issue keeps draging on...

Hey, no worries - you are putting in a lot of work here to provide free software and I appreciate that :-)
Updating now and will let you know...

Edit: nope, still not working.

asosnovsky · 2022-09-01T13:15:11Z

@jchhenderson wait I just realized. Are you running shortumation in a separate docker container not addon mode?

If yea, can you do the following:

Check that the tag is mapped to the latest version v0.7.0
if using compose do docker compose pull && docker compose build --no-cache
if using just docker docker pull asosnovsky/shortumation:v0.7.0

I'm starting to think you are experiencing a caching issue in docker.

Sholofly · 2022-09-01T13:25:53Z

I'm having the same issue. Running v7.0 on latest beta 2022.9.0b1. Installed as Add On.
I have my automation integration as a package: folder/integration.yaml in which I include the main integrations.yaml from the root of my config folder. Logging says:

2022-09-01:15:28:31|ERROR[/usr/local/lib/python3.10/site-packages/hypercorn/logging.py:100@exception] Error in ASGI Framework
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/hypercorn/asyncio/task_group.py", line 21, in _handle
    await invoke_asgi(app, scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/hypercorn/utils.py", line 247, in invoke_asgi
    await app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 269, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 92, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 147, in simple_response
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/exceptions.py", line 93, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/exceptions.py", line 82, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 670, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 266, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 65, in app
    response = await func(request)
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 227, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 162, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/usr/local/lib/python3.10/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/app/src/api/routes/automations.py", line 18, in list_autos
    automations.reload()
  File "/app/src/automations/manager.py", line 45, in reload
    batch.append(next(auto_it))
  File "/app/src/automations/loader.py", line 37, in load_and_iter_automations
    for ref in extract_automation_refs(hass_config):
  File "/app/src/automations/loader.py", line 59, in extract_automation_refs
    for ref in chain(
  File "/app/src/automations/loader.py", line 100, in extract_automation_package_refs
    yield from extract_automation_inline_package_refs(
  File "/app/src/automations/loader.py", line 172, in extract_automation_inline_package_refs
    raise AssertionError(
AssertionError: configurations.homeassistant.packages[media_player] must be a dictionary!

ghost · 2022-09-01T14:05:00Z

@jchhenderson wait I just realized. Are you running shortumation in a separate docker container not addon mode?

yes, I am running in a seperate docker container as I am running HA in docker as well. (not HA OS or supervisor).

Check that the tag is mapped to the latest version v0.7.0

the tag in docker-compose? yes, it's 0.7.0 image: asosnovsky/shortumation-armv7:v0.7.0

if using compose do docker compose pull && docker compose build --no-cache

made no difference, sorry.

edit: My full docker-compose (for Shortumation) is:

  shortumation: # http://dietpi:8000/web
    container_name: shortumation
    image: asosnovsky/shortumation-armv7:v0.7.0
    ports:
      - 8000:8000
    extra_hosts:
      - "host.docker.internal:host-gateway"
    volumes:
      - ./homeassistant:/config       # path is same as Homeassistant
    environment:
      - SUPERVISOR_TOKEN=<renoved>
      - HASSIO_WS=ws://host.docker.internal:8123/api/websocket

asosnovsky · 2022-09-01T15:07:54Z

@Sholofly

I'm having the same issue. Running v7.0 on latest beta 2022.9.0b1. Installed as Add On. I have my automation integration as a package: folder/integration.yaml in which I include the main integrations.yaml from the root of my config folder. Logging says:

2022-09-01:15:28:31|ERROR[/usr/local/lib/python3.10/site-packages/hypercorn/logging.py:100@exception] Error in ASGI Framework
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/hypercorn/asyncio/task_group.py", line 21, in _handle
    await invoke_asgi(app, scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/hypercorn/utils.py", line 247, in invoke_asgi
    await app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 269, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 92, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 147, in simple_response
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/exceptions.py", line 93, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/exceptions.py", line 82, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 670, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 266, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 65, in app
    response = await func(request)
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 227, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 162, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/usr/local/lib/python3.10/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/app/src/api/routes/automations.py", line 18, in list_autos
    automations.reload()
  File "/app/src/automations/manager.py", line 45, in reload
    batch.append(next(auto_it))
  File "/app/src/automations/loader.py", line 37, in load_and_iter_automations
    for ref in extract_automation_refs(hass_config):
  File "/app/src/automations/loader.py", line 59, in extract_automation_refs
    for ref in chain(
  File "/app/src/automations/loader.py", line 100, in extract_automation_package_refs
    yield from extract_automation_inline_package_refs(
  File "/app/src/automations/loader.py", line 172, in extract_automation_inline_package_refs
    raise AssertionError(
AssertionError: configurations.homeassistant.packages[media_player] must be a dictionary!

I think your error is a bit different than this one. Mind opening a separate issue and also include there some parts of your config.yaml.

Although I think I know exactly why yours broke :o will try to make a fix in the next release.

asosnovsky · 2022-09-01T15:13:37Z

@jchhenderson wait I just realized. Are you running shortumation in a separate docker container not addon mode?

yes, I am running in a seperate docker container as I am running HA in docker as well. (not HA OS or supervisor).

Check that the tag is mapped to the latest version v0.7.0

the tag in docker-compose? yes, it's 0.7.0 image: asosnovsky/shortumation-armv7:v0.7.0

if using compose do docker compose pull && docker compose build --no-cache

made no difference, sorry.

edit: My full docker-compose (for Shortumation) is:
  shortumation: # http://dietpi:8000/web
    container_name: shortumation
    image: asosnovsky/shortumation-armv7:v0.7.0
    ports:
      - 8000:8000
    extra_hosts:
      - "host.docker.internal:host-gateway"
    volumes:
      - ./homeassistant:/config       # path is same as Homeassistant
    environment:
      - SUPERVISOR_TOKEN=<renoved>
      - HASSIO_WS=ws://host.docker.internal:8123/api/websocket

@jchhenderson do you mind posting a full log of what's going on in shortumation?
i.e. docker-compose logs shortumation > logs.log and then copy/upload that logs.log to here?

ghost · 2022-09-01T15:16:18Z

here you go.

logs.log

asosnovsky · 2022-09-01T17:01:02Z

here ya go!

shortumation  | 2022-09-01:10:47:01|ERROR[/app/src/automations/loader.py:51@load_and_iter_automations] when=reading file contents,automation_path=/config/JCHH/automations/switches/switch_bedroom.yaml,error='id',automation=....

looks like the automation in /config/JCHH/automations/switches/switch_bedroom.yaml is missing an id field!

strange that HA supports this o.0

I will do some testing on my end to see how they handle it. But this is undocumented behavior on their part :/

In the meantime, if you add an id: <something> to the automation it should get picked up

ghost · 2022-09-01T19:10:30Z

Hi,

yes, I saw that as well. I have no IDs for any automation. I added a test ID for 1 of the automations but it made no difference.

Here is the automation...

- alias: "HA: Restart"
  id: "unique_name"
  initial_state: true #ensure automation is turned on when restarting HA
  description: ''
  trigger:
    - platform: homeassistant
      event: start
  [...]

...and here is that automation in Shortumation:

edited: Typos and clarity.

asosnovsky · 2022-09-01T19:23:42Z

@Sholofly so I reviewed your issue a bit more. I am pretty sure that you have an invalid configuration file.

Packages should always map to a dictionary of overrides. When using the same configuration as you I get this error in HA (which tracks with the error I am throwing in Shortumation)

also moved your issue here #159

asosnovsky · 2022-09-01T19:24:20Z

@jchhenderson did you reload the automations after adding the id?

also can you confirm this behavior for the automations without an id:

ghost · 2022-09-01T19:33:19Z

yes, I reloaded the automations. All automations showed the same error.
I am currently adding IDs too all of them anyway and will report back...

ghost · 2022-09-01T19:47:29Z

OK, I may have had a caching issue, as they are all showing up now. However I have a different problem. SHortumation is reporting my automations as invalid (HA is is OK with them though).

asosnovsky · 2022-09-01T21:22:55Z

@jchhenderson yea HA loads them but won't let you see automations that have no ids.
I just finished implementing a readonly mode for automations that are missing an id. Once testing is done you should be able to use it in the next release.

ghost · 2022-09-01T22:03:57Z

@jchhenderson yea HA loads them but won't let you see automations that have no ids.
I just finished implementing a readonly mode for automations that are missing an id. Once testing is done you should be able to use it in the next release.

Thanks, I have added IDs to all may automations now, but I am now getting the error that I posted above whereby Shortumation thinks every automation has an error. Should I create a new issue for this?

asosnovsky · 2022-09-02T01:18:00Z

I think you might need to reload the automations. Like in the developer tools > yaml > automations.

But in general if you can wait till the next update before opening another issue that would be better :)

asosnovsky · 2022-09-02T01:29:49Z

@jchhenderson for now I am closing this issue, if you are still seeing some errors please post them in another issue (or reopen this)

ghost · 2022-09-02T07:15:52Z

OK, thanks. I;ll wait until the next release and then open a new issue if it is still occurring.

Thanks so much!!

ghost · 2022-09-02T08:11:08Z

I worked out what my last error was so will raise a ticket now if I cannot find one exists 👍

ghost added bug Something isn't working triage labels Aug 30, 2022

asosnovsky added high priority and removed triage labels Aug 31, 2022

asosnovsky added this to Shortumations Roadmap Aug 31, 2022

asosnovsky moved this to Triage in Shortumations Roadmap Aug 31, 2022

asosnovsky added this to the Version 1 milestone Aug 31, 2022

asosnovsky moved this from Triage to In Progress in Shortumations Roadmap Aug 31, 2022

asosnovsky mentioned this issue Sep 1, 2022

AssertionError: configurations.homeassistant.packages[media_player] must be a dictionary! #159

Open

asosnovsky moved this from In Progress to Next Release in Shortumations Roadmap Sep 2, 2022

asosnovsky moved this from Next Release to Released but waiting confirmation from users in Shortumations Roadmap Sep 2, 2022

asosnovsky closed this as completed Sep 2, 2022

Repository owner moved this from Released but waiting confirmation from users to Next Release in Shortumations Roadmap Sep 2, 2022

asosnovsky moved this from Next Release to Done in Shortumations Roadmap Sep 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Running docker: "failed to find this automation in '/config' folder" (for packages, split config) #149

Running docker: "failed to find this automation in '/config' folder" (for packages, split config) #149

ghost commented Aug 30, 2022 •

edited by ghost

Loading

asosnovsky commented Aug 31, 2022

ghost commented Aug 31, 2022

asosnovsky commented Aug 31, 2022

ghost commented Aug 31, 2022

asosnovsky commented Sep 1, 2022 •

edited

Loading

ghost commented Sep 1, 2022 •

edited by ghost

Loading

asosnovsky commented Sep 1, 2022

Sholofly commented Sep 1, 2022 •

edited

Loading

ghost commented Sep 1, 2022 •

edited by ghost

Loading

asosnovsky commented Sep 1, 2022 •

edited

Loading

asosnovsky commented Sep 1, 2022

ghost commented Sep 1, 2022

asosnovsky commented Sep 1, 2022 •

edited

Loading

ghost commented Sep 1, 2022 •

edited by ghost

Loading

asosnovsky commented Sep 1, 2022 •

edited

Loading

asosnovsky commented Sep 1, 2022 •

edited

Loading

ghost commented Sep 1, 2022

ghost commented Sep 1, 2022

asosnovsky commented Sep 1, 2022

ghost commented Sep 1, 2022

asosnovsky commented Sep 2, 2022

asosnovsky commented Sep 2, 2022

ghost commented Sep 2, 2022

ghost commented Sep 2, 2022

Running docker: "failed to find this automation in '/config' folder" (for packages, split config) #149

Running docker: "failed to find this automation in '/config' folder" (for packages, split config) #149

Comments

ghost commented Aug 30, 2022 • edited by ghost Loading

asosnovsky commented Aug 31, 2022

ghost commented Aug 31, 2022

asosnovsky commented Aug 31, 2022

ghost commented Aug 31, 2022

asosnovsky commented Sep 1, 2022 • edited Loading

ghost commented Sep 1, 2022 • edited by ghost Loading

asosnovsky commented Sep 1, 2022

Sholofly commented Sep 1, 2022 • edited Loading

ghost commented Sep 1, 2022 • edited by ghost Loading

asosnovsky commented Sep 1, 2022 • edited Loading

asosnovsky commented Sep 1, 2022

ghost commented Sep 1, 2022

asosnovsky commented Sep 1, 2022 • edited Loading

ghost commented Sep 1, 2022 • edited by ghost Loading

asosnovsky commented Sep 1, 2022 • edited Loading

asosnovsky commented Sep 1, 2022 • edited Loading

ghost commented Sep 1, 2022

ghost commented Sep 1, 2022

asosnovsky commented Sep 1, 2022

ghost commented Sep 1, 2022

asosnovsky commented Sep 2, 2022

asosnovsky commented Sep 2, 2022

ghost commented Sep 2, 2022

ghost commented Sep 2, 2022

ghost commented Aug 30, 2022 •

edited by ghost

Loading

asosnovsky commented Sep 1, 2022 •

edited

Loading

ghost commented Sep 1, 2022 •

edited by ghost

Loading

Sholofly commented Sep 1, 2022 •

edited

Loading

ghost commented Sep 1, 2022 •

edited by ghost

Loading

asosnovsky commented Sep 1, 2022 •

edited

Loading

asosnovsky commented Sep 1, 2022 •

edited

Loading

ghost commented Sep 1, 2022 •

edited by ghost

Loading

asosnovsky commented Sep 1, 2022 •

edited

Loading

asosnovsky commented Sep 1, 2022 •

edited

Loading