Other deferrable loads get clobbered using the thermal model.

[purcell-lab]

Describe the bug

When using the thermal model, other deferrable loads seem to get clobbered; def_hours set to zero? = not scheduled:

        "def_load_config": [{}, {}, {}, 
          {"thermal_config": 
            { "heating_rate": -5.0, 
              "sense": "cool",
              "cooling_constant": 0.2, 
              "overshoot_temperature": 11.0, 
              "start_temperature": 15.9, 
              "desired_temperatures": [26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26]
            }
          }, {}, {}],
        "outdoor_temperature_forecast": [15, 15, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 15, 15, 18, 18, 21, 21, 23, 23, 25, 25, 27, 27, 27, 27, 28, 28, 28, 28, 28, 28, 27, 27, 25, 25, 23, 23, 21, 21, 21, 21, 20, 20, 19, 19],

Below I'm using the thermal model for deferrable load 3, my HVAC, but the other loads are not scheduled, you can see lots of PV_curtailment so I have lots of additional capacity available.

2024-10-04 22:53:00,271 - web_server - INFO - Passed runtime parameters: {'load_cost_forecast': [0.23, 0.2, 0.19, 0.18, 0.18, 0.18, 0.17, 0.17, 0.17, 0.17, 0.17, 0.17, 0.17, 0.17, 0.17, 0.16, 0.12, 0.09, 0.07, 0.07, 0.07, 0.04, 0.04, 0.04, 0.04, 0.04, 0.04, 0.04, 0.04, 0.04, 0.04, 0.04, 0.05, 0.06, 0.08, 0.25, 0.3, 0.34, 0.36, 0.37, 0.36, 0.35, 0.34, 0.34, 0.34, 0.21, 0.2, 0.2, 0.2], 'prod_price_forecast': [0.1, 0.08, 0.07, 0.07, 0.06, 0.06, 0.06, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.04, 0.01, -0.02, -0.04, -0.04, -0.04, -0.04, -0.04, -0.04, -0.05, -0.04, -0.04, -0.04, -0.04, -0.04, -0.04, -0.04, -0.04, -0.03, -0.01, 0.01, 0.06, 0.09, 0.11, 0.12, 0.11, 0.1, 0.09, 0.09, 0.09, 0.09, 0.08, 0.08, 0.08], 'load_power_forecast': [531, 300, 300, 300, 300, 300, 300, 700, 700, 1200, 400, 400, 300, 300, 300, 300, 800, 1000, 1000, 2000, 1000, 500, 500, 500, 600, 600, 3100, 3100, 300, 300, 300, 300, 300, 300, 300, 300, 600, 700, 700, 1700, 1200, 800, 600, 500, 400, 500, 600, 700, 300, 300, 300, 300, 300, 300, 300, 700, 700, 1200, 400, 400, 300, 300, 300, 300, 800, 1000, 1000, 2000, 1000, 500, 500, 500, 600, 600, 3100, 3100, 300, 300, 300, 300, 300, 300, 300, 300, 600, 700, 700, 1700, 1200, 800, 600, 500, 400, 500, 600, 700], 'prediction_horizon': 48, 'def_load_config': [{}, {}, {}, {'thermal_config': {'heating_rate': -5.0, 'sense': 'cool', 'cooling_constant': 0.2, 'overshoot_temperature': 11.0, 'start_temperature': 15.7, 'desired_temperatures': [26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26]}}, {}, {}], 'outdoor_temperature_forecast': [15, 15, 15, 15, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 15, 15, 18, 18, 21, 21, 23, 23, 25, 25, 27, 27, 27, 27, 28, 28, 28, 28, 28, 28, 27, 27, 25, 25, 23, 23, 21, 21, 21, 21, 20, 20], 'alpha': 1, 'beta': 0, 'num_def_loads': 6, 'def_total_hours': [0, 9, 0, 6, 9, 4], 'def_end_timestep': [0, 0, 0, 0, 0, 0], 'P_deferrable_nom': [300, 6333, 11520, 11000, 600, 11520], 'def_current_state': [0, 0, 0, 0, 0, 0], 'def_start_penalty': [1, 1, 1, 1, 1, 1], 'treat_def_as_semi_cont': [1, 1, 0, 0, 1, 0], 'set_def_constant': [0, 0, 0, 0, 0, 0], 'soc_init': 0.11, 'soc_final': 1}
2024-10-04 22:53:00,272 - web_server - INFO -  >> Setting input data dict
2024-10-04 22:53:00,272 - web_server - INFO - Setting up needed data
2024-10-04 22:53:00,278 - web_server - INFO - Retrieve hass get data method initiated...
2024-10-04 22:53:02,175 - web_server - INFO - Retrieving weather forecast data using method = scrapper
2024-10-04 22:53:05,624 - web_server - INFO -  >> Performing naive MPC optimization...
2024-10-04 22:53:05,625 - web_server - INFO - Performing naive MPC optimization
2024-10-04 22:53:05,656 - web_server - INFO - Perform an iteration of a naive MPC controller
2024-10-04 22:53:05,880 - web_server - DEBUG - Deferrable load 0: Proposed optimization window: 0 --> 0
2024-10-04 22:53:05,881 - web_server - DEBUG - Deferrable load 0: Validated optimization window: 0 --> 0
2024-10-04 22:53:05,890 - web_server - DEBUG - Deferrable load 1: Proposed optimization window: 0 --> 0
2024-10-04 22:53:05,891 - web_server - DEBUG - Deferrable load 1: Validated optimization window: 0 --> 0
2024-10-04 22:53:05,900 - web_server - DEBUG - Deferrable load 2: Proposed optimization window: 0 --> 0
2024-10-04 22:53:05,900 - web_server - DEBUG - Deferrable load 2: Validated optimization window: 0 --> 0
2024-10-04 22:53:05,920 - web_server - DEBUG - Deferrable load 3: Proposed optimization window: 0 --> 0
2024-10-04 22:53:05,920 - web_server - DEBUG - Deferrable load 3: Validated optimization window: 0 --> 0
2024-10-04 22:53:05,926 - web_server - DEBUG - Deferrable load 4: Proposed optimization window: 0 --> 0
2024-10-04 22:53:05,926 - web_server - DEBUG - Deferrable load 4: Validated optimization window: 0 --> 0
2024-10-04 22:53:05,935 - web_server - DEBUG - Deferrable load 5: Proposed optimization window: 0 --> 0
2024-10-04 22:53:05,936 - web_server - DEBUG - Deferrable load 5: Validated optimization window: 0 --> 0
2024-10-04 22:53:07,124 - web_server - INFO - Status: Optimal
2024-10-04 22:53:07,125 - web_server - INFO - Total value of the Cost function = -0.84
2024-10-04 22:53:08,284 - web_server - INFO - Passed runtime parameters: {'custom_unit_load_cost_id': {'entity_id': 'sensor.unit_load_cost', 'unit_of_measurement': '$/kWh', 'friendly_name': 'Load Cost'}, 'custom_unit_prod_price_id': {'entity_id': 'sensor.unit_prod_price', 'unit_of_measurement': '$/kWh', 'friendly_name': 'Prod Price'}}
2024-10-04 22:53:08,284 - web_server - INFO -  >> Setting input data dict
2024-10-04 22:53:08,284 - web_server - INFO - Setting up needed data
2024-10-04 22:53:08,288 - web_server - INFO -  >> Publishing data...
2024-10-04 22:53:08,289 - web_server - INFO - Publishing data to HASS instance
2024-10-04 22:53:08,318 - web_server - INFO - Successfully posted to sensor.p_pv_forecast = -11.5
2024-10-04 22:53:08,339 - web_server - INFO - Successfully posted to sensor.p_load_forecast = 531
2024-10-04 22:53:08,363 - web_server - INFO - Successfully posted to sensor.p_pv_curtailment = 0.0
2024-10-04 22:53:08,385 - web_server - INFO - Successfully posted to sensor.p_deferrable0 = 0.0
2024-10-04 22:53:08,407 - web_server - INFO - Successfully posted to sensor.p_deferrable1 = 0.0
2024-10-04 22:53:08,430 - web_server - INFO - Successfully posted to sensor.p_deferrable2 = 0.0
2024-10-04 22:53:08,451 - web_server - INFO - Successfully posted to sensor.p_deferrable3 = 0.0
2024-10-04 22:53:08,474 - web_server - INFO - Successfully posted to sensor.p_deferrable4 = 0.0
2024-10-04 22:53:08,496 - web_server - INFO - Successfully posted to sensor.p_deferrable5 = 0.0
2024-10-04 22:53:08,522 - web_server - INFO - Successfully posted to sensor.p_batt_forecast = 2203.5
2024-10-04 22:53:08,549 - web_server - INFO - Successfully posted to sensor.soc_batt_forecast = 8.14
2024-10-04 22:53:08,571 - web_server - INFO - Successfully posted to sensor.p_grid_forecast = -1661.0
2024-10-04 22:53:08,588 - web_server - INFO - Successfully posted to sensor.total_cost_fun_value = -0.62
2024-10-04 22:53:08,603 - web_server - INFO - Successfully posted to sensor.optim_status = Optimal
2024-10-04 22:53:08,621 - web_server - INFO - Successfully posted to sensor.unit_load_cost = 0.23
2024-10-04 22:53:08,639 - web_server - INFO - Successfully posted to sensor.unit_prod_price = 0.1

To Reproduce
Setup MPC call with one thermal model and one other deferrable load.

Expected behavior
I am expecting that the thermal model and other deferrable loads could co-exist. Here is my hvac scheduled without using the thermal model.

[martinarva]

I'm having the same issue, but for testing reasons i'm currently running another emhass instance, so it's not biting yet.

[Argogma]

I'm having the same problem, posted already in: https://community.home-assistant.io/t/emhass-an-energy-management-for-home-assistant/338126/2667

Hello together, I really want emhass to work on my system. Without the termal model, everything is working fine:

trigger_tibber_solcast_mpc_zwei: "curl -i -H 'Content-Type: application/json' -X POST -d '{"pv_power_forecast": {{ [states('sensor.inverter_power_kombiniert')|int(0)]| list + (state_attr('sensor.solcast_pv_forecast_prognose_heute', 'detailedHourly') | map(attribute='pv_estimate') | map('multiply', 1000) | map('round', 0) | list + state_attr('sensor.solcast_pv_forecast_prognose_morgen', 'detailedHourly') | map(attribute='pv_estimate') | map('multiply', 1000) | map('round', 0) | list)[now().hour:][:23] }}, "load_cost_forecast":{{states('sensor.tibber_forecast') }}, "alpha":1, "beta":0,"prediction_horizon":24, "soc_init":{{states('sensor.battery_state_of_capacity')|float(0)/100 }},"soc_final":0.6}' http://localhost:5000/action/naive-mpc-optim"

--> All three p_deferrable are working.
Now i wanted to implement ther thermal model:

publish_data_thermal: "curl -i -H 'Content-Type:application/json' -X POST -d '{"def_load_config": [{"thermal_config": {}},{},{}] }' http://localhost:5000/action/publish-data"

trigger_mpc_thermal: "curl -i -H 'Content-Type: application/json' -X POST -d '{"pv_power_forecast": {{ [states('sensor.inverter_power_kombiniert')|int(0)]| list + (state_attr('sensor.solcast_pv_forecast_prognose_heute', 'detailedHourly') | map(attribute='pv_estimate') | map('multiply', 1000) | map('round', 0) | list + state_attr('sensor.solcast_pv_forecast_prognose_morgen', 'detailedHourly') | map(attribute='pv_estimate') | map('multiply', 1000) | map('round', 0) | list)[now().hour:][:23] }}, "load_cost_forecast":{{states('sensor.tibber_forecast') }}, "alpha":1, "beta":0,"prediction_horizon":24, "soc_init":{{states('sensor.battery_state_of_capacity')|float(0)/100 }},"soc_final":0.6, "outdoor_temperature_forecast": [17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17], "def_load_config": [{ "thermal_config": { "heating_rate": 8.63,"cooling_constant": 0.036,"overshoot_temperature": 75,"start_temperature": {{ states('sensor.warmwasseristtemperatur') | float(0) }},"desired_temperatures": [{{ states('sensor.warmwasseristtemperatur') | float(0) }}, {{ states('sensor.warmwasseristtemperatur') | float(0) }}, {{ states('sensor.warmwasseristtemperatur') | float(0) }}, {{ states('sensor.warmwasseristtemperatur') | float(0) }}, {{ states('sensor.warmwasseristtemperatur') | float(0) }}, {{ states('sensor.warmwasseristtemperatur') | float(0) }}, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55]}},{},{}],"num_def_loads": 3,"def_total_hours":[0,12,12],"treat_def_as_semi_cont": [1, 1, 1],"P_deferrable_nom":[2000, 670, 230]}' http://localhost:5000/action/naive-mpc-optim"

The protocoll:

2024-10-22 13:25:55,653 - web_server - INFO - Passed runtime parameters: {'pv_power_forecast': [10226, 3736, 4722, 3955, 2454, 582, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 612, 3082, 6137, 8174], 'load_cost_forecast': [0.2579, 0.261, 0.2771, 0.3014, 0.3044, 0.3405, 0.3612, 0.3092, 0.2918, 0.2874, 0.2787, 0.2697, 0.2726, 0.2711, 0.2725, 0.2756, 0.2849, 0.3023, 0.3791, 0.3914, 0.3263, 0.2974, 0.2852, 0.2754], 'alpha': 1, 'beta': 0, 'prediction_horizon': 24, 'soc_init': 0.08, 'soc_final': 0.6, 'outdoor_temperature_forecast': [17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17], 'def_load_config': [{'thermal_config': {'heating_rate': 8.63, 'cooling_constant': 0.036, 'overshoot_temperature': 75, 'start_temperature': 62.0, 'desired_temperatures': [62.0, 62.0, 62.0, 62.0, 62.0, 62.0, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55]}}, {}, {}], 'num_def_loads': 3, 'def_total_hours': [0, 12, 12], 'treat_def_as_semi_cont': [1, 1, 1], 'P_deferrable_nom': [2000, 670, 230]}
2024-10-22 13:25:55,654 - web_server - INFO - >> Setting input data dict
2024-10-22 13:25:55,654 - web_server - INFO - Setting up needed data
2024-10-22 13:25:55,657 - web_server - INFO - Retrieve hass get data method initiated...
2024-10-22 13:26:01,430 - web_server - INFO - Retrieving weather forecast data using method = list
2024-10-22 13:26:01,431 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2024-10-22 13:26:01,432 - web_server - INFO - Retrieve hass get data method initiated...
2024-10-22 13:26:14,439 - web_server - INFO - >> Performing naive MPC optimization...
2024-10-22 13:26:14,439 - web_server - INFO - Performing naive MPC optimization
2024-10-22 13:26:14,452 - web_server - INFO - Perform an iteration of a naive MPC controller
2024-10-22 13:26:14,513 - web_server - DEBUG - Deferrable load 0: Proposed optimization window: 0 --> 0
2024-10-22 13:26:14,513 - web_server - DEBUG - Deferrable load 0: Validated optimization window: 0 --> 0
2024-10-22 13:26:14,516 - web_server - DEBUG - Deferrable load 1: Proposed optimization window: 0 --> 0
2024-10-22 13:26:14,516 - web_server - DEBUG - Deferrable load 1: Validated optimization window: 0 --> 0
2024-10-22 13:26:14,518 - web_server - DEBUG - Deferrable load 2: Proposed optimization window: 0 --> 0
2024-10-22 13:26:14,518 - web_server - DEBUG - Deferrable load 2: Validated optimization window: 0 --> 0
2024-10-22 13:26:14,907 - web_server - INFO - Status: Optimal
2024-10-22 13:26:14,907 - web_server - INFO - Total value of the Cost function = 1.64
2024-10-22 13:26:25,928 - web_server - INFO - Passed runtime parameters: {'def_load_config': [{'thermal_config': {}}, {}, {}]}
2024-10-22 13:26:25,928 - web_server - INFO - >> Setting input data dict
2024-10-22 13:26:25,928 - web_server - INFO - Setting up needed data
2024-10-22 13:26:25,931 - web_server - INFO - >> Publishing data...
2024-10-22 13:26:25,931 - web_server - INFO - Publishing data to HASS instance
2024-10-22 13:26:25,957 - web_server - INFO - Successfully posted to sensor.p_pv_forecast = 10226
2024-10-22 13:26:25,975 - web_server - INFO - Successfully posted to sensor.p_load_forecast = 219.67
2024-10-22 13:26:25,995 - web_server - INFO - Successfully posted to sensor.p_hybrid_inverter = 5226.0
2024-10-22 13:26:26,012 - web_server - INFO - Successfully posted to sensor.p_deferrable0 = 2000.0
2024-10-22 13:26:26,026 - web_server - INFO - Successfully posted to sensor.p_deferrable1 = 0.0
2024-10-22 13:26:26,041 - web_server - INFO - Successfully posted to sensor.p_deferrable2 = 0.0
2024-10-22 13:26:26,053 - web_server - INFO - Successfully posted to sensor.temp_predicted0 = 62.0
2024-10-22 13:26:26,067 - web_server - INFO - Successfully posted to sensor.p_batt_forecast = -5000.0
2024-10-22 13:26:26,082 - web_server - INFO - Successfully posted to sensor.soc_batt_forecast = 53.0
2024-10-22 13:26:26,097 - web_server - INFO - Successfully posted to sensor.p_grid_forecast = -3006.33
2024-10-22 13:26:26,114 - web_server - INFO - Successfully posted to sensor.total_cost_fun_value = 1.64
2024-10-22 13:26:26,127 - web_server - INFO - Successfully posted to sensor.optim_status = Optimal
2024-10-22 13:26:26,143 - web_server - INFO - Successfully posted to sensor.unit_load_cost = 0.2579
2024-10-22 13:26:26,156 - web_server - INFO - Successfully posted to sensor.unit_prod_price = 0.078

Last run optimization results table

index	P_PV	P_Load	P_deferrable0	P_deferrable1	P_deferrable2	P_grid_pos	P_grid_neg	P_grid	P_batt	SOC_opt	P_hybrid_inverter	unit_load_cost	unit_prod_price	cost_profit	cost_fun_profit	predicted_temp_heater0	target_temp_heater0
2024-10-22 13:00:00+02:00	10226	219	2000	0	0	0	-3006	-3006	-5000	0.530	5226	0.258	0.078	0.234	0.234	62.00	62.0
2024-10-22 14:00:00+02:00	3736	243	0	0	0	0	0	0	-3492	0.844	243	0.261	0.078	-0.000	-0.000	69.01	62.0
...

The thermal_deferrable_load is my warm-water-tank (200L with a 2kW heating rod). I have 2 main problems:

1. if i want all desired_temperatures = 55°C, but if the starting temperatur =47°C the model wants to heat with 20kw instead of 2kW.--> this is why i work with my real heating temperature. Any suggestion how to avoid this problem?

index	P_PV	P_Load	P_deferrable0	P_deferrable1	P_deferrable2	P_grid_pos	P_grid_neg	P_grid	P_batt	SOC_opt	P_hybrid_inverter	unit_load_cost	unit_prod_price	cost_profit	cost_fun_profit	predicted_temp_heater0	target_temp_heater0
2024-10-22 10:00:00+02:00	750	238	20483	0	0	19341	0	19341	630	0.100	1380	0.292	0.078	-5.640	-5.640	47.00	55
2024-10-22 11:00:00+02:00	1233	204	2000	0	0	971	0	971	0	0.100	1233	0.278	0.078	-0.270	-0.270	124.58	55
2024-10-22 12:00:00+02:00	2437	206	2000	0	0	-230	0	-230	0	0.100	2437	0.265	0.078	0.061	0.061	94.48	55
2024-10-22 13:00:00+02:00	2812	219	2000	0	0	-592	0	-592	0	0.100	2812	0.258	0.078	0.153	0.153	75.22	55
2024-10-22 14:00:00+02:00	3736	243	2000	0	0	0	0	0	-1492	0.234	2243	0.261	0.078	-0.000	-0.000	62.89	55
2024-10-22 15:00:00+02:00	4722	220	0	0	0	0	-5216	-5216	714	0.170	5436	0.277	0.078	0.407	0.407	55.00	55

2. The bigger problem: With the thermal model, the othe deferrable loads are not working anymore (always 0).

If needed I can post my emhass-addon yaml.
Thanks a lot!