EMHASS makes non profitable decisions about battery charge, despite battery charge/discharge weights

[Octofinger]

Describe the bug
EMHASS naive_mpc_optim makes incorrect decisions on charging and discharging the battery if the energy price is more or less at a constant low.
Charging and discharging the battery comes at a cost, so I have set the corresponding parameters for weighting them

  - weight_battery_discharge: 0.5
  - weight_battery_charge: 0.5

So when the price is at a constant low, EMHASS alternately charges and discharges the battery, at almost full power, each time period.
In my scenario, purchase price was at a (more or less) constant 0.38 SEK/kWh, and sales price at 0.15 SEK/kWh. No profit is made, but battery wear is obvious.
The numbers don't add up to be profitable.

To Reproduce
It happens when the electricity price is at a rather constant low in daytime when PV production is good.

Expected behavior
EMHASS should consider the battery charge/discharge weights and avoid a forced charge/discharge when the profit is less than the battery charge/discharge weight.

Screenshots

Home Assistant installation type

• Home Assistant Core 2024.3.0 (docker)

Your hardware

• OS: Debian bookworm
• Architecture: amd64 Intel NUC

EMHASS installation type

• Docker Standalone v0.8.6

Additional context

[davidusb-geek]

Yes but hard to say why it is behaving like that.
What prediction horizon are you using? And it is a constant value or you change it in time?

[purcell-lab]

This is not the expected outcome, I have seen similar results in the past which has usually been a consequence of incorrect input values.

Can you update this issue with a screen shot of all the system optimisation charts and the optimisation table?

They will contain the full picture and will enable us to understand why you are seeing these results.

[Octofinger]

My prediction horizon is actually moving because of how energy prices are published by Nordpool (Nordic electricity exchange). At 14:00 every day, the next day is published, so from 14:00 to midnight, I use a full 24 hour prediction window, but after midnight, it's reduced by the hour. At 13:00 it's at its shortest as I only have 11 hours of energy prices left for that day, until at 14 when the next day is published again.

Today, I see the same behavior. The behavior is not changing the closer I get, I usually see it already the day before. I run the optimization every five minutes.
So at the time of writing this, I have the following results.
optim_status is optimal

Runtime parameters to naive_mpc_optim:

Passed runtime parameters: {'prediction_horizon': 15, 'soc_init': 0.42, 'soc_final': 0.5, 'def_total_hours': [], 'load_cost_forecast': [0.45, 0.36, 0.35, 0.34, 0.34, 0.34, 0.35, 0.39, 0.8, 0.87, 0.89, 0.84, 0.82, 0.76, 0.72], 'prod_price_forecast': [0.22, 0.13, 0.12, 0.11, 0.11, 0.11, 0.12, 0.16, 0.57, 0.64, 0.66, 0.61, 0.59, 0.53, 0.49], 'pv_power_forecast': [3055, 4725, 6250, 7400, 8110, 8406, 8282, 7715, 6705, 5354, 3549, 1130, 0, 0, 0, 0, 0, 0, 0, 31, 255, 585, 1032, 1902]}

Optimization charts. As you can see, consumption isn't a factor as it was very level at around 1 kWh/h the day before (I use the basic naive consumption prediction)

Optimization table:

index | P_PV | P_Load | P_grid_pos | P_grid_neg | P_grid | P_batt | SOC_opt | unit_load_cost | unit_prod_price | cost_profit | cost_fun_profit
-- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | --
2024-05-11 09:00:00+02:00 | 2632 | 510 | 0 | -5679 | -5679 | 3556 | 0.050 | 0.45 | 0.22 | 1.249 | 1.249
2024-05-11 10:00:00+02:00 | 4725 | 887 | 0 | 0 | 0 | -3837 | 0.430 | 0.36 | 0.13 | -0.000 | -0.000
2024-05-11 11:00:00+02:00 | 6250 | 473 | 0 | -9239 | -9239 | 3463 | 0.050 | 0.35 | 0.12 | 1.109 | 1.109
2024-05-11 12:00:00+02:00 | 7400 | 492 | 0 | -1707 | -1707 | -5200 | 0.565 | 0.34 | 0.11 | 0.188 | 0.188
2024-05-11 13:00:00+02:00 | 8110 | 1078 | 0 | -11496 | -11496 | 4465 | 0.075 | 0.34 | 0.11 | 1.265 | 1.265
2024-05-11 14:00:00+02:00 | 8406 | 808 | 0 | -2397 | -2397 | -5200 | 0.590 | 0.34 | 0.11 | 0.264 | 0.264
2024-05-11 15:00:00+02:00 | 8282 | 490 | 0 | -3644 | -3644 | -4147 | 1.000 | 0.35 | 0.12 | 0.437 | 0.437
2024-05-11 16:00:00+02:00 | 7715 | 838 | 0 | -6876 | -6876 | 0 | 1.000 | 0.39 | 0.16 | 1.100 | 1.100
2024-05-11 17:00:00+02:00 | 6705 | 753 | 0 | -5951 | -5951 | 0 | 1.000 | 0.80 | 0.57 | 3.392 | 3.392
2024-05-11 18:00:00+02:00 | 5354 | 493 | 0 | -4860 | -4860 | 0 | 1.000 | 0.87 | 0.64 | 3.111 | 3.111
2024-05-11 19:00:00+02:00 | 3549 | 584 | 0 | -6459 | -6459 | 3495 | 0.617 | 0.89 | 0.66 | 4.263 | 4.263
2024-05-11 20:00:00+02:00 | 1130 | 601 | 0 | 0 | 0 | -528 | 0.669 | 0.84 | 0.61 | -0.000 | -0.000
2024-05-11 21:00:00+02:00 | 0 | 729 | 0 | 0 | 0 | 729 | 0.589 | 0.82 | 0.59 | -0.000 | -0.000
2024-05-11 22:00:00+02:00 | 0 | 431 | 0 | 0 | 0 | 431 | 0.542 | 0.76 | 0.53 | -0.000 | -0.000
2024-05-11 23:00:00+02:00 | 0 | 380 | 0 | 0 | 0 | 380 | 0.500 | 0.72 | 0.49 | -0.000 | -0.000

Here's my full config if you can spot anything erroneous in there. There are probably a lot of stale unnecessary config parameters set that are no longer needed in later emhass versions, but I haven't gotten to cleaning those up yet.

# Configuration file for EMHASS

retrieve_hass_conf:
  - freq: 60 # The time step to resample retrieved data from hass in minutes
  - days_to_retrieve: 2 # We will retrieve data from now and up to days_to_retrieve days
  - var_PV: 'sensor.pv_total_dc_power' # Photovoltaic produced power sensor in Watts
  - var_load: 'sensor.template_emhass_no_var_load_2' # Household power consumption sensor in Watts (deferrable loads should be substracted)
  - load_negative: False # Set to True if the retrived load variable is negative by convention
  - set_zero_min: True # A special treatment for a minimum value saturation to zero. Values below zero are replaced by nans
  - var_replace_zero:  # A list of retrived variables that we would want  to replace nans with zeros
    - 'sensor.pv_total_dc_power'
  - var_interp:  # A list of retrived variables that we would want to interpolate nan values using linear interpolation
    - 'sensor.pv_total_dc_power'
    - 'sensor.template_emhass_no_var_load_2'
  - method_ts_round: 'first' # Set the method for timestamp rounding, options are: first, last and nearest

optim_conf:
  - set_use_battery: True # consider a battery storage
  - delta_forecast: 1 # days
  - num_def_loads: 0
  - P_deferrable_nom: [] # Watts
    #    - 900.0
  - def_total_hours: []  # hours
    #    - 1
  - def_start_timestep: 0
  - def_end_timestep: 0
  - treat_def_as_semi_cont: []  # treat this variable as semi continuous
    #    - False
  - set_def_constant: []  # set as a constant fixed value variable with just one startup for each 24h

  - weather_forecast_method: 'scrapper' # options are 'scrapper' and 'csv'
  - load_forecast_method: 'naive' # options are 'csv' to load a custom load forecast from a CSV file or 'naive' for a persistance model
#  - load_cost_forecast_method: 'list' # options are 'hp_hc_periods' for peak and non-peak hours contracts and 'csv' to load custom cost from CSV file
  - load_cost_forecast_method: 'hp_hc_periods' # options are 'hp_hc_periods' for peak and non-peak hours contracts and 'csv' to load custom cost from CSV file
#  - prod_price_forecast_method: 'list' # options are 'constant' for constant fixed value or 'csv' to load custom price forecast from a CSV file
  - list_hp_periods: # list of different tariff periods (only needed if load_cost_forecast_method='hp_hc_periods')
    - period_hp_1:
      - start: '08:00'
      - end: '09:00'
    - period_hp_2:
      - start: '17:00'
      - end: '18:00'
  - load_cost_hp: 0.1 # peak hours load cost in €/kWh (only needed if load_cost_forecast_method='hp_hc_periods')
  - load_cost_hc: 0.1 # non-peak hours load cost in €/kWh (only needed if load_cost_forecast_method='hp_hc_periods')
  - prod_price_forecast_method: 'constant' # options are 'constant' for constant fixed value or 'csv' to load custom price forecast from a CSV file
  - prod_sell_price: 0.1
  - set_total_pv_sell: False # consider that all PV power is injected to the grid (self-consumption with total sell)
  - lp_solver: 'PULP_CBC_CMD' # set the name of the linear programming solver that will be used
  - lp_solver_path: 'empty' # set the path to the LP solver
  - set_nocharge_from_grid: False # avoid battery charging from the grid
  - set_nodischarge_to_grid: False # avoid battery discharging to the grid
  - set_battery_dynamic: False # add a constraint to limit the dynamic of the battery power in power per time unit
  - battery_dynamic_max: 0.9 # maximum dynamic positive power variation in percentage of battery maximum power
  - battery_dynamic_min: -0.9 # minimum dynamic negative power variation in percentage of battery maximum power
  - weight_battery_discharge: 0.5
  - weight_battery_charge: 0.5

plant_conf:
  - P_to_grid_max: 15000 # The maximum power that can be supplied by the utility grid in Watts
  - P_from_grid_max: 15000 # The maximum power that can be supplied by the utility grid in Watts
  - module_model: # The PV module model
    - 'CSUN_Eurasia_Energy_Systems_Industry_and_Trade_CSUN295_60M'
  - inverter_model: # The PV inverter model
    - 'Fronius_International_GmbH__Fronius_Primo_5_0_1_208_240__240V_'
  - surface_tilt: # The tilt angle of your solar panels
    - 25
  - surface_azimuth: # The azimuth angle of your PV installation
    - 102
  - modules_per_string: # The number of modules per string
    - 23
  - strings_per_inverter: # The number of used strings per inverter
    - 1
  - Pd_max: 4700 # If your system has a battery (set_use_battery=True), the maximum discharge power in Watts
  - Pc_max: 5200 # If your system has a battery (set_use_battery=True), the maximum charge power in Watts
  - eta_disch: 0.95 # If your system has a battery (set_use_battery=True), the discharge efficiency
  - eta_ch: 0.95 # If your system has a battery (set_use_battery=True), the charge efficiency
  - Enom: 9600 # If your system has a battery (set_use_battery=True), the total capacity of the battery stack in Wh
  - SOCmin: 0.05 # If your system has a battery (set_use_battery=True), the minimun allowable battery state of charge
  - SOCmax: 1.0 # If your system has a battery (set_use_battery=True), the minimun allowable battery state of charge
  - SOCtarget: 0.6 # If your system has a battery (set_use_battery=True), the desired battery state of charge at the end of each optimization cycle

[purcell-lab]

• weight_battery_discharge: 0.5
• weight_battery_charge: 0.5

Have you tried with lower weights? I generally find 0.05 or 0.1 creates my desired behaviour.

[Octofinger]

• weight_battery_discharge: 0.5
• weight_battery_charge: 0.5

Have you tried with lower weights? I generally find 0.05 or 0.1 creates my desired behaviour.

Mind you that my currency is SEK. There's about 11 SEK on one EUR, so this approximates to 0.05 EUR/kWh battery weight. It goes well with my expected battery life cost. I paid roughly 60 000 SEK for the battery, and it's 80% capacity warranty is 6000 cycles, so around 10 SEK per cycle. The battery holds 10 kWh, so 1 SEK per kWh on total. I divide that in half for charge and the other half for discharge.
I don't see how lowering the battery weight would reduce emhass' preference to force charge/discharge, rather than the other way around.

But I'll set up a separate standalone EMHASS just to test different configurations and see if I can both reproduce and remove this behavior.

[Octofinger]

I think I've found a rather extreme, but still useful scenario to reproduce a highly odd behavior when using battery charge/discharge weights.
Here's a scenario with constant load, constant prices, constant PV and same start SOC and end SOC:
I send this data to dayahead-optim

'{"prediction_horizon": 23, "soc_init": 0.5, "soc_final": 0.5, "def_total_hours": [], "load_cost_forecast": [0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2], "prod_price_forecast": [0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1], "pv_power_forecast": [5000,5000,5000,5000,5000,5000,5000,5000,5000,5000,5000,5000,5000,5000,5000,5000,5000,5000,5000,5000,5000,5000,5000,5000], "load_power_forecast": [450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450]}'

First test is zero charge/discharge weights.
The result is expected where it just doesn't do much:

But, now I set 0.5 as charge/discharge weights. Now it goes mad and seems to believe there's actually a cost benefit of charging and discharging the battery:

Something's apparently very wrong with the battery weights. It's increasing the battery use, rather than the other way around.

Here's another last shot. I set battery weights to 1, so it should come at a cost of 1 SEK/kWh to use the battery.
The POST JSON has zero solar PV and only minor fluctuations in electricity prices throughout the day. Every fourth hour they go up or down by 0.2 SEK/kWh. Now, with a weight of 1 SEK/kWh, emhass should not see a profit in charging the battery and then selling it for a price difference of only 0.2 SEK/kWh (it would count as a loss of 0.8 SEK/kWh). It should just leave the battery and just buy from grid. But here's what happens:

'{"prediction_horizon": 23, "soc_init": 0.5, "soc_final": 0.5, "def_total_hours": [], "load_cost_forecast": [0.2,0.2,0.2,0.2,0.4,0.4,0.4,0.4,0.2,0.2,0.2,0.2,0.4,0.4,0.4,0.4,0.2,0.2,0.2,0.2,0.4,0.4,0.4,0.4], "prod_price_forecast": [0.1,0.1,0.1,0.1,0.3,0.3,0.3,0.3,0.1,0.1,0.1,0.1,0.3,0.3,0.3,0.3,0.1,0.1,0.1,0.1,0.3,0.3,0.3,0.3], "pv_power_forecast": [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0], "load_power_forecast": [450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450,450]}'

Tell me. Have I completely misunderstood the use of this configuration parameter or is there something wrong in how emhass uses it? I've really tried to read the code but I con't understand it well enough to debug this in the code.