Doc tidy-ups, automations to monitor predbat and givtcp, custom load_…

…today (#415)

* remove extraneous comment
* Add creating own house load sensor. Misc tidy-ups
* Formatting tidy-up's
* Grammar tweaks
* Formatting and typo tidy-up's
* Add automation scripts to monitor givtcp & predbat
* Expanded development guide
* Added details of using terminal window
* Add FAQ for filtering predbat.plan_html warnings

---------

Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>

.cspell/custom-dictionary-workspace.txt

@@ -112,6 +112,7 @@ timestr
 timezone
 Trefor
 treforsiphone
+unstaged
 Wallbox
 xaxis
 yaxis

apps/predbat/predbat.py

@@ -7959,7 +7959,7 @@ def calculate_plan(self, recompute=True):
                 save="best",
                 end_record=self.end_record,
             )
-            # round charge_limit_best (kWh) to 2 decimal places and discharge_limits_best (percentage) to nearest whole number  gc
+            # round charge_limit_best (kWh) to 2 decimal places and discharge_limits_best (percentage) to nearest whole number
             self.charge_limit_best = [self.dp2(elem) for elem in self.charge_limit_best]
             self.discharge_limits_best = [self.dp0(elem) for elem in self.discharge_limits_best]
 

docs/configuration-guide.md

@@ -8,28 +8,28 @@ charging. You might want to also set to the car charging plan so you can predict
 
 You should try to tune **inverter_loss**, **battery_loss** and **battery_loss_discharge** to the correct % loss for your system
 in order to get more accurate predictions. Around 4% for each is good for a hybrid inverter. Also set **inverter_hybrid** to
-True or False depending on if you have a Hybrid or AC Coupled battery.
+True or False depending on if you have a Hybrid or AC-Coupled battery.
 
 The setting **input_number.metric_battery_cycle** (_expert mode_) can be used to put a cost on using your battery for charging
 and discharging. In theory if you think your battery will last say 6000 complete cycles and cost you £4000 and is 9.5kWh then
 each cycle is 19kWh and so the cost is £4000 / 19 / 6000 = 3.5p. If you configure this number higher then more expensive plans
-will be selected which avoid charging and discharging your battery as much. The default is 2p but can be set to 0 if you want
+will be selected which avoids charging and discharging your battery as much. The default is 2p but can be set to 0 if you want
 to turn this off. Note that the cycle cost will not be included in the cost predictions, just taken into account in the planning
-stage. _Note: Setting this to a non-zero zero will increase your daily cost, but will reduce your home battery usage._
+stage. _Note: Setting this to a non-zero value will increase your daily cost, but will reduce your home battery usage._
 
 Cloud coverage is modelled by using difference between PV and PV10 is used to work out a cloud factor, this modulates the PV
-output predictions up and down accordingly as if there was passing clouds. This can have an impact on planning, especially for
+output predictions up and down accordingly as if there were passing clouds. This can have an impact on planning, especially for
 things like freeze charging which could assume the PV will cover the house load but it might not due to clouds.
 
 Below is a guide to some of the tariff options, in theory most tariffs will work out of the box but still it's worth reviewing your settings.
 
 ## Fixed daily rates
 
-- In this case you will just be predicting the battery levels, no charging or discharging is required although it won't hurt if you leave these options enabled.
+With a fixed daily rate tariff you will just be predicting the battery levels, no charging or discharging is required although it won't hurt if you leave these options enabled.
 
 ## Cheap night rate with bad export rate (e.g. Octopus Go, Economy 7 etc)
 
-- In this scenario you will want to charge overnight based on the next days solar forecast.
+In this scenario you will want to charge overnight based on the next day's solar forecast.
 
 Recommended settings - these must be changed in Home Assistant once Predbat is running:
 
@@ -50,7 +50,7 @@ metric_min_improvement - 0         # Charge less if it's cost neutral
 
 ## Cheap night rate, with a good export rate (e.g. Intelligent Octopus with Octopus Outgoing)
 
-Follow the instructions from Cheap Night rate above, but also you will want to have automatic discharge when the export rates are profitable.
+Follow the instructions from the _Cheap Night rate_ above, but also you will also want to have automatic discharge occurring when the export rates are profitable.
 
 ```yaml
 calculate_best_charge - True       # You want the tool to calculate charging

docs/creating-charts.md

@@ -3,9 +3,12 @@
 To create the fancy charts:
 
 - Install Apex Charts [https://github.com/RomRider/apexcharts-card](https://github.com/RomRider/apexcharts-card)
-- There are multiple charts, for each section of the example file create a new apexcharts card and copy the YAML into it
+- There are multiple charts, for each section of the [example chart yaml file](https://github.com/springfall2008/batpred/blob/main/example_chart.yml), create a new apexcharts card
+and copy the YAML into it
 - Customise as you like
 
+See the [video guides](video-guides.md) for a walkthrough of what the different charts show.
+
 Example charts:
 
 ![image](https://github.com/springfall2008/batpred/assets/48591903/28f29756-2502-4079-9c75-398e8a1a0699)

docs/customisation.md

@@ -110,7 +110,7 @@ if **car_charging_planned** is set correctly in apps.yaml to detect your car bei
 ## Calculation options
 
 **switch.predbat_calculate_best_charge** If set to False then charge windows will not be calculated and the
-default inverter settings are used, when True predbat will decide the charge window automatically.
+default inverter settings are used, when True Predbat will decide the charge window automatically.
 
 **switch.predbat_calculate_best_discharge** If set to False then force discharge windows will not be calculated,
 when True they will be calculated. Default is True.

docs/developing.md

@@ -39,8 +39,8 @@ This includes:
     can connect to a copy of VS Code installed on your own
     machine
 - A terminal where you can run commands against your codebase
-e.g. running `mkdocs`, linting files etc.
-- Software like `mkdocs` (and other software we may include)
+e.g. running `pre-commit` to run the automatic code quality checks, linting files, etc.
+- Software like `mkdocs` to re-generate the documentation files (and other software we may include)
 pre-installed in it
 
 The Codespaces environment is entirely separate from your HA
@@ -82,7 +82,7 @@ Once you start your Codespaces environment, it'll take a minute to
 create a VM for you, and to install the software we've asked it to
 install in there. It will also clone your repository and chosen
 branch into it for you, and the VM will be authenticated with GitHub
-so you can commit and push straight back to GitHub.
+so you can commit and push straight back to your fork of Predbat on GitHub.
 
 You can choose between running the IDE in the browser, or having
 your local installation of VS Code connect to the environment that GitHub
@@ -168,7 +168,9 @@ as what works on GitHub READMEs, for example, does not
 necessarily work with the published documentation site
 (which follows a more strict Markdown standard).
 
-To run the live preview, run `mkdocs serve` - this will cause `mkdocs` to build a
+There are a number of terminal commands that you can use in the Codespaces environment. Open a terminal window in Codespaces by choosing menu > Terminal > New Terminal.
+
+To run the live preview, enter `mkdocs serve` in the terminal window - this will cause `mkdocs` to build a
 local temporary version of the documentation site,
 and to temporarily publish it on port 8000 - it will
 show the link where you can access the documentation.
@@ -238,7 +240,7 @@ installed automatically. You can run it manually, or automatically.
 
 Running `pre-commit` manually:
 
-- Running `pre-commit` will run all the checks against any files that you
+- In a terminal window, running `pre-commit` will run all the checks against any files that you
 have modified and staged.
 
 - Alternatively, running `pre-commit run --all-files` will run all the checks
@@ -249,14 +251,14 @@ those changes will not be staged. You will need to stage those changes too
 before committing.
 
 - You may notice `pre-commit` mentioning about stashing changes - this is
-because when it runs, any changes that aren't stages are stashed (saved
+because when it runs, any changes that aren't staged are stashed (saved
 away temporarily) so it runs against only the staged changes;
 after it has run, it pulls back those stashed changes, so they appear
 again (still unstaged).
 
 Running `pre-commit` automatically:
 
-- If you run `pre-commit install` it will install a pre-commit hook -
+- If you run `pre-commit install` in a terminal window it will install a pre-commit hook -
 this is a file which tells `git` to run some code each type you do a
 particular action (a pre-commit hook runs at the start of processing
 a commit, but there are other hooks e.g. pre-push).

docs/energy-rates.md

@@ -2,17 +2,19 @@
 
 ## Octopus Energy Plugin
 
-- If you want to use real pricing data and have Octopus Energy then ensure you have the Octopus Energy plugin installed and working
+If you want to use real pricing data and have Octopus Energy then ensure you have the Octopus Energy plugin installed and working
 ([https://github.com/BottlecapDave/HomeAssistant-OctopusEnergy](https://github.com/BottlecapDave/HomeAssistant-OctopusEnergy/))
-- The Octopus Energy Plugin also provides support for Intelligent Octopus charging to support car charging
-- When **octopus_intelligent_charging** is True and you are on Intelligent Octopus import the car charging plan will be
+
+The Octopus Energy Plugin also provides support for Intelligent Octopus charging to support car charging.
+
+When **octopus_intelligent_charging** is True and you are on Intelligent Octopus import the car charging plan will be
 extracted from Octopus and used for Predbat to plan, and it may charge the home battery using these slots also.
 
 ### Octopus Saving sessions
 
 For Predbat to automatically manage saving sessions you will need to make sure that **octopus_saving_session** is set
 in apps.yaml to point to the saving session binary sensor supported by the Octopus Energy plugin (see template apps.yaml
-for the default name)
+for the default name).
 
 When a saving session is available it will be automatically joined by Predbat and then should appear as a joined session
 within the next 30 minutes.
@@ -24,35 +26,33 @@ using the **octopus_saving_session_octopoints_per_penny** configuration item in
 As the saving session import and export rates are very high compared to normal you would expect Predbat to export during the entire
 period if the battery is large amount, a pre-charge may happen at some point during the day to maintain the right level for the session.
 
-If you are using expert mode, for best results: **switch.predbat_combine_charge_slots** (_expert mode_) is turned off.
+If you are using expert mode, for best results: **switch.predbat_combine_charge_slots** (_expert mode_) should be turned off.
 Set **input_number.rate_low_threshold** (_expert mode_) and **input_number.rate_high_threshold** (_expert mode_) to 0 for automatic mode.
 
 For forced export you need to ensure that **switch.predbat_calculate_best_discharge** is enabled and that **switch.predbat_set_discharge_freeze_only** is disabled.
 If you do not have an export tariff then forced export will not apply.
 
 ## Rate bands
 
-- you can configure your rate bands (assuming they repeat) using rates_import/rates_export (see below)
+You can manually configure your rate bands (assuming they repeat) using rates_import/rates_export (see below).
 
 ## Rate offsets
 
-- Note that you can tune future unknown energy rates by adjusting **input_number.predbat_metric_future_rate_offset_import**
+Note that you can tune future unknown energy rates by adjusting **input_number.predbat_metric_future_rate_offset_import**
 (_expert mode_) and **input_number.predbat_metric_future_rate_offset_export** (_expert mode_) inside Home Assistant
-to set the predicted offset for future unknown rates
+to set the predicted offset for future unknown rates.
 
 ## Future Agile energy rates
 
 In the energy market it's possible to calculate the Octopus Agile rates from around 10am UK time using public data, you can
-enable this in apps.yaml for Import, Export or both. This will approximate next days rates based on the spot prices.
-The approximation is only used until the real rates are released around 4pm
+enable this in apps.yaml for Import, Export or both. This will approximate next day's rates based on the spot prices.
+The approximation is only used until the real Octopus Agile rates are released around 4pm.
 
 CAUTION: You may violate the terms and conditions of the Nordpool site if you use this data and as such the authors of
 Predbat accept no responsibility for any violations:
 
 <https://www.nordpoolgroup.com/en/About-us/terms-and-conditions-for-useofwebsite/>
 
-``
-
 ## Nordpool market energy rates
 
 ```yaml

docs/faq.md

@@ -1,6 +1,6 @@
 # FAQ
 
-## I've installed Batpred but I don't see the correct entities
+## I've installed Predbat but I don't see the correct entities
 
 - First look at AppDaemon.log (can be found in the list of log files in the System/Log area of the GUI).
 See if any errors are warnings are found. If you see an error it's likely something is configured wrongly,
@@ -11,11 +11,11 @@ The default solcast sensor names maybe wrong, you might need to update the apps.
 
 ## Why is my predicted charge % higher or lower than I might expect?
 
-- Batpred is based on costing, so it will try to save you money. If you have the PV 10% option enabled it will also
+- Predbat is based on costing, so it will try to save you money. If you have the PV 10% option enabled it will also
 take into account the more worse case scenario and how often it might happen, so if the forecast is a bit unreliable
 it's better to charge more and not risk getting stung importing.
 - Have you checked your energy rates for import and export are correct, maybe check the rates graph and confirm.
-If you do something like have export>import then Batpred will try to export as much as possible.
+If you do something like have export>import then Predbat will try to export as much as possible.
 - Have you tuned Solcast to match your output accurately?
 - Have you tuned the **metric_min_improvement**, **best_soc_min** and **best_soc_keep settings**?
 - Do you have predicted car charging during the time period?
@@ -37,13 +37,33 @@ a script that changes the reserve %, this will cause problems - please disable o
 
 ## It's all running but I'm not getting very good results
 
-- You might want to tune **best_soc_keep** to set a minimum target battery level, e.g. I use 2.0 (for 2kWh, which is just over 20% on a 9.5kWh battery)
-- Have a read of the user configuration guide above depending on your tariff different settings maybe required
-- Check your solar production is well calibrated (you can compare solcast vs actually in home assistant energy tab or on the GivEnergy portal)
+- You might want to tune **best_soc_keep** to set a minimum target battery level, e.g. I use 2.0 (for 2kWh, which is just over 20% on a 9.5kWh battery).
+If you set **best_soc_keep** too high then predbat could need to charge the battery in unfavourable import rates, so try to set it to a fairly low value,
+especially if you have a small battery. If you set it to zero then predbat may not charge at all, so use 0.1 as a minimum.
+- Have a read of the [energy rates configuration guide](energy-rates.md) as depending on your tariff different settings maybe required
+- Check your solar production is well calibrated (you can compare solcast vs actually in the Home Assistant energy tab or on the GivEnergy portal)
 - Make sure your inverter max AC rate has been set correctly
 - If you have an EV that you charge then you will want some sort of car charging sensor or use the basic car charging hold feature or your load predictions maybe unreliable
 - Do you have a solar diverter? If so maybe you want to try using the IBoost model settings.
 - Perhaps set up the calibration chart and let it run for 24 hours to see how things line up
 - If your export slots are too small compared to expected check your inverter_limit is set correctly (see below)
 
-If you are still having trouble feel free to raise a ticket for support to post on the [GivTCP Facebook group](https://www.facebook.com/groups/615579009972782).
+## Predbat is causing warning messages in the Home Assistant Core log
+
+- If you have a large **input_number.predbat_forecast_plan_hours** then you may see warning
+messages in the Home Assistant Core log about the size of the predbat.plan_html entity.
+This is just a warning, the entity isn't stored in the database, but you can suppress it by adding the following
+to your configuration.yaml:
+
+```yaml
+# Filter out 'message too large' warnings from Predbat
+logger:
+  default: warning
+  filters:
+      - "State attributes for predbat.plan_html exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored"
+```
+
+## My question isn't covered here
+
+If you are still having trouble feel free to raise a [Github ticket](https://github.com/springfall2008/batpred/issues) for support,
+or post on the [GivTCP Facebook group](https://www.facebook.com/groups/615579009972782).