utilities: Convert juju base into Ubuntu series

[fnordahl]

Juju 3.x replaced the series status key with a base key that consists of Distribution type and version number.

To avoid maintenance burden we add a Launchpad module that implements functions to look up available Ubuntu series data.

Update the get_machine_series helper function to determine Ubuntu series from base when no series key is available.

[codecov]

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (00af63d) 89.14% compared to head (76dfea8) 89.15%.

Files	Patch %	Lines
zaza/utilities/juju.py	86.66%	2 Missing ⚠️

Additional details and impacted files

@@               Coverage Diff               @@
##           libjuju-3.1     #622      +/-   ##
===============================================
+ Coverage        89.14%   89.15%   +0.01%     
===============================================
  Files               44       45       +1     
  Lines             4689     4713      +24     
===============================================
+ Hits              4180     4202      +22     
- Misses             509      511       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[lmlg]

LGTM overall, just a couple nitpicks

[lmlg]

I'm frankly not really fond of using HTTP requests to get the Ubuntu version mapping, simple as it is, since it restraints somewhat the environments in which this can be run, as well as placing a dependency on an external asset. OTOH, alternatives like including charmhelpers aren't particularly endearing either. At the very least, I would include an escape valve, like an environment variable that allows one to bypass the need for internet access.

[fnordahl]

Indeed this part is tricky, and I understand your reservations for an external network dependency. I guess that's why the previous attempts at this has ended up hard coding with the perpetual per-release maintenance burden that entails.

I strongly dislike anything that invites manual work, and IMHO it is worth spending quite a bit of extra time on development to avoid it.

What about a fallback to a on-disk file with a documented format and perhaps a helpful error message in case this request fails?

[lmlg]

Yes, I'd be onboard with that and it's essentially what I was hinting at when I mentioned the environment variable thingy (I'm opened to any other solution as well, like an optional parameter to the base function).

[ajkavanagh]

I was reading through both your comments (@fnordahl and @lmlg) and just wanted to add some thoughts for your consideration:

I really like @lmlg reaction to a code change that adds a new requirement; in this case requiring network access to launchpad.

I also really like @fnordahl constant hunt to reduce manual work and automate where possible, particular around things that would need maintaining year-to-year as new releases come into being. This is the kind of busywork we all want to avoid.

So, in this case, how would/should we proceed?

So my thoughts revolve around keeping it simple (less code = less maintenance/testing requirements), which is also an antidote to gold-plating, but making sure it is still appropriate for it's environment. In this case, this is zaza code, which is exclusively a test framework, and does run in a variety of places, but all (I think?) are in non-air-gapped locations. Some might require proxies, but I don't think we've hit that for launchpad.

i.e. I would probably favour @fnordahl 's solution of simplicity over adding more complexity in ENV variable that would have to be tested, add documentation on how to use it, for a use-case that hasn't yet come up. Maybe add a TODO comment that basically says "add a ENV variable here if an air-gapped solution is required".

Maintaining an additional text file of releases feels very counter-productive as it's entirely what @fnordahl is seeking to avoid!

Anyway, I hope this helps! Just an 3rd party's thoughts after reading your comments. Thanks.

[wolsen]

note: an alternative implementation is simply to calculate the base from the distro-info output. The distro-info is updated across releases.

The following does the trick and is future-proof as long as:

1. The LTS structure doesn't change
2. All releases continue to ship twice per year
3. The distro-info continues to be updated

import re
import subprocess
from typing import List


BASE_REGEX = r'^(?:[\w+]*@)?(?P<year>[\d]{1,2})\.(?P<month>[\d]{2})$'


def _get_known_series() -> List[str]:
    """Returns the list of the known Ubuntu series from the distro-info.

    Returns a list of all known ubuntu series from the distro-info command.
    """
    known_series = subprocess.check_output(['distro-info', '--all']).decode('utf-8').strip().split('\n')
    return known_series


def get_base_from_series(series: str) -> str:
    """Returns the Ubuntu base format from the series name.

    Converts the series name to an Ubuntu base, e.g. focal => 20.04,
    jammy => 22.04, etc.

    :param series: the Ubuntu series name.
    :raises: ValueError if the series is not a known Ubuntu release name
    """
    known_series = _get_known_series()
    series = series.strip().lower()

    index = known_series.index(series)
    if index < 0:
        raise ValueError(f"{series} is not a known Ubuntu series name")

    # Only Ubuntu release to ever ship late (first LTS had its challenges)
    if series == 'dapper':
        return "[email protected]"

    year = (index >> 1) + (index % 2) + 4
    month = '04' if (index % 2) == 1 else '10'
    return f"ubuntu@{year}.{month}"


def get_series_from_base(base: str) -> str:
    """Returns the Ubuntu series name from the base format.

    Converts the base name to an Ubuntu series name, e.g. 20.04 => focal,
    22.04 => jammy, etc.

    :param base: the base name that includes the distro release and the channel.
                 May optionally include a distro@ prefix.
    """
    known_series = _get_known_series()

    match = re.match(BASE_REGEX, base)
    if not match:
        raise ValueError(f"{base} is not of the correct format. Expected to be of form [distro@]YY.MM")

    year = int(match.group("year"))
    month = int(match.group("month"))

    offset = -1 if month <= 6 else 0
    # 2 releases per year, starting in 2004. First release was in the fall
    # so we account for that with the offset.
    index = (year - 4) * 2 + offset
    return known_series[index]


def is_lts(base_or_series: str) -> bool:
    """Returns True iff the base or series specified is an LTS release.

    :param base_or_series: either a base or series.
    """
    match = re.match(BASE_REGEX, base_or_series)
    if not match:
        # Looks like we have a series. Convert it to base.
        match = re.match(BASE_REGEX, get_base_from_series(base_or_series))

    year = int(match.group("year"))
    month = int(match.group("month"))
    # LTS releases are only in even numbered years.
    if int(year) % 2 > 0:
        return False

    # LTS release is only in the first half of the year.
    return int(month) <= 6

[fnordahl]

Nice alternative indeed, and parsing the output ourselves is actually a plus as the distro-info Python package is not particularly compatible with virtual environments (not really pip installable).

A potential downside could be that whether it works or not depends on the test executor being up to date, as this information comes from a system deb package. From previous experience those have a tendency to end up running pretty old versions from time to time.

I guess we could run the check on one of the deployed nodes to ensure up to date data.

[freyes]

I will close this PR since this patch found its way into master as part of the juju-3.x support.

dfbdbec