FablibManager.list_sites(filter=..) may be broken

[sajith]

We have an example fields=['Name','ConnectX-5 Available', 'NVMe Total']) in the API docs of FablibManager.list_sites().

https://fabric-fablib.readthedocs.io/en/latest/fablib.html#fabrictestbed_extensions.fablib.fablib.FablibManager.list_sites

Passing that example fields does not work when output argument of list_sites() is set to text or pandas:

$ python
Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from fabrictestbed_extensions.fablib.fablib import FablibManager
>>> fablib = FablibManager()
>>> fablib.list_sites(fields=['Name','ConnectX-5 Available', 'NVMe Total'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "fabrictestbed-extensions/fabrictestbed_extensions/fablib/fablib.py", line 1074, in list_sites
    ).list_sites(
      ^^^^^^^^^^^
  File "fabrictestbed-extensions/fabrictestbed_extensions/fablib/resources.py", line 961, in list_sites
    return self.get_fablib_manager().list_table(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "fabrictestbed-extensions/fabrictestbed_extensions/fablib/fablib.py", line 2341, in list_table
    table = self.create_list_table(data, fields=fields)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "fabrictestbed-extensions/fabrictestbed_extensions/fablib/fablib.py", line 2366, in create_list_table
    row.append(entry[field])
               ~~~~~^^^^^^^
KeyError: 'Name'

This is from a quick test in a Python REPL and a Jupyter notebook.

Further, when output is set to json, the returned JSON has all the fields, not just the fields that are asked for. That may be intentional, but it still is surprising behavior.

[sajith]

Turned out that filter_function also does not work as documented:

>>> fablib.list_sites(filter_function=lambda s: s['ConnectX-5 Available'] > 3 and s['NVMe Available'] <= 10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "fabrictestbed_extensions/fablib/fablib.py", line 1074, in list_sites
    ).list_sites(
      ^^^^^^^^^^^
  File "fabrictestbed_extensions/fablib/resources.py", line 961, in list_sites
    return self.get_fablib_manager().list_table(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "fabrictestbed_extensions/fablib/fablib.py", line 2319, in list_table
    data = list(filter(filter_function, data))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<stdin>", line 1, in <lambda>
KeyError: 'ConnectX-5 Available'

[sajith]

This will work:

fablib.list_sites(
    filter_function=lambda s: s["nic_connectx_5_available"] > 1,
    fields=["name", "nic_connectx_5_available", "nvme_available"],
)

However:

• That will "leak" the "non-pretty" names to fablib users.
• That is probably not the right UX, because you would expect to be able to filter/list by names on the column header.