The big "rebranding" effort

[sajith]

For Hysterical Raisins (TM), this repository, the Python package it produces, and the PyPI project are all named "fabrictestbed-extensions", although we often call it FABlib, FABLib, FABLIB, fablib, and so on. This naming is often confusing people who are new to FABRIC project look for "fablib" under fabric-testbed GitHub organization, they will not be able to find such a repository. I can personally attest that this very thing confused me when I was really new to FABRIC project.

The "extensions" part of the name has become questionable over time, although it might have been justifiable during the exploratory stage of this repository. This repo and Python package is perhaps more appropriately called "fabrictestbed-fablib". Being able to name this simply "fablib" would have been nice, but that name was taken by another project a long time ago.

There are several pieces to renaming, and the whole set of changes is contingent to discussion and agreement within the greater team. This issue is meant to describe the ideas and to begin the discussion.

1. Renaming the GitHub repository

This is probably the easiest part. GitHub handles renaming quite well (see docs), and encourage people to do git remote set-url origin NEW_URL on their local clones.

2. Renaming the Python package

Renaming the package should be fairly simple -- we'll just have change it in the package name in pyproject.toml and deal with the fallout. Any complication from would be the next step, namely, module re-organization.

3. Module re-organization

Changes to the way we currently organize modules are going to be a little more involved, but possible. Both package renaming and module re-organization are breaking changes, so we should proceed more deliberately, and communicate the changes clearly with fablib users.

The problem with module names are that they are currently of this form:

from fabrictestbed_extensions.fablib.fablib import FablibManager as fablib_manager

It would be much nicer if we could make it a little shorter:

from fabrictestbed_fablib import FablibManager

It is not advisable to simply rename the modules without notice though. If we do that, we will be breaking many existing notebooks when their authors upgrade their fablib versions, and giving them some unexpected busywork. So we should do this in several steps:

1. Move the important fabrictestbed_extensions.fablib code under fabrictestbed_fablib.
2. Maintain backward compatibility as much as possible by adding some fabrictestbed_extensions compatibility shims.
3. Advertise the fact that we are planning some breaking changes.
4. Give people sufficient time and maybe a rough time frame (perhaps a span of releases) regarding when we will remove the shim.
5. Print deprecation warnings about the imports that will be totally going away in the future.
6. Update all the notesbooks on jupyter-examples repo, and update fabric-docker-images repo, and roll out the changes to FABRIC-hosted JupyterHub.
7. Update the README to indicate the fact that the project is being renamed (and that the old one will stop receiving updates) and that the API is changing, to alert readers on GitHub and PyPI.
8. Finally, remove backward compatibility modules at a convenient time.
9. Live happily ever after.

Let us use #164 to track both package renaming and the first two steps of module re-organization.

4. The PyPI projects

The PyPI project "fabrictestbed-fablib" exists at the time of writing.

I went ahead and claimed the name fabrictestbed-fablib on both PyPI and on TestPyPI by uploading a bogus rc1 package. I will invite others too, in order to achieve parity with fabrictestbed-extensions and in order to improve bus factor.

People will continue to be able to install the already-published versions of fabrictestbed-extensions. We can probably yank them from PyPI, but I am not sure if this is necessary or even advisable. They will not continue to work as expected at all times in the future anyway, because the testbed against which it is meant to be used keeps evolving.

[ibaldin]

Can you provide examples of shims and deprecation warning placement?

[sajith]

I am working on that at #172. Basically, we will have a directory structure like this:

src/
├── fabrictestbed_extensions  # The "shim" modules live here
│   ├── fablib  
│   │   ├── component.py
│   │   ├── fablib.py
│   │   ├── facility_port.py
│   │   ├── __init__.py
│   │   ├── interface.py
│   │   ├── network_service.py
│   │   └── node.py
│   └ [...]
└── fabrictestbed_fablib      # The "real" modules live here
    ├── abc_fablib.py
    ├── component.py
    ├── fablib.py
    ├── facility_port.py
    ├── __init__.py
    ├── interface.py
    ├── network_service.py
    └── node.py

We will move all the previous fabrictestbed_extensions.fablib.* modules under fabrictestbed_fablib. And then under fabrictestbed_extensions.fablib, we will place shim modules that basically re-export fabrictestbed_fablib modules but with deprecation warnings. As an example, here's fabrictestbed_extensions.fablib.fablib.py:

import warnings

from fabrictestbed_fablib.fablib import FablibManager

warnings.warn(
    "fabrictestbed_extensions.fablib.fablib module is deprecated;"
    " please import fabrictestbed_fablib.fablib instead",
    DeprecationWarning,
    stacklevel=2,
)

__all__ = [FablibManager]

The warnings currently look like this:

>>> from fabrictestbed_extensions.fablib.fablib import FablibManager
<stdin>:1: DeprecationWarning: fabrictestbed_extensions module is deprecated
<stdin>:1: DeprecationWarning: fabrictestbed_extensions.fablib.fablib module is deprecated; please import fabrictestbed_fablib.fablib instead

When importing fabrictestbed_fablib:

>>> from fabrictestbed_fablib.fablib import FablibManager   # no warnings

The two top-level modules means that we won't be able to use Flit for packaging. Flit is stricter about packaging, and will include only the top-level module that matches the project name in wheel packages. We can use setuptools as the build backend though, by changing two lines in pyproject.toml.