Binder integration not working

[davidwagner]

The Binder integration is not working. For instance, go to https://inferentialthinking.com/chapters/01/3/1/Literary_Characters.html, hover over the spaceship icon, and click on Binder. This attempts to launch a Binder notebook, but everything times out and eventually (after a long delay) fails. Detailed Binder error logs below:

Found built image, launching...
Launching server...
Server requested
2023-11-15T20:37:28.001334Z [Normal] Successfully assigned ovh2/jupyter-data-2d8-2dtextbook-2d7o2lotw1 to user-202211a-node-a8b748
2023-11-15T20:37:28Z [Normal] Container image "jupyterhub/mybinder.org-tc-init:2020.12.4-0.dev.git.5363.hbea79967" already present on machine
2023-11-15T20:37:28Z [Normal] Created container tc-init
2023-11-15T20:37:29Z [Normal] Started container tc-init
2023-11-15T20:37:30Z [Normal] Pulling image "2lmrrh8f.gra7.container-registry.ovh.net/mybinder-builds/r2d-g5b5b759data-2d8-2dtextbook-a22c49:55cf34960f5b4cb2fc58ee730e079b64fdc2c56f"
Launch attempt 1 failed, retrying...
Server requested
2023-11-15T20:39:50.595880Z [Normal] Successfully assigned ovh2/jupyter-data-2d8-2dtextbook-2dyxost78l to user-202211a-node-a8b748
2023-11-15T20:39:51Z [Normal] Container image "jupyterhub/mybinder.org-tc-init:2020.12.4-0.dev.git.5363.hbea79967" already present on machine
2023-11-15T20:39:51Z [Normal] Created container tc-init
2023-11-15T20:39:51Z [Normal] Started container tc-init
2023-11-15T20:39:53Z [Normal] Pulling image "2lmrrh8f.gra7.container-registry.ovh.net/mybinder-builds/r2d-g5b5b759data-2d8-2dtextbook-a22c49:55cf34960f5b4cb2fc58ee730e079b64fdc2c56f"
Launch attempt 2 failed, retrying...
Server requested
2023-11-15T20:45:44.593565Z [Normal] Successfully assigned ovh2/jupyter-data-2d8-2dtextbook-2dtksko6dg to user-202211a-node-5eb9bc
2023-11-15T20:45:45Z [Normal] Container image "jupyterhub/mybinder.org-tc-init:2020.12.4-0.dev.git.5363.hbea79967" already present on machine
2023-11-15T20:45:45Z [Normal] Created container tc-init
2023-11-15T20:45:45Z [Normal] Started container tc-init
2023-11-15T20:45:47Z [Normal] Pulling image "2lmrrh8f.gra7.container-registry.ovh.net/mybinder-builds/r2d-g5b5b759data-2d8-2dtextbook-a22c49:55cf34960f5b4cb2fc58ee730e079b64fdc2c56f"
Spawn failed: Timeout
Launch attempt 3 failed, retrying...
Server requested
2023-11-15T20:51:02.935382Z [Normal] Successfully assigned ovh2/jupyter-data-2d8-2dtextbook-2dxul7lrsv to user-202211a-node-5eb9bc
2023-11-15T20:51:03Z [Normal] Container image "jupyterhub/mybinder.org-tc-init:2020.12.4-0.dev.git.5363.hbea79967" already present on machine
2023-11-15T20:51:03Z [Normal] Created container tc-init
2023-11-15T20:51:04Z [Normal] Started container tc-init
2023-11-15T20:51:04Z [Normal] Pulling image "2lmrrh8f.gra7.container-registry.ovh.net/mybinder-builds/r2d-g5b5b759data-2d8-2dtextbook-a22c49:55cf34960f5b4cb2fc58ee730e079b64fdc2c56f"
Failed to launch image 2lmrrh8f.gra7.container-registry.ovh.net/mybinder-builds/r2d-g5b5b759data-2d8-2dtextbook-a22c49:55cf34960f5b4cb2fc58ee730e079b64fdc2c56f

[davidwagner]

I suspect this is related to Binder's transition from classic Jupyter notebooks to Jupyter Lab, and the change in URL structure on binder.org. Hypotheses: 1) I am guessing that one easy/pragmatic solution might be to migrate to Jupyter Lab, i.e., change urlpath=tree/ to labpath=. 2) It's possible we might be able to solve this by using the latest version of nbinteract in .binder/requirements.txt instead of pinning to an older version.

Some testing:

• https://mybinder.org/v2/gh/data-8/textbook/main?urlpath=tree/chapters/01/3/1/Literary_Characters.ipynb (current spaceship link in textbook) - 500 Internal Server Error
• https://mybinder.org/v2/gh/data-8/textbook/HEAD?labpath=chapters%2F01%2F3%2F1%2FLiterary_Characters.ipynb (link generated by mybinder.org) - Successfully launches notebook in Jupyter Lab
• https://mybinder.org/v2/gh/data-8/textbook/HEAD?labpath=tree%2Fchapters%2F01%2F3%2F1%2FLiterary_Characters.ipynb - Opens Jupyter Lab launcher but not the notebook
• https://mybinder.org/v2/gh/data-8/textbook/HEAD?filepath=chapters%2F01%2F3%2F1%2FLiterary_Characters.ipynb - 500 Internal Server Error
• https://mybinder.org/v2/gh/data-8/textbook/HEAD?filepath=tree%2Fchapters%2F01%2F3%2F1%2FLiterary_Characters.ipynb - 404 Not Found
• https://mybinder.org/v2/gh/data-8/textbook/main?urlpath=/tree/chapters/01/3/1/Literary_Characters.ipynb - 500 Internal Server Error
• https://mybinder.org/v2/gh/data-8/textbook/main?urlpath=/chapters/01/3/1/Literary_Characters.ipynb - 404 Not Found
• https://mybinder.org/v2/gh/data-8/textbook/main?urlpath=/tree/ - successfully launches a classic-style Jupyter Notebook showing the root of the Github repo; when I navigate to and manually launch the notebook chapters/01/3/1/Literary_Character.ipynb from there, I get a 500 Internal Server Error
• https://mybinder.org/v2/gh/data-8/textbook/main?urlpath=/notebooks/chapters/01/3/1/Literary_Characters.ipynb - 500 Internal Server Error

When I open https://mybinder.org/v2/gh/data-8/textbook/main?urlpath=/tree/ and manually navigate to and launch the notebook, then open a terminal and view .jupyter-server-log.txt, I see the following excerpt in the logs:

[E 2023-11-16 07:59:36.727 ServerApp] Uncaught exception GET /binder/jupyter/user/data-8-textbook-iynf9ktf/notebooks/chapters/01/3/1/Literary_Characters.ipynb (23.93.106.121)
    HTTPServerRequest(protocol='http', host='notebooks.gesis.org', method='GET', uri='/binder/jupyter/user/data-8-textbook-iynf9ktf/notebooks/chapters/01/3/1/Literary_Characters.ipynb', version='HTTP/1.1', remote_ip='23.93.106.121')
    Traceback (most recent call last):
      File "/srv/conda/envs/notebook/lib/python3.10/site-packages/tornado/web.py", line 1786, in _execute
        result = await result
      File "/srv/conda/envs/notebook/lib/python3.10/site-packages/tornado/gen.py", line 786, in run
        yielded = self.gen.send(value)
      File "/srv/conda/envs/notebook/lib/python3.10/site-packages/nbclassic/notebook/handlers.py", line 101, in get
        self.write(self.render_template('notebook.html',
      File "/srv/conda/envs/notebook/lib/python3.10/site-packages/jupyter_server/base/handlers.py", line 544, in render_template
        return template.render(**ns)
      File "/srv/conda/envs/notebook/lib/python3.10/site-packages/jinja2/environment.py", line 1301, in render
        self.environment.handle_exception()
      File "/srv/conda/envs/notebook/lib/python3.10/site-packages/jinja2/environment.py", line 936, in handle_exception
        raise rewrite_traceback_stack(source=source)
      File "/srv/conda/envs/notebook/lib/python3.10/site-packages/nbclassic/templates/notebook.html", line 1, in top-level template code
        {% extends "page.html" %}
      File "/etc/jupyter/templates/page.html", line 1, in top-level template code
        {% extends "templates/page.html" %}
      File "/srv/conda/envs/notebook/lib/python3.10/site-packages/nbclassic/templates/page.html", line 190, in top-level template code
        {% block header %}
      File "/srv/conda/envs/notebook/lib/python3.10/site-packages/nbclassic/templates/notebook.html", line 115, in block 'header'
        {% for exporter in get_frontend_exporters() %}
      File "/srv/conda/envs/notebook/lib/python3.10/site-packages/nbclassic/notebook/handlers.py", line 46, in get_frontend_exporters
        for name in get_export_names():
      File "/srv/conda/envs/notebook/lib/python3.10/site-packages/nbconvert/exporters/base.py", line 151, in get_export_names
        e = get_exporter(exporter_name)(config=config)
      File "/srv/conda/envs/notebook/lib/python3.10/site-packages/nbconvert/exporters/base.py", line 111, in get_exporter
        if getattr(exporter(config=config), "enabled", True):
      File "/srv/conda/envs/notebook/lib/python3.10/site-packages/nbinteract/exporter.py", line 45, in __init__
        self.template_path.append(os.path.dirname(__file__))
    AttributeError: 'InteractExporter' object has no attribute 'template_path'. Did you mean: 'template_paths'?
[E 2023-11-16 07:59:36.750 NotebookApp] {
      "Host": "notebooks.gesis.org",
      "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
      "Referer": "https://notebooks.gesis.org/binder/jupyter/user/data-8-textbook-iynf9ktf/tree/chapters/01/3/1",
      "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/119.0"
    }
[E 2023-11-16 07:59:36.750 NotebookApp] 500 GET /binder/jupyter/user/data-8-textbook-iynf9ktf/notebooks/chapters/01/3/1/Literary_Characters.ipynb (23.93.106.121) 1887.67ms referer=https://notebooks.gesis.org/binder/jupyter/user/data-8-textbook-iynf9ktf/tree/chapters/01/3/1

I don't know how to interpret that or understand why that is failing. Maybe we need to update nbinteract? I notice that .binder/requirements.txt is older and contains nbinteract==0.0.12, and the latest released version of nbinteract is 0.2.6. Could pinning to an old version of nbinteract be causing problems? Searching around a bit, I found SamLau95/nbinteract#148, which seems to suggest that nbconvert 6.0 got rid of template_path, and it looks like nbinteract was changed to avoid using template_path. So perhaps we need to be using the latest version of nbinteract, to fix this problem.

[davidwagner]

Well, no joy in trying to use the latest version of nbinteract. https://mybinder.org/v2/gh/data-8/textbook/latest_nbinteract?urlpath=tree/chapters/01/3/1/Literary_Characters.ipynb consistently times out, and I can't figure out how to access any error logs to check why.

So, back to migrating to Jupyter Lab, I suppose....

[davidwagner]

It looks like the way to migrate to Jupyter Lab might be by adding notebook_interface: jupyterlab under launch_buttons in _config.yml. See PR #185 for my attempt at testing.

Incidentally, more testing of URLs:

• https://mybinder.org/v2/gh/data-8/textbook/main?urlpath=lab/tree/chapters/01/3/1/Literary_Characters.ipynb - launches notebook in Jupyter Lab successfully

Possibly related: jupyter-book/jupyter-book#1906

[davidwagner]

OK, I've migrated the Binder links over to Jupyter Lab and deployed it live. Now the Binder links work.