Cannot compile in offline mode

[sylque]

Platform pioarduino/platform-espressif32 cannot be used without an Internet connection:

Processing lolin_d32 (platform: https://github.com/pioarduino/platform-espressif32/releases/download/stable/platform-espressif32.zip; board: lolin_d32; framework: arduino)
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ConnectionError: Traceback (most recent call last):
  File "C:\Users\S\.platformio\penv\Lib\site-packages\urllib3\connection.py", line 199, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\urllib3\util\connection.py", line 60, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\python3\Lib\socket.py", line 962, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno 11001] getaddrinfo failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\S\.platformio\penv\Lib\site-packages\urllib3\connectionpool.py", line 789, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\urllib3\connectionpool.py", line 490, in _make_request
    raise new_e
  File "C:\Users\S\.platformio\penv\Lib\site-packages\urllib3\connectionpool.py", line 466, in _make_request
    self._validate_conn(conn)
  File "C:\Users\S\.platformio\penv\Lib\site-packages\urllib3\connectionpool.py", line 1095, in _validate_conn
    conn.connect()
  File "C:\Users\S\.platformio\penv\Lib\site-packages\urllib3\connection.py", line 693, in connect
    self.sock = sock = self._new_conn()
                       ^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\urllib3\connection.py", line 206, in _new_conn
    raise NameResolutionError(self.host, self, e) from e
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x000001A2DAA68510>: Failed to resolve 'api.github.com' ([Errno 11001] getaddrinfo failed)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\S\.platformio\penv\Lib\site-packages\requests\adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\urllib3\connectionpool.py", line 843, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\urllib3\util\retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /repos/espressif/Arduino-esp32/releases/latest (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x000001A2DAA68510>: Failed to resolve 'api.github.com' ([Errno 11001] getaddrinfo failed)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\__main__.py", line 103, in main
    cli()  # pylint: disable=no-value-for-parameter
    ^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\cli.py", line 85, in invoke
    return super().invoke(ctx)
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\click\core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\click\decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\run\cli.py", line 145, in cli
    process_env(
  File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\run\cli.py", line 208, in process_env
    ).process()
      ^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\run\processor.py", line 81, in process
    install_project_env_dependencies(
  File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\package\commands\install.py", line 132, in install_project_env_dependencies
    _install_project_env_platform(project_env, options),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\package\commands\install.py", line 149, in _install_project_env_platform
    PlatformPackageManager().install(
  File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\package\manager\platform.py", line 60, in install
    p.configure_project_packages(project_env, project_targets)
  File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\platform\base.py", line 185, in configure_project_packages
    self.configure_default_packages(options, targets or [])
  File "C:\Users\S\.platformio\platforms\espressif32@src-88f34194123b4b05ff6ef6b8e68c7f60\platform.py", line 50, in configure_default_packages
    api_data = requests.get(ARDUINO_CORE_API_URL, timeout=10).json()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\requests\api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\requests\api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\requests\sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\requests\sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\S\.platformio\penv\Lib\site-packages\requests\adapters.py", line 700, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /repos/espressif/Arduino-esp32/releases/latest (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x000001A2DAA68510>: Failed to resolve 'api.github.com' ([Errno 11001] getaddrinfo failed)"))

============================================================

An unexpected error occurred. Further steps:

* Verify that you have the latest version of PlatformIO using
  `python -m pip install -U platformio` command

* Try to find answer in FAQ Troubleshooting section
  https://docs.platformio.org/page/faq/index.html

* Report this problem to the developers
  https://github.com/platformio/platformio-core/issues

============================================================

Platform platform = platformio/espressif32 has no such problem.

[sivar2311]

Maybe not all necessary parts of the tool-chain are installed yet.
Build the project while online. This may install missing parts from online resources.
If the project builds fine, switch to offline mode and try to build again.

[sylque]

I've already tried that: the projects builds fine while online, but never while offline.

[sylque]

When compiling, it seems pioarduino/platform-espressif32 always try to load from https://api.github.com/repos/espressif/arduino-esp32/zipball/3.0.7, causing both this issue and issue #60.

[sivar2311]

Specifying platform = https://github.com/pioarduino/platform-espressif32/releases/download/stable/platform-espressif32.zip
will require a online connection, because you say: "Hey look at this address and grab the platform form there."

Then pio will download the platform except it is already installed on your computer.

If you use platform = espressif32 pio will use the highest available version from your computer.
If there is no espressif32 platform available it will try to download - which also requires an online connection.

[sylque]

@sivar2311, that is not correct:

1. PlatformIO supports offline compiling even when you specify an URL for the platform.
2. As I mentioned earlier, the issue is not about accessing https://github.com/pioarduino/platform-espressif32/releases/download/stable/platform-espressif32.zip , but about accessing https://api.github.com/repos/espressif/arduino-esp32/zipball/3.0.7.

[sivar2311]

Afaik platform = https://github.com/pioarduino/platform-espressif32/releases/download/stable/platform-espressif32.zip will always look for the latest available stable-version (currently 51.03.07).

Whereby using platform = https://github.com/pioarduino/platform-espressif32/releases/download/51.03.07/platform-espressif32.zip compiles even when there is no internet connection (if it was downloaded before).

[sylque]

@sivar2311 , that's correct, and https://github.com/platformio/platform-espressif32.git will always look for the latest available debug version, except when there's no available Internet connection. But again, the problem seems to lie in https://api.github.com/repos/espressif/arduino-esp32/zipball/3.0.7, not in the platform URL.

[sivar2311]

and https://github.com/platformio/platform-espressif32.git will always look for the latest available debug version

No. If the platform is downloaded once there is no check for updated available versions. You will stick on the version that was downloaded once. Except you delete the platform folder on your harddrive.

[sylque]

@sivar2311 I see, so what I understand is that https://github.com/pioarduino/platform-espressif32/releases/download/stable/platform-espressif32.zip contains some additional code to actively check for the latest version. But it seems it does so without checking first if the network is available, hence this issue.

So the obvious workaround is to come back to the former way of using this repo:

platform = https://github.com/pioarduino/platform-espressif32/releases/download/51.03.07/platform-espressif32.zip

instead of the way documented in the home page.

[Jason2866]

First pioarduino != Platformio Platform handling is and will always be different since not using Platformio registry for Platform. For upcoming Arduino Core 3.2.0 the Platformio registry will not be used for anything (since needed toolchains will not be available from)

I will change that offline compile will be possible again.

[Jason2866]

@sylque Try

platform = https://github.com/pioarduino/platform-espressif32.git#main_offline

after first install where always internet is needed ;-) Cut connection and try if it is still working. Can't test since no PC in reach...

[sylque]

@Jason2866, it seems to work, thanks.
Does it point on the latest stable version? If it does, I think it should rather be called #stable_offline.
Any other way to refresh it instead of manually deleting the platforms folder? Can pio pkg update be used?

[Jason2866]

@sylque Thx for testing, the branch will be merged in main. When being online the check will be tried. If no rate limit happens :-) and a newer stable release version exists it will be automatically installed.

pio pkg update is not useable with pioarduino since there are no newer packages in Platformio registry or no used packages from pioarduino are there

[Jason2866]

Merged in branch main and release package updated. After deleting the hidden folders .platformio/platforms and .platformio/cache the new version will be fetched and on- and offline compile is possible again with

[env:stable]
platform = https://github.com/pioarduino/platform-espressif32/releases/download/stable/platform-espressif32.zip
board = ...
...

[sylque]

@Jason2866 Seems to work like a charm, thank you!