Upgrade airflow

[kolok]

upgrade to 2.10.0

TODO :

• development_validate_and_process_dagruns failed

70a8b4a96059
*** Found local files:
***   * /opt/airflow/logs/dag_id=development_validate_and_process_dagruns/run_id=scheduled__2024-08-24T16:55:00+00:00/task_id=fetch_and_parse_data/attempt=1.log
[2024-08-24, 17:00:03 UTC] {local_task_job_runner.py:123} ▶ Pre task execution logs
[2024-08-24, 17:00:04 UTC] {base.py:84} INFO - Retrieving connection 'lvao-development'
[2024-08-24, 17:00:04 UTC] {logging_mixin.py:190} WARNING - /opt/***/dags/development/ingest_validated_dataset_to_db.py:48 UserWarning: pandas only supports SQLAlchemy connectable (engine/connection) or database string URI or sqlite3 DBAPI2 connection. Other DBAPI2 objects are not tested. Please consider using SQLAlchemy.
[2024-08-24, 17:00:04 UTC] {taskinstance.py:3301} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 767, in _execute_task
    result = _execute_callable(context=context, **execute_callable_kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 733, in _execute_callable
    return ExecutionCallableRunner(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/operator_helpers.py", line 252, in run
    return self.func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/baseoperator.py", line 406, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/operators/python.py", line 238, in execute
    return_value = self.execute_callable()
                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/operators/python.py", line 256, in execute_callable
    return runner.run(*self.op_args, **self.op_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/operator_helpers.py", line 252, in run
    return self.func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/airflow/dags/development/ingest_validated_dataset_to_db.py", line 48, in fetch_and_parse_data
    df_sql = pd.read_sql_query(
             ^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/pandas/io/sql.py", line 526, in read_sql_query
    return pandas_sql.read_query(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/pandas/io/sql.py", line 2738, in read_query
    cursor = self.execute(sql, params)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/pandas/io/sql.py", line 2672, in execute
    cur = self.con.cursor()
          ^^^^^^^^^^^^^^^
AttributeError: 'Engine' object has no attribute 'cursor'
[2024-08-24, 17:00:04 UTC] {taskinstance.py:1225} INFO - Marking task as UP_FOR_RETRY. dag_id=development_validate_and_process_dagruns, task_id=fetch_and_parse_data, run_id=scheduled__2024-08-24T16:55:00+00:00, execution_date=20240824T165500, start_date=20240824T170003, end_date=20240824T170004
[2024-08-24, 17:00:04 UTC] {taskinstance.py:340} ▶ Post task execution logs

Je tombe sur cette incompatibilité de version : pandas-dev/pandas#57049

UPDATE

J'ai dû fixer pandas dans sa version 2.1.4 pour dépendre des versions <2.0 pour sqlalchemy et numpy, j'ai laissé un commentaire dans le code