@ZoranPandovski Hi, I created the issue for updating the Airtable Handler.

Draft PR: #10108

@ZoranPandovski @MinuraPunchihewa Hi, for this issue, I am planning on not only modifying the handler to use the SDK but also transforming the Airtable handler from Database handler to an API (Application) handler because it seems more intuitive for me since it uses restful api calls.

However, before proceeding further, I just want to discuss with you whether you agree on this because I think it is a big change and I don't want to delete/redo existing functionalities that may have some intended purposes.

The pyairtable has formulas which are used to filter and query records. In the current implementation, we are retrieving all records, merge them into pandas.Dataframe and use duckdb to do SQL operations. If we want to use their formulas, then I think API (Application) handler is a better fit but it's more involved to implement.

Final decision

I will just make the handler to use the Python SDK (pyairtable) as intended for this issue and in the future we can think about whether it is worth to use the formulas.

Hey @HahaBill,
Sorry for the late reply, I am in agreement with you that it is probably best to convert this to an API handler. If possible, go ahead and make this update.

Hey @HahaBill, Sorry for the late reply, I am in agreement with you that it is probably best to convert this to an API handler. If possible, go ahead and make this update.

No worries! Alright, I will then convert the handler into API handler.

@ZoranPandovski @MinuraPunchihewa I've been struggling to make the SDK work in mindsdb, but finally, I have found a solution today: https://stackoverflow.com/questions/79234164/partially-initialized-module-pyairtable. It works!

This requires the Pydantic version to be less than 2.10. I saw that in requirements.txt in mindsdb that the Pydantic version has to be more or equal to 2.7.0. Is there an issue if we set the Pydantic version to be >=2.7.0 and <2.10?

But anyway, I am almost done. I only need more testing!

@HahaBill That's great!
Let's pin the version of pydantic to <2.10 in the requirements for the integration for now. Let's decide later if we want to update the main version as well.

@HahaBill That's great! Let's pin the version of pydantic to <2.10 in the requirements for the integration for now. Let's decide later if we want to update the main version as well.

Alright, got it!

@ZoranPandovski @MinuraPunchihewa Ready for a review! PR: #10108

I saw that there had been some updates regarding the Pydantic version, it seems to work fine now.