Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance: reduce for cycles when handling dataframe #245

Open
Wh1isper opened this issue Nov 22, 2024 · 1 comment
Open

Performance: reduce for cycles when handling dataframe #245

Wh1isper opened this issue Nov 22, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@Wh1isper
Copy link
Collaborator

Wh1isper commented Nov 22, 2024

Problem

Thanks @cyantangerine in #244:

I suggest that we should have a check for all code to replace 'for' to pandas.apply morely. I saw a lot of implement like this in code. It can improve the performance a lot.

Proposed Solution

for is usally bad in performance when handling pandas.dataframe, but unfortunately, there are many times when we don't take that into account.

If anyone is interested in helping us improve performance, please feel free to draft a pull reqeust!

Additional context

@cyantangerine
Copy link
Contributor

cyantangerine commented Nov 22, 2024

Improving performance for data processing and model trainning by changing code style as a Pythoner.

Problem

I saw a lot of implement of operation to dataframe or series using 'for' cycling. In Python, not like C lang, it can cause performance problem. Because of the Python interpreter, 'for' runs slowly than C. But, the pandas are implied by C, so if we use pandas method like 'apply' instead, we are running C to solve the dataframe, which are faster. The pic show the result.

Relative work: #244

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants