Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] kernel for date_trunc and trunc that has a scalar format #11860

Closed
revans2 opened this issue Dec 11, 2024 · 1 comment · Fixed by #11833
Closed

[FEA] kernel for date_trunc and trunc that has a scalar format #11860

revans2 opened this issue Dec 11, 2024 · 1 comment · Fixed by #11833
Labels
performance A performance related task/issue

Comments

@revans2
Copy link
Collaborator

revans2 commented Dec 11, 2024

Is your feature request related to a problem? Please describe.
The current implementation of trunc and date_trunc use a kernel where the format is a column.

This works great. In my tests I saw a 180x speedup over the CPU (16 cores). But we could save a lot of memory if the format is a scalar, which I think is the most common case.

In the best case where the format string is "DD", a new kernel we would save about 6 bytes per row of input. On a date that is 150% increase in memory usage. For a timestamp that is only a 75% increase in memory usage. But for the worst case on a date it is "QUARTER" or a 275% increase in memory usage. For a timestamp it is "MICROSECOND", which would be 187.5% increase in memory usage. This is probably minor, but it would be nice.

@revans2 revans2 added ? - Needs Triage Need team to review and classify feature request New feature or request labels Dec 11, 2024
@ttnghia
Copy link
Collaborator

ttnghia commented Dec 11, 2024

The current JNI implementation is already optimized for memory. Although the scala is promoted into a column, such column has only one row thus there is no change in memory usage.

However, I agree that we can optimize further, but in term of performance. Currently, a format string is parsed when processing every row, even there is only one format value (column format of size one). We can do better by parsing the scalar format (on host) before calling the kernel, saving time for the GPU by not doing so again. I'll post a JNI PR shortly.

@ttnghia ttnghia linked a pull request Dec 13, 2024 that will close this issue
@sameerz sameerz added performance A performance related task/issue and removed feature request New feature or request ? - Needs Triage Need team to review and classify labels Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance A performance related task/issue
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants