Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fuse mapping ops #144

Open
pixelspark opened this issue Mar 5, 2023 · 0 comments
Open

Fuse mapping ops #144

pixelspark opened this issue Mar 5, 2023 · 0 comments
Assignees

Comments

@pixelspark
Copy link
Collaborator

Is your feature request related to a problem? Please describe.

When there are consecutive mapping operations (Neg, Relu, etc.) we should not execute these serially each in their own shader - instead we should just write a shader that does neg(relu(input)) in one go (if at least the intermediate result from Relu in this example is not used elsewhere).

Describe the solution you'd like

Fusing should happen in the optimizer. We can introduce a custom op type wonnx.Map that takes one input and an attribute describing the functions to perform consecutively (in the above example it would contain Relu,Neg).

To also accomodate binary functions (Add, Sub, etc.) we might even allow an arbitrary number of inputs and have the attribute describe (in RPN) the desired operations, e.g. neg(relu(add(a, sub(b, c)))) would have three inputs (b, c, a in that order) and the attribute could contain Push, Push, Sub, Push, Add, Relu, Neg. The compiler can then simply write out the WGSL corresponding to this.

Describe alternatives you've considered

Fusing would also be possible at the shape inference stage.

We should check if the current ConvRelu optimization (which fuses Conv and Relu works properly if the output from Conv is also used further on.

Additional context

@pixelspark pixelspark self-assigned this Mar 5, 2023
@pixelspark pixelspark changed the title Fuse math ops Fuse mapping ops Mar 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant