New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Vector softmax #1172

Merged

jamesroxbypb merged 3 commits into asplos from vector_softmax

Apr 4, 2024

Collaborator

jamesroxbypb commented Apr 4, 2024

A vector softmax operator with a sweep capability,

pjr added 3 commits

April 4, 2024 08:47


          [SOFTMAX] Single column rapid test

e6b705a


          [SWEEP] A sweep with simple wallclock profiling

10a11be


          [MERGE] Merge

9cf1c28

jamesroxbypb merged commit 30d9392 into asplos

53 of 54 checks passed

jamesroxbypb deleted the vector_softmax branch

April 4, 2024 18:52

github-actions bot reviewed

View reviewed changes

reference_designs/ipu-xrt/vector_softmax/exp.cc

Comment on lines +54 to +56

+              void exp_bf16_vector(bfloat16 *a_in, bfloat16 *c_out) {
+                dut(a_in, c_out);
+              }

Contributor

github-actions bot Apr 4, 2024

[clang-format] _{reported by reviewdog 🐶}

Suggested change

      
            void exp_bf16_vector(bfloat16 *a_in, bfloat16 *c_out) {
          
              dut(a_in, c_out);
          
            }
          
            void exp_bf16_vector(bfloat16 *a_in, bfloat16 *c_out) { dut(a_in, c_out); }

reference_designs/ipu-xrt/vector_softmax/test.cpp

		#include <math.h>
		>>>>>>> asplos

Contributor

github-actions bot Apr 4, 2024

[clang-format] _{reported by reviewdog 🐶}

Suggested change

      
            >>>>>>> asplos
          
                >>>>>>> asplos

reference_designs/ipu-xrt/vector_softmax/test.cpp

               #include <sstream>
               #include <stdfloat>
               #include <string>
               #include <vector>
+              <<<<<<< HEAD

Contributor

github-actions bot Apr 4, 2024

[clang-format] _{reported by reviewdog 🐶}

Suggested change

      
            <<<<<<< HEAD
          
                <<<<<<< HEAD

reference_designs/ipu-xrt/vector_softmax/test.cpp

               #include <sstream>
               #include <stdfloat>
               #include <string>
               #include <vector>
+              <<<<<<< HEAD
+              #include <math.h>
+              =======

Contributor

github-actions bot Apr 4, 2024

[clang-format] _{reported by reviewdog 🐶}

Suggested change

      
            =======
          
                =======

reference_designs/ipu-xrt/vector_softmax/test.cpp

+              <<<<<<< HEAD
+              #include <math.h>
+              =======
+              >>>>>>> asplos
               #include "xrt/xrt_bo.h"
               #include "xrt/xrt_device.h"
               #include "xrt/xrt_kernel.h"
               constexpr bool VERIFY = true;

Contributor

github-actions bot Apr 4, 2024

[clang-format] _{reported by reviewdog 🐶}

Suggested change

      
            constexpr bool VERIFY = true;
          
                constexpr bool VERIFY = true;

reference_designs/ipu-xrt/vector_softmax/test.cpp

    
                        running += ez;

                        RefVec[t + i] = exp(AVec[t + i]);

                        RefVec[t+i] = exp(AVec[t+i]);

Contributor

github-actions bot Apr 4, 2024

[clang-format] _{reported by reviewdog 🐶}

Suggested change

      
                      RefVec[t+i] = exp(AVec[t+i]);
          
                      RefVec[t + i] = exp(AVec[t + i]);

reference_designs/ipu-xrt/vector_softmax/test.cpp

		}

Contributor

github-actions bot Apr 4, 2024

[clang-format] _{reported by reviewdog 🐶}

Suggested change

reference_designs/ipu-xrt/vector_softmax/test.cpp

                       for (uint32_t i = 0; i < TILE_SIZE; i++) {
-                        RefVec[t + i] /= running;
+                        RefVec[t+i] /= running;

Contributor

github-actions bot Apr 4, 2024

[clang-format] _{reported by reviewdog 🐶}

Suggested change

      
                      RefVec[t+i] /= running;
          
                      RefVec[t + i] /= running;

reference_designs/ipu-xrt/vector_softmax/test.cpp

                       }
-                    }
+                    }

Contributor

github-actions bot Apr 4, 2024

[clang-format] _{reported by reviewdog 🐶}

Suggested change

reference_designs/ipu-xrt/vector_softmax/test.cpp

Comment on lines +246 to +248

+                    float cpu_time =
+                        std::chrono::duration_cast<std::chrono::microseconds>(cpu_stop - cpu_start)
+                            .count();

Contributor

github-actions bot Apr 4, 2024

[clang-format] _{reported by reviewdog 🐶}

Suggested change

      
                  float cpu_time =
          
                      std::chrono::duration_cast<std::chrono::microseconds>(cpu_stop - cpu_start)
          
                          .count();
          
                  float cpu_time = std::chrono::duration_cast<std::chrono::microseconds>(
          
                                       cpu_stop - cpu_start)
          
                                       .count();

github-actions bot reviewed

View reviewed changes

reference_designs/ipu-xrt/vector_softmax/aie2.py

@@ @@ -16,7 +16,7 @@ @@
               def my_eltwise_add():
                   word_size_in = 2
-                  N = 65536  # *1024
+                  N = 262144 #*1024

Contributor

github-actions bot Apr 4, 2024

[black] _{reported by reviewdog 🐶}

Suggested change

      
                N = 262144 #*1024
          
                N = 262144  # *1024

reference_designs/ipu-xrt/vector_softmax/sweep.py

		@@ -0,0 +1,20 @@
		import os;

Contributor

github-actions bot Apr 4, 2024

[black] _{reported by reviewdog 🐶}

Suggested change

      
            import os;
          
            import os

reference_designs/ipu-xrt/vector_softmax/sweep.py

		@@ -0,0 +1,20 @@
		import os;

		for action in ["rm -f","touch"]:

Contributor

github-actions bot Apr 4, 2024

[black] _{reported by reviewdog 🐶}

Suggested change

      
            for action in ["rm -f","touch"]:
          
            for action in ["rm -f", "touch"]:

reference_designs/ipu-xrt/vector_softmax/sweep.py

Comment on lines +8 to +9

		for s in [16384,32768,65536,131072,262144]:
		for i in [64,128,256,512,1024]:

Contributor

github-actions bot Apr 4, 2024

[black] _{reported by reviewdog 🐶}

Suggested change

      
            for s in [16384,32768,65536,131072,262144]:
          
                for i in [64,128,256,512,1024]:
          
            for s in [16384, 32768, 65536, 131072, 262144]:
          
                for i in [64, 128, 256, 512, 1024]:

reference_designs/ipu-xrt/vector_softmax/sweep.py

+                      make_all = f"make all"
+                      os.system(make_all)
+                      make_profile = f"make profile"
+                      os.system(make_profile)

Contributor

github-actions bot Apr 4, 2024

[black] _{reported by reviewdog 🐶}

Suggested change

      
                    os.system(make_profile)    
          
                    os.system(make_profile)

Contributor

github-actions bot commented Apr 4, 2024

Coverage Report

Created: 2024-04-04 19:05

Click here for information about interpreting this report.

Filename	Function Coverage	Line Coverage	Region Coverage	Branch Coverage
Totals	-	-	-	-

Generated by llvm-cov -- llvm version 14.0.0

fifield pushed a commit to fifield/mlir-aie that referenced this pull request


          Vector softmax (Xilinx#1172)

Co-authored-by: pjr <[email protected]>

fifield pushed a commit to fifield/mlir-aie that referenced this pull request


          Vector softmax (Xilinx#1172)

7e20ebe

Co-authored-by: pjr <[email protected]>

fifield pushed a commit to fifield/mlir-aie that referenced this pull request


          Vector softmax (Xilinx#1172)

0a5474f

Co-authored-by: pjr <[email protected]>

fifield pushed a commit to fifield/mlir-aie that referenced this pull request


          Vector softmax (Xilinx#1172)

11ab732

Co-authored-by: pjr <[email protected]>

fifield pushed a commit to fifield/mlir-aie that referenced this pull request


          Vector softmax (Xilinx#1172)

92cfc66

Co-authored-by: pjr <[email protected]>

fifield pushed a commit to fifield/mlir-aie that referenced this pull request


          Vector softmax (Xilinx#1172)

6497bda

Co-authored-by: pjr <[email protected]>

fifield pushed a commit to fifield/mlir-aie that referenced this pull request


          Vector softmax (Xilinx#1172)

0a0ac67

Co-authored-by: pjr <[email protected]>

fifield pushed a commit to fifield/mlir-aie that referenced this pull request


          Vector softmax (Xilinx#1172)

d032536

Co-authored-by: pjr <[email protected]>

fifield pushed a commit to fifield/mlir-aie that referenced this pull request


          Vector softmax (Xilinx#1172)

f5b8825

Co-authored-by: pjr <[email protected]>

fifield pushed a commit to fifield/mlir-aie that referenced this pull request


          Vector softmax (Xilinx#1172)

ccd8048

Co-authored-by: pjr <[email protected]>

fifield pushed a commit to fifield/mlir-aie that referenced this pull request


          Vector softmax (Xilinx#1172)

4ece705

Co-authored-by: pjr <[email protected]>

fifield pushed a commit to fifield/mlir-aie that referenced this pull request


          Keep pkt header (Xilinx#1125)

adc93a5

Signed-off-by: Abhishek Varma <[email protected]>
Co-authored-by: Javier Setoain <[email protected]>
Co-authored-by: James Newling <[email protected]>
Co-authored-by: Maksim Levental <[email protected]>
Co-authored-by: Joseph Melber <[email protected]>
Co-authored-by: Abhishek Varma <[email protected]>
Co-authored-by: erwei-xilinx <[email protected]>
Co-authored-by: AndraBisca <[email protected]>

Started writing the objfifo intro tutorial

Vectorize vec scalar (Xilinx#1135)

Added new programming guide section placedholders (Xilinx#1138)

[EXAMPLE] An element-wise add example (Xilinx#1148)

Co-authored-by: pjr <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jack Lo <[email protected]>

Moved objfifo design example 1 to programming_guide sections 3.

Started objectFifo programming guide

Continue section 3 guide.

ObjFifo guide: access patterns

Add example to objFifo guide

Update objfifo guide

Separate section 3 of the guide into 3 subsections

[ASPLOS] Weight expand asplos (Xilinx#1158)

Co-authored-by: pjr <[email protected]>
Co-authored-by: Phil James-Roxby <[email protected]>

Reorganize subsections in section 3 of the guide.

Updated sections 3a and 3b

Pjr vector exp (Xilinx#1166)

Co-authored-by: pjr <[email protected]>
Co-authored-by: Joseph Melber <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

[SOFTMAX] Single column rapid test (Xilinx#1168)

Co-authored-by: pjr <[email protected]>
Co-authored-by: Joseph Melber <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Vector softmax (Xilinx#1172)

Co-authored-by: pjr <[email protected]>

[MERGE] This has gone horribly wrong, so fixing up in place

Update exp.cc

Merge resolved

Added new programming guide section placedholders (Xilinx#1174)

Swapped section-2 and section-3 (Xilinx#1177)

Update section-1 and section-3 examples (Xilinx#1179)

Update section-3 (Xilinx#1181)

Reorganize tutorials and reference_designs to programming_examples (Xilinx#1182)

Fix for mmult lit (Xilinx#1187)

Revert "Fix for mmult lit" (Xilinx#1188)

[ASPLOS][WIP] python host code example (Xilinx#1185)

Extract arg parse from test.py (Xilinx#1192)

Add finished write-up for sections 2a and 2b

Add objectfifo bindings to quick references

Update section2 subsection list

[ASPLOS][WIP] initial version of asplos24 tutorial description (Xilinx#1184)

Co-authored-by: Jack Lo <[email protected]>

Fix for lit tests (Xilinx#1189)

Rename tutorials folder (Xilinx#1190) (Xilinx#1197)

Co-authored-by: AndraBisca <[email protected]>
Co-authored-by: Jack Lo <[email protected]>

[ASPLOS] Rename directories (Xilinx#1196)

Co-authored-by: Jeff Fifield <[email protected]>
Co-authored-by: AndraBisca <[email protected]>
Co-authored-by: Jack Lo <[email protected]>

Add section 2c. Update tiles in sections 2a and 2b.

Add generic aie array description paragraph (Xilinx#1191)

Co-authored-by: Joseph Melber <[email protected]>
Co-authored-by: Jack Lo <[email protected]>

ReLU with tracing (Xilinx#1204)

ReLU example with tracing

Co-authored-by: pjr <[email protected]>
Co-authored-by: Joseph Melber <[email protected]>

Ml eltwise add and mul (Xilinx#1207)

Move around of the eltwise add (put it in ml) and a new eltwise mul kernel

Co-authored-by: pjr <[email protected]>
Co-authored-by: Jeff Fifield <[email protected]>

Moved test_lib to runtime_lib/test_lib for now

Pjr reduce (Xilinx#1222) Reduce programming examples

Co-authored-by: pjr <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

[ASPLOS][WIP] Passthrough kernel in basic examples (Xilinx#1216)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

fix paths run.lit passthrough _kernel (Xilinx#1225)

Fixed CMakeLists.txt reference to test_utils.h (Xilinx#1223)

Minor CMakeLists.txt and Makefile fixes for programming_examples (Xilinx#1227)

fifield pushed a commit to fifield/mlir-aie that referenced this pull request


          Keep pkt header (Xilinx#1125)

eb09a5c

Signed-off-by: Abhishek Varma <[email protected]>
Co-authored-by: Javier Setoain <[email protected]>
Co-authored-by: James Newling <[email protected]>
Co-authored-by: Maksim Levental <[email protected]>
Co-authored-by: Joseph Melber <[email protected]>
Co-authored-by: Abhishek Varma <[email protected]>
Co-authored-by: erwei-xilinx <[email protected]>
Co-authored-by: AndraBisca <[email protected]>

Started writing the objfifo intro tutorial

Vectorize vec scalar (Xilinx#1135)

Added new programming guide section placedholders (Xilinx#1138)

[EXAMPLE] An element-wise add example (Xilinx#1148)

Co-authored-by: pjr <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jack Lo <[email protected]>

Moved objfifo design example 1 to programming_guide sections 3.

Started objectFifo programming guide

Continue section 3 guide.

ObjFifo guide: access patterns

Add example to objFifo guide

Update objfifo guide

Separate section 3 of the guide into 3 subsections

[ASPLOS] Weight expand asplos (Xilinx#1158)

Co-authored-by: pjr <[email protected]>
Co-authored-by: Phil James-Roxby <[email protected]>

Reorganize subsections in section 3 of the guide.

Updated sections 3a and 3b

Pjr vector exp (Xilinx#1166)

Co-authored-by: pjr <[email protected]>
Co-authored-by: Joseph Melber <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

[SOFTMAX] Single column rapid test (Xilinx#1168)

Co-authored-by: pjr <[email protected]>
Co-authored-by: Joseph Melber <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Vector softmax (Xilinx#1172)

Co-authored-by: pjr <[email protected]>

[MERGE] This has gone horribly wrong, so fixing up in place

Update exp.cc

Merge resolved

Added new programming guide section placedholders (Xilinx#1174)

Swapped section-2 and section-3 (Xilinx#1177)

Update section-1 and section-3 examples (Xilinx#1179)

Update section-3 (Xilinx#1181)

Reorganize tutorials and reference_designs to programming_examples (Xilinx#1182)

Fix for mmult lit (Xilinx#1187)

Revert "Fix for mmult lit" (Xilinx#1188)

[ASPLOS][WIP] python host code example (Xilinx#1185)

Extract arg parse from test.py (Xilinx#1192)

Add finished write-up for sections 2a and 2b

Add objectfifo bindings to quick references

Update section2 subsection list

[ASPLOS][WIP] initial version of asplos24 tutorial description (Xilinx#1184)

Co-authored-by: Jack Lo <[email protected]>

Fix for lit tests (Xilinx#1189)

Rename tutorials folder (Xilinx#1190) (Xilinx#1197)

Co-authored-by: AndraBisca <[email protected]>
Co-authored-by: Jack Lo <[email protected]>

[ASPLOS] Rename directories (Xilinx#1196)

Co-authored-by: Jeff Fifield <[email protected]>
Co-authored-by: AndraBisca <[email protected]>
Co-authored-by: Jack Lo <[email protected]>

Add section 2c. Update tiles in sections 2a and 2b.

Add generic aie array description paragraph (Xilinx#1191)

Co-authored-by: Joseph Melber <[email protected]>
Co-authored-by: Jack Lo <[email protected]>

ReLU with tracing (Xilinx#1204)

ReLU example with tracing

Co-authored-by: pjr <[email protected]>
Co-authored-by: Joseph Melber <[email protected]>

Ml eltwise add and mul (Xilinx#1207)

Move around of the eltwise add (put it in ml) and a new eltwise mul kernel

Co-authored-by: pjr <[email protected]>
Co-authored-by: Jeff Fifield <[email protected]>

Moved test_lib to runtime_lib/test_lib for now

Pjr reduce (Xilinx#1222) Reduce programming examples

Co-authored-by: pjr <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

[ASPLOS][WIP] Passthrough kernel in basic examples (Xilinx#1216)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

fix paths run.lit passthrough _kernel (Xilinx#1225)

Fixed CMakeLists.txt reference to test_utils.h (Xilinx#1223)

Minor CMakeLists.txt and Makefile fixes for programming_examples (Xilinx#1227)

fifield pushed a commit to fifield/mlir-aie that referenced this pull request


          Keep pkt header (Xilinx#1125)

Signed-off-by: Abhishek Varma <[email protected]>
Co-authored-by: Javier Setoain <[email protected]>
Co-authored-by: James Newling <[email protected]>
Co-authored-by: Maksim Levental <[email protected]>
Co-authored-by: Joseph Melber <[email protected]>
Co-authored-by: Abhishek Varma <[email protected]>
Co-authored-by: erwei-xilinx <[email protected]>
Co-authored-by: AndraBisca <[email protected]>

Started writing the objfifo intro tutorial

Vectorize vec scalar (Xilinx#1135)

Added new programming guide section placedholders (Xilinx#1138)

[EXAMPLE] An element-wise add example (Xilinx#1148)

Co-authored-by: pjr <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jack Lo <[email protected]>

Moved objfifo design example 1 to programming_guide sections 3.

Started objectFifo programming guide

Continue section 3 guide.

ObjFifo guide: access patterns

Add example to objFifo guide

Update objfifo guide

Separate section 3 of the guide into 3 subsections

[ASPLOS] Weight expand asplos (Xilinx#1158)

Co-authored-by: pjr <[email protected]>
Co-authored-by: Phil James-Roxby <[email protected]>

Reorganize subsections in section 3 of the guide.

Updated sections 3a and 3b

Pjr vector exp (Xilinx#1166)

Co-authored-by: pjr <[email protected]>
Co-authored-by: Joseph Melber <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

[SOFTMAX] Single column rapid test (Xilinx#1168)

Co-authored-by: pjr <[email protected]>
Co-authored-by: Joseph Melber <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Vector softmax (Xilinx#1172)

Co-authored-by: pjr <[email protected]>

[MERGE] This has gone horribly wrong, so fixing up in place

Update exp.cc

Merge resolved

Added new programming guide section placedholders (Xilinx#1174)

Swapped section-2 and section-3 (Xilinx#1177)

Update section-1 and section-3 examples (Xilinx#1179)

Update section-3 (Xilinx#1181)

Reorganize tutorials and reference_designs to programming_examples (Xilinx#1182)

Fix for mmult lit (Xilinx#1187)

Revert "Fix for mmult lit" (Xilinx#1188)

[ASPLOS][WIP] python host code example (Xilinx#1185)

Extract arg parse from test.py (Xilinx#1192)

Add finished write-up for sections 2a and 2b

Add objectfifo bindings to quick references

Update section2 subsection list

[ASPLOS][WIP] initial version of asplos24 tutorial description (Xilinx#1184)

Co-authored-by: Jack Lo <[email protected]>

Fix for lit tests (Xilinx#1189)

Rename tutorials folder (Xilinx#1190) (Xilinx#1197)

Co-authored-by: AndraBisca <[email protected]>
Co-authored-by: Jack Lo <[email protected]>

[ASPLOS] Rename directories (Xilinx#1196)

Co-authored-by: Jeff Fifield <[email protected]>
Co-authored-by: AndraBisca <[email protected]>
Co-authored-by: Jack Lo <[email protected]>

Add section 2c. Update tiles in sections 2a and 2b.

Add generic aie array description paragraph (Xilinx#1191)

Co-authored-by: Joseph Melber <[email protected]>
Co-authored-by: Jack Lo <[email protected]>

ReLU with tracing (Xilinx#1204)

ReLU example with tracing

Co-authored-by: pjr <[email protected]>
Co-authored-by: Joseph Melber <[email protected]>

Ml eltwise add and mul (Xilinx#1207)

Move around of the eltwise add (put it in ml) and a new eltwise mul kernel

Co-authored-by: pjr <[email protected]>
Co-authored-by: Jeff Fifield <[email protected]>

Moved test_lib to runtime_lib/test_lib for now

Pjr reduce (Xilinx#1222) Reduce programming examples

Co-authored-by: pjr <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

[ASPLOS][WIP] Passthrough kernel in basic examples (Xilinx#1216)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

fix paths run.lit passthrough _kernel (Xilinx#1225)

Fixed CMakeLists.txt reference to test_utils.h (Xilinx#1223)

Minor CMakeLists.txt and Makefile fixes for programming_examples (Xilinx#1227)

efurst pushed a commit to efurst/mlir-aie that referenced this pull request


          Vector softmax (Xilinx#1172)

43b1a23

Co-authored-by: pjr <[email protected]>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet