M(Multi-vector parrallelization)?

[dj-park]

I do understand that by adjusting PE value and SIMD value, we can achieve different degrees of parallelism.
But in FINN-R paper(https://arxiv.org/pdf/1809.04570.pdf), M parameter, which is "multi-vector parallelization", is discussed. From the paper, it sounds like we can adjust M parameter too to adjust parallelism.

Given the generated HLS code below, is "numReps" related to this M parameter?

#define AP_INT_MAX_W 216
#include "bnn-library.h"

// includes for network parameters
#include "weights.hpp"
#include "activations.hpp"
#include "mvau.hpp"
#include "thresh.h"

// defines for network parameters
#define MW1 27
#define MH1 64
#define SIMD1 27
#define PE1 8
#define WMEM1 8
#define TMEM1 8
#define numReps 900

void MatrixVectorActivation_0(hls::stream<ap_uint<216>> &in0,
                    hls::stream<ap_uint<8>> &out
                    )
{
#pragma HLS INTERFACE axis register port=in0
#pragma HLS INTERFACE axis register port=out
#include "params.h"
#pragma HLS ARRAY_PARTITION variable=weights.m_weights complete dim=1
#pragma HLS ARRAY_PARTITION variable=threshs.m_thresholds complete dim=1
#pragma HLS ARRAY_PARTITION variable=threshs.m_thresholds complete dim=3
Matrix_Vector_Activate_Batch<MW1, MH1, SIMD1, PE1, 1, Slice<ap_int<8>>, Slice<ap_uint<1>>, Recast<Binary>>
                (in0, out, weights, threshs, numReps, ap_resource_lut());
}

Also, from

finn/src/finn/custom_op/fpgadataflow/matrixvectoractivation.py

Line 393 in b3bdff1

mmv = 1

, is variable mmv also related? Is this not implemented yet?

Thanks in advance.

[fpjentzsch]

Hi,

on the hlslib back-end, multiple components have supported this "M" or "MMV" degree of parallelism for years. On the FINN compiler side, integration of this functionality is currently in progress in this PR: #789.

Support for the most common layers (MVAU, VVAU, padding, pooling) should be ready within the next couple weeks.

[fpjentzsch]

Regarding the numReps parameter: The main purpose of this parameter is to define how many iterations the C simulation of the HLS component must run for. During actual operation on hardware, each component is continuously free-running.

It is related to M in the sense that, when M is larger than one, fewer "repetitions" are needed to compute all output pixels of the feature map. E.g., if you process M=2 pixels per cycle, numReps will be halved. In any case, the FINN compiler sets this parameter automatically and you should not need to touch it.

[dj-park]

Thanks for the answer. Yes, I do see MMV is set to 1 in the generated hls code as shown below, but maybe we can play around with it.

Matrix_Vector_Activate_Batch<MW1_mva_5, MH1_mva_5, SIMD1_mva_5, PE1_mva_5, 1, Recast<XnorMul>, Slice<ap_uint<1>>, Identity>
                (in0, out, mva_5_weights, mva_5_thresh, numReps_mva_5, ap_resource_lut());
}

Okay, so numReps should not affect the hardware implementation, is it correct?

[dj-park]

@fpjentzsch In case that you haven't received the notification..