LowerConvsToMatMul transformation not working in finn.

[MarkosMuche]

I am trying to convert a brevitas model to onnx-finn. When I run the code onnx_model_wrapper.transform(LowerConvsToMatMul()), I get AttributeError: 'NoneType' object has no attribute 'transpose'. I converted from brevitas to onnx using export_qonnx. Why do I get that error?

[fpjentzsch]

Hi,
did you run the ConvertQONNXtoFINN transformation (like in this builder step) and any "tidy up" or streamlining transformations that might be needed before applying LowerConvsToMatMul?

[MarkosMuche]

Hi,

yes. I have applied the following transformations before LowerConvsToMatMul

filter_fn = default_filter_function_generator(max_multithreshold_bit_width=8)
model = model.transform(ConvertQuantActToMultiThreshold(filter_function=filter_fn))

model = model.transform(InferShapes())
model = model.transform(
    ConvertQONNXtoFINN(
        filter_function=default_filter_function_generator(
            max_multithreshold_bit_width=8
        )
    )
)
model = model.transform(LowerConvsToMatMul())

My model is composed of 2d convolutions. At first, my model was composed of 1d Convolutions. But it didn't support the LowerConvsToMatMul operation. So, I added a dummy dimension.

the following is one of the layers in brevitas.

            in_channels=2,
            out_channels=64,
            kernel_size=(1, 5),  
            padding=(0, 2),
            weight_bit_width=8,
            weight_quant=Int8WeightPerTensorFloat,
        )```

[fpjentzsch]

By the way, you can make 1D convolutions work by adding the dummy dimension with this transformation, so you don't have to add the dummy dimension in Brevitas.

[MarkosMuche]

model = model.transform(Change3DTo4DTensors())
model = model.transform(LowerConvsToMatMul())

I run that code and I still get the following error.

Traceback (most recent call last):

  File "finn/deps/qonnx/src/qonnx/core/modelwrapper.py", line 140, in transform
    (transformed_model, model_was_changed) = transformation.apply(transformed_model)
  File "finn/deps/qonnx/src/qonnx/transformation/lower_convs_to_matmul.py", line 77, in apply
    k_w = k[1]
IndexError: list index (1) out of range

[MarkosMuche]

I think I know the problem now. My qonnx model has DequantizeLinear nodes. for example the following is an input to my conv layer.

X
name: /relu1/act_quant/export_handler/DequantizeLinear_output_0
W
name: /conv2/weight_quant/export_handler/DequantizeLinear_output_0
B
name: conv2.bias

[MarkosMuche]

I have another question. I converted from brevitas to qonxx using

Then, I started applying the conversion steps to finn. But, I get an error.

model = ModelWrapper(export_path)
output_dir = export_path+".output"
synth_clk_period_ns = 5.0
generate_outputs = [DataflowOutputType.STITCHED_IP, DataflowOutputType.OOC_SYNTH] 

cfg = DataflowBuildConfig(
    output_dir=output_dir,
    synth_clk_period_ns=synth_clk_period_ns,
    generate_outputs=generate_outputs
)

# Run the conversion steps
onnx_model_wrapper = step_tidy_up(onnx_model_wrapper, cfg)
onnx_model_wrapper = step_streamline(onnx_model_wrapper, cfg)
onnx_model_wrapper = step_qonnx_to_finn(onnx_model_wrapper, cfg)

But, I get the following error for the second step.

   model = step_streamline(model, cfg)
  File "finn/src/finn/builder/build_dataflow_steps.py", line 314, in step_streamline
    model = model.transform(Streamline())
  finn/deps/qonnx/src/qonnx/core/modelwrapper.py", line 140, in transform
    (transformed_model, model_was_changed) = transformation.apply(transformed_model)
finn/src/finn/transformation/streamline/__init__.py", line 98, in apply
    model = model.transform(trn)
  finn/deps/qonnx/src/qonnx/core/modelwrapper.py", line 140, in transform
    (transformed_model, model_was_changed) = transformation.apply(transformed_model)
  finn/src/finn/transformation/streamline/absorb.py", line 267, in apply
    assert W is not None, "Initializer for conv weights is not set."
AssertionError: Initializer for conv weights is not set.

[MarkosMuche]

Thank you @fpjentzsch. I was wrong using a wrong conversion function export_onnx_qcdq. Then, I used export_qonnx. Then, I followed the steps. I followed the following notebook. https://github.com/Xilinx/finn/blob/main/notebooks/basics/1_brevitas_network_import_via_QONNX.ipynb.
Note: Finn doesn't support Unsigned int for QuantRelu and QuantIdentity layers.