loading of transfer learning MXNet resnet 18 for inference

[rd-peter-braun]

So I am fairly new to the party: I trained resnet18 instance according to the https://docs.djl.ai/master/docs/demos/jupyter/transfer_learning_on_cifar10.html example (MXNet as Engine).
and saved it.
For inference I reload it by getting the original model from MXNet ModelZoo, modify the output layer according to the trained classes,
create a new Model Instance, Set its main block with the one from the modified original and load the saved params.
Like this:
` Criteria.Builder<Image, Classifications> criteriaBuilder = Criteria.builder()
.setTypes(Image.class, Classifications.class)
.optArtifactId(props.getModel().getArtifactId())
.optEngine(props.getModel().getEngine());
props.getModel().getModelZooFilter().forEach(criteriaBuilder::optFilter);

        System.out.println(ModelZoo.listModels());
        Criteria<Image, Classifications> criteria = criteriaBuilder.build();

        Model model = criteria.loadModel();

        // adjust output layer to number of classes from trainset
        SequentialBlock newBlock = new SequentialBlock();
        SymbolBlock block = (SymbolBlock) model.getBlock();
        block.removeLastBlock();
        newBlock.add(block);
        newBlock.add(Blocks.batchFlattenBlock());
        newBlock.add(Linear.builder().setUnits(numClasses).build());
        model.setBlock(newBlock);

        Model customModel = Model.newInstance(props.getModel().getModelName());
        customModel.setBlock(newBlock);
        customModel.load(props.getModel().getModelDir().resolve(props.getModel().getModelName()), props.getModel().getModelName());`

So this basically works but only if I keep the Model model AND the Model customModel object references alive.
When originally loaded model reference is garbage collected, this has an influence on the copied custom Model.
Predicton then results in an exception:
ai.djl.translate.TranslateException: java.lang.IllegalStateException: Native resource has been released already. at ai.djl.inference.Predictor.batchPredict(Predictor.java:196) ~[api-0.28.0.jar:na]

So probably my reconstruction/reloading strategie of the original model architecture is not quite the best practice.

How shoud this be done or is there some way to decouple the dependency from my "template" model to the custom one.

Thx Peter

[frankfliu]

@rd-peter-braun

I created a PR that show case how to load a DJL trained model with BlockFactory,
see: https://github.com/deepjavalibrary/djl/blob/master/examples/src/main/java/ai/djl/examples/training/transferlearning/TrainResnetWithCifar10.java#L142

I removed symbolic training for MXNet since MXNet engine is deprecated. If you are looking for add layers to existing model, please see this example: https://github.com/deepjavalibrary/djl/blob/master/examples/src/main/java/ai/djl/examples/training/transferlearning/TransferFreshFruit.java#L66

[rd-peter-braun]

Thank you very much - a few questions:

• I checked Pytorch Model Zoo and I saw that there is basically one embedding image classification model - the resnet18 example. Correct?
• About the load example TrainResnetWithCifar10: the loaded djl://ai.djl.zoo/resnet/0.0.2/resnetv1 model probably has already an output size of 10. Correct?
• About the documentation: https://docs.djl.ai/docs/demos/jupyter/transfer_learning_on_cifar10.html - this is obsolete/deprecated?
• And finally ATLearn - there is no such thing in java - right?

[frankfliu]

@rd-peter-braun

I created a PR to update the jupyter notebook example: deepjavalibrary/djl-demo#473

[frankfliu]

@rd-peter-braun

Regarding ATLearn: We originally plan to add it in Java, however, currently we don't have anyone actively maintain it.