How to export a PyTorch model to MATLAB? - matlab

Question

I have created and trained a model in PyTorch, and I would like to be able to use the trained model within a MATLAB program. Based on this post I have been exporting the model to ONNX and then attempting to load the ONNX model in MATLAB. I followed this pytorch tutorial on how to export a model to ONNX.
However, I get this error
Error using nnet.internal.cnn.onnx.importONNXNetwork>iHandleTranslationIssues
Unable to import the network because of the following issues:
1 operator(s) : Unable to create an input layer for ONNX input #1 (with name 'input') because its data format is unknown or not supported as a MATLAB input layer.
If you know the input format, pass it by using the "InputDataFormats" parameter.
The input shape declared in the ONNX file is '(batch_size, 12)'.
1 operator(s) : Unable to create an output layer for ONNX network output #1 (with name 'output') because its data format is unknown or not supported as a MATLAB
output layer. If you know the output format, pass it using the 'OutputDataFormats' parameter.
To import the ONNX network as a dlnetwork, set the 'TargetNetwork' value to 'dlnetwork'.
To import the ONNX network as a layer graph with weights, use importONNXLayers.
To import the ONNX network as a function, use importONNXFunction.
Error in nnet.internal.cnn.onnx.importONNXNetwork (line 37)
iHandleTranslationIssues(translationIssues);
Error in importONNXNetwork (line 113)
Network = nnet.internal.cnn.onnx.importONNXNetwork(modelfile, varargin{:});
Here is a minimal example to create the error
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import torch.onnx
class FFNN(nn.Module):
def __init__(self):
super(FFNN, self).__init__()
input_size = 12
self.layer1 = nn.Linear(input_size, 24)
self.layer2 = nn.Linear(24, 24)
self.layer3 = nn.Linear(24, 12)
self.norm1 = nn.BatchNorm1d(12, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
self.layer4 = nn.Linear(12, 6)
self.layer5 = nn.Linear(6, 1)
def forward(self, x):
x = F.relu(self.layer1(x))
x = F.relu(self.layer2(x))
x = self.norm1(self.layer3(x))
x = F.relu(self.layer4(x))
out = self.layer5(x)
return out
net = FFNN()
net.eval()
batch_size = 1
input_size = 12
x = torch.randn(batch_size, input_size, requires_grad=True) # random model input for onnx
out = net(x)
input_names = ['x1', 'x2', 'x3', 'x4', 'x5', 'x6', 'x7', 'Qm', 'l1', 'uPre', 'basal', 'dG']
output_names = ['output']
# Export the model
torch.onnx.export(net, # model being run
x, # model input (or a tuple for multiple inputs)
'model.onnx', # where to save the model (can be a file or file-like object)
export_params=True, # store the trained parameter weights inside the model file
opset_version=10, # the ONNX version to export the model to
do_constant_folding=True, # whether to execute constant folding for optimization
input_names = ['input'], # the model's input names
output_names = ['output'], # the model's output names
dynamic_axes={'input' : {0 : 'batch_size'}, # variable length axes
'output' : {0 : 'batch_size'}})
nnMPC = importONNXNetwork("model.onnx"); % produces error
However I can validate the model in ONNX in python and it loads correctly.
So I think the problem is how I am loading it into MATLAB.
(my validation code)
import onnx
import onnxruntime
import numpy as np
onnx_model = onnx.load('model.onnx')
onnx.checker.check_model(onnx_model)
ort_session = onnxruntime.InferenceSession('model.onnx')
def to_numpy(tensor):
return tensor.detach().cpu().numpy() if tensor.requires_grad else tensor.cpu().numpy()
# compute ONNX Runtime output prediction
ort_inputs = {ort_session.get_inputs()[0].name: to_numpy(x)}
ort_outs = ort_session.run(None, ort_inputs)
# compare ONNX Runtime and PyTorch results
np.testing.assert_allclose(to_numpy(out), ort_outs[0], rtol=1e-03, atol=1e-05)
print("Exported model has been tested with ONNXRuntime, and the result looks good!")