[Python] Cannot figure out why previously working FastAI learner inference now fails with CUDA...

I have a Jetson Orin and the internal drive failed. I had to flash from scratch.

I had some working Fastai 2.0 based code with a PyTorch version ~1 year ago and also a previous version of CUDA, I believe some version of 11.x. The Jetpack version was 5.x.

I now flashed the latest version of Jetpack on the Orin with Ubuntu 22.04. It is now Jetpack 6.0 with CUDA 12.

PyTorch has CUDA support installed:

Python 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>>
>>> device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
>>> print(f"Using device: {device}")
Using device: cuda


Here are the versions of PyTorch and FastAI:

>>> import fastai
>>> print(fastai.__version__)
2.7.17
>>> import torch
>>> print(torch.__version__)
2.4.0a0+07cecf4168.nv24.05


My previously working Image classification code now fails at the inference call on this line:

# Perform batch inference
torch.cuda.synchronize()
start_inference = time.time()
with torch.no_grad():

# Put the model into inference mode
accumulated_output = []
logger.info("Setting inference mode now ...")
with torch.inference_mode():
for data in infer_loader:
data = data.cuda()
logger.info(f"Calling learner now: {data} ...")
logger.info(f"Learner is: {learner}")
output = learner(data)
logger.info("Done calling learner ...")
for tensor_output in output.data.cpu():
accumulated_output.append(tensor_output)

# Time the inference work
torch.cuda.synchronize()
end_inference = time.time()
logger.info(f"Inference time: {end_inference - start_inference}")


The error is caught in my exception handle due to this line:

output = learner(data)


The error message is:

Failed to perform inference: hasattr(): attribute name must be string


A more complete output is shown here:

[2024-10-07 18:44:06,229] __main__ {process_frame_utils.py:258} INFO - Creating data loader now ...
[2024-10-07 18:44:06,230] __main__ {process_frame_utils.py:274} INFO - Setting inference mode now ...
[2024-10-07 18:44:06,757] __main__ {process_frame_utils.py:278} INFO - Calling learner now: tensor([[[[-2.1179, -2.1179, -2.1179, ..., -2.1179, -2.1179, -2.1179],
...
[-1.8044, -1.8044, -1.8044, ..., -1.8044, -1.8044, -1.8044]]]], device='cuda:0') ...
[2024-10-07 18:44:06,759] __main__ {process_frame_utils.py:279} INFO - Learner is: <fastai.learner.Learner object at 0xfffe851c4490>
[2024-10-07 18:44:06,798] __main__ {process_frame_utils.py:370} ERROR - Error: Failed to perform inference: hasattr(): attribute name must be string


I've tried retraining my model several times. Something has changed in the libraries between my previously working configuration and now, however I have been unable to figure out what the error means.

How can I fix this?

Continue reading...