[Python] PyTorch and Streamlit interaction error: "Tried to instantiate class...

I'm developing an application that uses Streamlit and PyTorch (through the transformers library) to analyze system requirements with a language model. However, I'm encountering the following recurring error:

Examining the path of torch.classes raised:
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.12/site-packages/streamlit/web/bootstrap.py", line 347, in run
if asyncio.get_running_loop().is_running():
^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: no running event loop

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.12/site-packages/streamlit/watcher/local_sources_watcher.py", line 217, in get_module_paths
potential_paths = extract_paths(module)
^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/.local/lib/python3.12/site-packages/streamlit/watcher/local_sources_watcher.py", line 210, in <lambda>
lambda m: list(m.__path__._path),
^^^^^^^^^^^^^^^^
File "/home/ubuntu/.local/lib/python3.12/site-packages/torch/_classes.py", line 13, in __getattr__
proxy = torch._C._get_custom_class_python_wrapper(self.name, attr)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Tried to instantiate class '__path__._path', but it does not exist! Ensure that it is registered via torch::class_


The application works correctly, but these errors appear constantly in the terminal. I've already tried:

1. Using lazy imports for PyTorch modules
2. Setting environment variables:
   export TOKENIZERS_PARALLELISM=false
   export PYTHONWARNINGS=ignore::FutureWarning
3. Using a custom startup script
4. Moving the model loading to a separate function with lazy initialization

My code for loading the model is structured like this:

import re
import json
import numpy as np
from typing import Dict, List, Optional
import os

# Désactiver les avertissements de PyTorch avec Streamlit
os.environ["TOKENIZERS_PARALLELISM"] = "false"

# Chargement différé pour éviter les conflits entre PyTorch et Streamlit
def _import_transformers():
try:
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
return AutoModelForCausalLM, AutoTokenizer, pipeline
except ImportError:
print("Transformers n'est pas installé. Mode simulation activé.")
return None, None, None

# Configuration globale
MODEL_NAME = "TinyLlama/TinyLlama-1.1B-Chat-v1.0" # Modèle plus léger et rapide
MAX_NEW_TOKENS = 512
DEVICE = "cpu" # Utiliser "cuda" si vous avez un GPU

# Initialiser le LLM une seule fois (lazy loading)
_llm = None
_tokenizer = None
_model_loading_attempted = False

def _get_llm():
"""Initialise et retourne le modèle LLM (lazy loading)"""
global _llm, _tokenizer, _model_loading_attempted

# Si on a déjà essayé de charger le modèle sans succès, ne pas réessayer
if _model_loading_attempted and _llm is None:
return None, None

# Si le modèle n'est pas encore chargé, le charger
if _llm is None:
try:
print("Chargement du modèle LLM... (première utilisation)")

# Import des modules nécessaires seulement quand on en a besoin
AutoModelForCausalLM, AutoTokenizer, _ = _import_transformers()
if AutoModelForCausalLM is None:
_model_loading_attempted = True
return None, None

_tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
_llm = AutoModelForCausalLM.from_pretrained(
MODEL_NAME,
device_map=DEVICE,
torch_dtype="auto",
low_cpu_mem_usage=True, # Réduire l'utilisation de la mémoire CPU
# Ajouter des options pour réduire l'utilisation de mémoire si nécessaire :
# load_in_8bit=True, # Pour la quantification 8-bit
# Trust remote code est nécessaire pour certains modèles
trust_remote_code=True
)

# Vérifier si le GPU est disponible
if DEVICE == "cuda":
import torch
if torch.cuda.is_available():
print(f"Device set to use {torch.cuda.get_device_name(0)}")
else:
print("CUDA demandé mais non disponible, utilisation du CPU à la place")

print(f"Modèle {MODEL_NAME} chargé avec succès!")
_model_loading_attempted = True
except Exception as e:
print(f"Erreur lors du chargement du modèle: {e}")
print("Utilisation du mode de simulation à la place.")
_model_loading_attempted = True
return None, None

return _llm, _tokenizer


Environment:

• Python 3.12
• Streamlit 1.45.1
• PyTorch (latest version installed)
• Transformers (latest version)
• Ubuntu 22.04
• CUDA 12.2

How can I eliminate these compatibility errors between Streamlit's hot-reloading system and PyTorch? It seems to be an issue related to Streamlit's introspection of modules loaded in memory.

Is there any configuration in Streamlit to disable inspection of certain modules, or any way to make PyTorch more compatible with this type of introspection?

Continue reading...