Criei um script de treinamento de uma rede GAN que roda normalmente em CPU, mas quando eu tento rodar em GPU eu tenho um erro, pela saída acredito que seja uma biblioteca/arquivo faltante ou um problema com o JIT mais já pesquisei bastante e ainda não descobri como resolver. No código o tensorflow reconhece a GPU Created device /device:GPU:0 with 5592 MB memory: -> device: 0, name: NVIDIA GeForce RTX 3060 Ti, pci bus id: 0000:01:00.0, compute capability: 8.6 A seguir o erro /home/erick/miniconda3/lib/python3.12/site-packages/keras/src/layers/core/dense.py:87: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead. super().__init__(activity_regularizer=activity_regularizer, **kwargs) WARNING: All log messages before absl::InitializeLog() is called are written to STDERR W0000 00:00:1741627981.458921 42705 gpu_backend_lib.cc:579] Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice. This may result in compilation or runtime failures, if the program we try to run uses routines from libdevice. Searched for CUDA in the following directories: ./cuda_sdk_lib ipykernel_launcher.runfiles/cuda_nvcc ipykern/cuda_nvcc /usr/local/cuda /home/erick/miniconda3/lib/python3.12/site-packages/tensorflow/python/platform/../../../nvidia/cuda_nvcc /home/erick/miniconda3/lib/python3.12/site-packages/tensorflow/python/platform/../../../../nvidia/cuda_nvcc /home/erick/miniconda3/lib/python3.12/site-packages/tensorflow/python/platform/../../cuda . You can choose the search directory by setting xla_gpu_cuda_data_dir in HloModule's DebugOptions. For most apps, setting the environment variable XLA_FLAGS=--xla_gpu_cuda_data_dir=/path/to/cuda will work. W0000 00:00:1741627982.194819 42599 gpu_backend_lib.cc:617] libdevice is required by this HLO module but was not found at ./libdevice.10.bc error: libdevice not found at ./libdevice.10.bc 2025-03-10 14:33:02.196119: E tensorflow/compiler/mlir/tools/kernel_gen/tf_framework_c_interface.cc:228] INTERNAL: Generating device code failed. 2025-03-10 14:33:02.197081: W tensorflow/core/framework/op_kernel.cc:1829] UNKNOWN: JIT compilation failed. 2025-03-10 14:33:02.197141: I tensorflow/core/framework/local_rendezvous.cc:405] Local rendezvous is aborting with status: UNKNOWN: JIT compilation failed. --------------------------------------------------------------------------- UnknownError Traceback (most recent call last) Cell In[10], line 6 3 extra_values = tf.constant([[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0]]) # Tensor de shape (1,2) 4 noise = tf.concat([tf.random.normal([1, noise_dim]), extra_values], axis=1) # Agora shape será (1, 102) ----> 6 generated_image = generator(noise, training=False) 8 plt.imshow(generated_image[0, :, :, 0], cmap='gray') File ~/miniconda3/lib/python3.12/site-packages/keras/src/utils/traceback_utils.py:122, in filter_traceback.<locals>.error_handler(*args, **kwargs) 119 filtered_tb = _process_traceback_frames(e.__traceback__) 120 # To get the full stack trace, call: 121 # `keras.config.disable_traceback_filtering()` --> 122 raise e.with_traceback(filtered_tb) from None 123 finally: 124 del filtered_tb File ~/miniconda3/lib/python3.12/site-packages/tensorflow/python/framework/ops.py:6002, in raise_from_not_ok_status(e, name) 6000 def raise_from_not_ok_status(e, name) -> NoReturn: 6001 e.message += (" name: " + str(name if name is not None else "")) -> 6002 raise core._status_to_exception(e) from None UnknownError: Exception encountered when calling BatchNormalization.call(). {{function_node __wrapped__Rsqrt_device_/job:localhost/replica:0/task:0/device:GPU:0}} JIT compilation failed. [Op:Rsqrt] name: Arguments received by BatchNormalization.call(): • inputs=tf.Tensor(shape=(1, 12544), dtype=float32) • training=False • mask=None Na mensagem ele indica erro com a camada BatchNormalization() mas mesmo retirando essas camadas o erro permanece Saída retirando as camadas de BatchNormalization() W0000 00:00:1741633586.368110 1208 gpu_backend_lib.cc:617] libdevice is required by this HLO module but was not found at ./libdevice.10.bc error: libdevice not found at ./libdevice.10.bc 2025-03-10 16:06:26.369008: E tensorflow/compiler/mlir/tools/kernel_gen/tf_framework_c_interface.cc:228] INTERNAL: Generating device code failed. 2025-03-10 16:06:26.369903: W tensorflow/core/framework/op_kernel.cc:1829] UNKNOWN: JIT compilation failed. 2025-03-10 16:06:26.369945: I tensorflow/core/framework/local_rendezvous.cc:405] Local rendezvous is aborting with status: UNKNOWN: JIT compilation failed. --------------------------------------------------------------------------- UnknownError Traceback (most recent call last) Cell In[11], line 2 1 discriminator = dis.make_discriminator_model(num_of_labels=num_of_classes) ----> 2 decision = discriminator([generated_image,extra_values]) 3 print (decision) File ~/miniconda3/lib/python3.12/site-packages/keras/src/utils/traceback_utils.py:122, in filter_traceback.<locals>.error_handler(*args, **kwargs) 119 filtered_tb = _process_traceback_frames(e.__traceback__) 120 # To get the full stack trace, call: 121 # `keras.config.disable_traceback_filtering()` --> 122 raise e.with_traceback(filtered_tb) from None 123 finally: 124 del filtered_tb File ~/miniconda3/lib/python3.12/site-packages/tensorflow/python/framework/ops.py:6002, in raise_from_not_ok_status(e, name) 6000 def raise_from_not_ok_status(e, name) -> NoReturn: 6001 e.message += (" name: " + str(name if name is not None else "")) -> 6002 raise core._status_to_exception(e) from None UnknownError: Exception encountered when calling BatchNormalization.call(). {{function_node __wrapped__Rsqrt_device_/job:localhost/replica:0/task:0/device:GPU:0}} JIT compilation failed. [Op:Rsqrt] name: Arguments received by BatchNormalization.call(): • inputs=tf.Tensor(shape=(1, 32), dtype=float32) • training=False • mask=None O código esta sendo executado em um ambiente conda rodando no WSL2, a GPU é uma RTX 3060ti Continue reading...
