[Python] Random errors when using xarray on netCDF files

I'm using xarray to extract data from various netcdf files to analyses and plot different meteorological data. This is an example of a script I'm using for this.

import xarray as xr
import numpy as np
import matplotlib.pyplot as plt
from skimage.measure import block_reduce

data = xr.open_mfdataset("filepath\*.nc", parallel=True)

tas = np.array(data['tas'])
time = np.array(data['time'])

x = np.array(data['projection_x_coordinate'])
y = np.array(data['projection_y_coordinate'])

lat = np.array(data['latitude'])
lon = np.array(data['longitude'])

temp = block_reduce(tas, block_size=(12,1,1), func=np.mean, cval=np.mean(tas))

time = np.arange(1884,2023)


plt.plot(time, temp[:, 23, 62], 'r', label='London')
plt.plot(time, temp[:, 22, 55], 'c', label='Headley')
plt.plot(time, temp[:, 23, 59], 'g', label='Egham')
plt.legend()
plt.xlabel('Time (1884-2022)')
plt.ylabel('Temperature [C]')
plt.title('Heat Island Effect for Average Monthly Temperature')


My issue is is that the code seems to work and not work randomly. Sometimes I get the expected output and sometimes I get errors such as:

OSError: [Errno -51] NetCDF: Unknown file format: b'C:filepath\\tas_hadukgrid_uk_12km_mon_197201-197212.nc'

RuntimeError: NetCDF: Not a valid ID

KeyError: [<class 'netCDF4._netCDF4.Dataset'>, ('C:filepath\\tas_hadukgrid_uk_12km_mon_197401-197412.nc',), 'r', (('clobber', True), ('diskless', False), ('format', 'NETCDF4'), ('persist', False)), '621f2ad1-8dca-4c52-83d3-1f7fe6b4972d']


I also get warning such as:

C:\Users\miniconda3\envs\spyder-env\Lib\site-packages\paramiko\transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated
"class": algorithms.Blowfish,


And the kernel restarts.

I'm using spyder as an IDE but the problems persist in Jupyter and VScode. I have uninstalled and reinstalled all my packages multiple times as well as miniconda. I have also factory reset my laptop to make sure there were no conda files interfering with fresh installs but nothing seems to work.

Any help at all would be appreciated.

Continue reading...