Here is the situation: I have data saved into pandas HDF5 files. Some data is compressed using lzo and some using blosc:zstd. Under RHEL-7, I was able to read both types of files. Then, I was forcefully upgraded to RHEL-8 by our IT. Since then, I can only read blosc:zstd compressed files and attempts to read the lzo files give me the following: ... return pd.read_hdf(...) ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".../pyenv/versions/3.13.5_1/lib/python3.13/site-packages/pandas/io/pytables.py", line 457, in read_hdf return store.select( ~~~~~~~~~~~~^ key, ^^^^ ...<6 lines>... auto_close=auto_close, ^^^^^^^^^^^^^^^^^^^^^^ ) ^ File ".../pyenv/versions/3.13.5_1/lib/python3.13/site-packages/pandas/io/pytables.py", line 911, in select return it.get_result() ~~~~~~~~~~~~~^^ File ".../pyenv/versions/3.13.5_1/lib/python3.13/site-packages/pandas/io/pytables.py", line 2034, in get_result results = self.func(self.start, self.stop, where) File ".../pyenv/versions/3.13.5_1/lib/python3.13/site-packages/pandas/io/pytables.py", line 895, in func return s.read(start=_start, stop=_stop, where=_where, columns=columns) ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".../pyenv/versions/3.13.5_1/lib/python3.13/site-packages/pandas/io/pytables.py", line 4913, in read df = super().read(where=where, columns=columns, start=start, stop=stop) File ".../pyenv/versions/3.13.5_1/lib/python3.13/site-packages/pandas/io/pytables.py", line 4662, in read result = self._read_axes(where=where, start=start, stop=stop) File ".../pyenv/versions/3.13.5_1/lib/python3.13/site-packages/pandas/io/pytables.py", line 3845, in _read_axes values = selection.select() File ".../pyenv/versions/3.13.5_1/lib/python3.13/site-packages/pandas/io/pytables.py", line 5447, in select return self.table.table.read_where( ~~~~~~~~~~~~~~~~~~~~~~~~~~~^ self.condition.format(), start=self.start, stop=self.stop ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File ".../pyenv/versions/3.13.5_1/lib/python3.13/site-packages/tables/table.py", line 1644, in read_where p.nrow for p in self._where(condition, condvars, start, stop, step) ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "tables/tableextension.pyx", line 1047, in tables.tableextension.Row.__next__ File "tables/tableextension.pyx", line 1094, in tables.tableextension.Row._Row__next__indexed File "tables/tableextension.pyx", line 746, in tables.tableextension.Table._read_chunk tables.exceptions.HDF5ExtError: HDF5 error back trace File "H5D.c", line 1061, in H5Dread can't synchronously read data File "H5D.c", line 1008, in H5D__read_api_common can't read data File "H5VLcallback.c", line 2092, in H5VL_dataset_read_direct dataset read failed File "H5VLcallback.c", line 2048, in H5VL__dataset_read dataset read failed File "H5VLnative_dataset.c", line 373, in H5VL__native_dataset_read can't read data File "H5Dio.c", line 401, in H5D__read can't read data File "H5Dchunk.c", line 2919, in H5D__chunk_read unable to read raw data chunk File "H5Dchunk.c", line 4548, in H5D__chunk_lock data pipeline read failed File "H5Z.c", line 1354, in H5Z_pipeline required filter 'lzo' is not registered File "H5PLint.c", line 265, in H5PL_load can't find plugin. Check either HDF5_VOL_CONNECTOR, HDF5_PLUGIN_PATH, default location, or path set by H5PLxxx functions File "H5PLpath.c", line 804, in H5PL__find_plugin_in_path_table search in path /usr/local/hdf5/lib/plugin encountered an error File "H5PLpath.c", line 856, in H5PL__find_plugin_in_path can't open directory (/usr/local/hdf5/lib/plugin). Please verify its existence End of HDF5 error back trace Problems reading chunk records. I have lzo and lzo-devel installed. I have attempted to recompile python, to reinstall all the packages, and to compile HDF5 from source. Nothing of the above resulted in a different outcome. Could you, please, help me to understand how to enable reading of lzo compressed pandas HDF5 files again? Continue reading...
