[Python] How to solve ERROR: Could not build wheels for sparse-dot-topn-for-blocks, which is...

I want to perform a string grouping operation on a dataset of ~7000 variants. After much research, I've found the string-grouper library to be matching my use case. This is an explainer blog post by van den Berg.

However, I am unable to install the library. Here's the error that I am receiving:

Building wheels for collected packages: sparse-dot-topn-for-blocks
Building wheel for sparse-dot-topn-for-blocks (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for sparse-dot-topn-for-blocks (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [245 lines of output]
C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\dist.py:314: InformationOnly: Normalizing '0.3.1-3' to '0.3.1.post3'
self.metadata.version = self._normalize_version(self.metadata.version)
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-39
creating build\lib.win-amd64-cpython-39\sparse_dot_topn_for_blocks
copying sparse_dot_topn_for_blocks\awesome_cossim_topn.py -> build\lib.win-amd64-cpython-39\sparse_dot_topn_for_blocks
copying sparse_dot_topn_for_blocks\__init__.py -> build\lib.win-amd64-cpython-39\sparse_dot_topn_for_blocks
running egg_info
writing sparse_dot_topn_for_blocks.egg-info\PKG-INFO
writing dependency_links to sparse_dot_topn_for_blocks.egg-info\dependency_links.txt
writing requirements to sparse_dot_topn_for_blocks.egg-info\requires.txt
writing top-level names to sparse_dot_topn_for_blocks.egg-info\top_level.txt
reading manifest file 'sparse_dot_topn_for_blocks.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'sparse_dot_topn_for_blocks.egg-info\SOURCES.txt'
copying sparse_dot_topn_for_blocks\array_wrappers.pxd -> build\lib.win-amd64-cpython-39\sparse_dot_topn_for_blocks
copying sparse_dot_topn_for_blocks\array_wrappers.pyx -> build\lib.win-amd64-cpython-39\sparse_dot_topn_for_blocks
copying sparse_dot_topn_for_blocks\sparse_dot_topn.pyx -> build\lib.win-amd64-cpython-39\sparse_dot_topn_for_blocks
copying sparse_dot_topn_for_blocks\sparse_dot_topn_parallel.cpp -> build\lib.win-amd64-cpython-39\sparse_dot_topn_for_blocks
copying sparse_dot_topn_for_blocks\sparse_dot_topn_parallel.h -> build\lib.win-amd64-cpython-39\sparse_dot_topn_for_blocks
copying sparse_dot_topn_for_blocks\sparse_dot_topn_source.cpp -> build\lib.win-amd64-cpython-39\sparse_dot_topn_for_blocks
copying sparse_dot_topn_for_blocks\sparse_dot_topn_source.h -> build\lib.win-amd64-cpython-39\sparse_dot_topn_for_blocks
copying sparse_dot_topn_for_blocks\sparse_dot_topn_threaded.pyx -> build\lib.win-amd64-cpython-39\sparse_dot_topn_for_blocks
running build_ext
C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\normal\Lib\site-packages\Cython\Compiler\Main.py:381: FutureWarning: Cython directive 'language_le
vel' not set, using '3str' for now (Py3). This has changed from earlier releases! File: C:\Users\zaida\AppData\Local\Temp\pip-install-kbeq40ku\sparse-dot-topn-for-blocks_0e7bc776223c4685b4535c096adeb9be\sparse_dot_topn_for_blocks\array_wrappers.pxd
tree = Parsing.p_module(s, pxd, full_module_name)
Compiling ./sparse_dot_topn_for_blocks/array_wrappers.pyx because it changed.
[1/1] Cythonizing ./sparse_dot_topn_for_blocks/array_wrappers.pyx
building 'sparse_dot_topn_for_blocks.array_wrappers' extension
creating build\temp.win-amd64-cpython-39
creating build\temp.win-amd64-cpython-39\Release
creating build\temp.win-amd64-cpython-39\Release\sparse_dot_topn_for_blocks
"C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.38.33130\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\
Users\zaida\DataspellProjects\test_string_grouper_2\venv\include -IC:\Users\zaida\AppData\Local\Programs\Python\Python39\include -IC:\Users\zaida\AppData\Local\P
rograms\Python\Python39\Include -IC:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\normal\Lib\site-packages\numpy\core\include "-IC:\Program Files (x86)\
Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.38.33130\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\incl
ude" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program File
s (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows
Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" /EHsc /Tp./sparse_dot_topn_for_blocks/array_wrappers.cpp /Fobuild\temp.win-amd64-cpython-39\Release\./sparse_dot_topn_for_blocks/array_wrappers.obj -Ox
array_wrappers.cpp
"C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.38.33130\bin\HostX86\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MAN
IFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\zaida\DataspellProjects\test_string_grouper_2\venv\libs /LIBPATH:C:\Users\zaida\AppData\Local\Programs\Python\
Python39\libs /LIBPATH:C:\Users\zaida\AppData\Local\Programs\Python\Python39 /LIBPATH:C:\Users\zaida\DataspellProjects\test_string_grouper_2\venv\PCbuild\amd64 "
/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.38.33130\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK
\4.8\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\
um\x64" /EXPORTyInit_array_wrappers build\temp.win-amd64-cpython-39\Release\./sparse_dot_topn_for_blocks/array_wrappers.obj /OUT:build\lib.win-amd64-cpython-39
\sparse_dot_topn_for_blocks\array_wrappers.cp39-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-39\Release\./sparse_dot_topn_for_blocks\array_wrappers.cp39-win_amd64.lib
Creating library build\temp.win-amd64-cpython-39\Release\./sparse_dot_topn_for_blocks\array_wrappers.cp39-win_amd64.lib and object build\temp.win-amd64-cpython-39\Release\./sparse_dot_topn_for_blocks\array_wrappers.cp39-win_amd64.exp
Generating code
Finished generating code
C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\normal\Lib\site-packages\Cython\Compiler\Main.py:381: FutureWarning: Cython directive 'language_le
vel' not set, using '3str' for now (Py3). This has changed from earlier releases! File: C:\Users\zaida\AppData\Local\Temp\pip-install-kbeq40ku\sparse-dot-topn-for-blocks_0e7bc776223c4685b4535c096adeb9be\sparse_dot_topn_for_blocks\sparse_dot_topn.pyx
tree = Parsing.p_module(s, pxd, full_module_name)

Error compiling Cython file:
------------------------------------------------------------
...
# April 14, 2021

# distutils: language = c++

from libcpp.vector cimport vector
from array_wrappers cimport ArrayWrapper_int, ArrayWrapper_float, ArrayWrapper_double
^
------------------------------------------------------------

sparse_dot_topn_for_blocks\sparse_dot_topn.pyx:23:0: 'array_wrappers.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
# April 14, 2021

# distutils: language = c++

from libcpp.vector cimport vector
from array_wrappers cimport ArrayWrapper_int, ArrayWrapper_float, ArrayWrapper_double
^
------------------------------------------------------------

sparse_dot_topn_for_blocks\sparse_dot_topn.pyx:23:0: 'array_wrappers\ArrayWrapper_int.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
# April 14, 2021

# distutils: language = c++

from libcpp.vector cimport vector
from array_wrappers cimport ArrayWrapper_int, ArrayWrapper_float, ArrayWrapper_double
^
------------------------------------------------------------

sparse_dot_topn_for_blocks\sparse_dot_topn.pyx:23:0: 'array_wrappers\ArrayWrapper_float.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
# April 14, 2021

# distutils: language = c++

from libcpp.vector cimport vector
from array_wrappers cimport ArrayWrapper_int, ArrayWrapper_float, ArrayWrapper_double
^
------------------------------------------------------------

sparse_dot_topn_for_blocks\sparse_dot_topn.pyx:23:0: 'array_wrappers\ArrayWrapper_double.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
) except +;

cpdef ArrayWrapper_template(vector[float_ft] vCx):
# raise Exception("In sparse_dot_topn.pyx")
if float_ft is float:
return ArrayWrapper_float(vCx)
^
------------------------------------------------------------

sparse_dot_topn_for_blocks\sparse_dot_topn.pyx:116:9: 'ArrayWrapper_float' is not a constant, variable or function identifier

Error compiling Cython file:
------------------------------------------------------------
...
cpdef ArrayWrapper_template(vector[float_ft] vCx):
# raise Exception("In sparse_dot_topn.pyx")
if float_ft is float:
return ArrayWrapper_float(vCx)
elif float_ft is double:
return ArrayWrapper_double(vCx)
^
------------------------------------------------------------

sparse_dot_topn_for_blocks\sparse_dot_topn.pyx:118:9: 'ArrayWrapper_double' is not a constant, variable or function identifier

Error compiling Cython file:
------------------------------------------------------------
...

if nnz_max_is_too_small:

# raise Exception("In sparse_dot_topn.pyx")

c_indices = np.asarray(ArrayWrapper_int(vCj)).squeeze(axis=0)
^
------------------------------------------------------------

sparse_dot_topn_for_blocks\sparse_dot_topn.pyx:209:25: 'ArrayWrapper_int' is not a constant, variable or function identifier

Error compiling Cython file:
------------------------------------------------------------
...

if nnz_max_is_too_small:

# raise Exception("In sparse_dot_topn.pyx")

c_indices = np.asarray(ArrayWrapper_int(vCj)).squeeze(axis=0)
^
------------------------------------------------------------

sparse_dot_topn_for_blocks\sparse_dot_topn.pyx:209:25: 'ArrayWrapper_int' is not a constant, variable or function identifier

Error compiling Cython file:
------------------------------------------------------------
...

if nnz_max_is_too_small:

# raise Exception("In sparse_dot_topn.pyx")

c_indices = np.asarray(ArrayWrapper_int(vCj)).squeeze(axis=0)
^
------------------------------------------------------------

sparse_dot_topn_for_blocks\sparse_dot_topn.pyx:296:25: 'ArrayWrapper_int' is not a constant, variable or function identifier

Error compiling Cython file:
------------------------------------------------------------
...

if nnz_max_is_too_small:

# raise Exception("In sparse_dot_topn.pyx")

c_indices = np.asarray(ArrayWrapper_int(vCj)).squeeze(axis=0)
^
------------------------------------------------------------

sparse_dot_topn_for_blocks\sparse_dot_topn.pyx:296:25: 'ArrayWrapper_int' is not a constant, variable or function identifier
Compiling ./sparse_dot_topn_for_blocks/sparse_dot_topn.pyx because it changed.
[1/1] Cythonizing ./sparse_dot_topn_for_blocks/sparse_dot_topn.pyx
Traceback (most recent call last):
File "C:\Users\zaida\DataspellProjects\test_string_grouper_2\venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
main()
File "C:\Users\zaida\DataspellProjects\test_string_grouper_2\venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "C:\Users\zaida\DataspellProjects\test_string_grouper_2\venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 251, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\build_meta.py", line 404, in build_wheel
return self._build_with_temp_dir(
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\build_meta.py", line 389, in _build_with_temp_dir
self.run_setup()
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\build_meta.py", line 311, in run_setup
exec(code, locals())
File "<string>", line 65, in <module>
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\__init__.py", line 103, in setup
return distutils.core.setup(**attrs)
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
return run_commands(dist)
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
dist.run_commands()
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 969, in run_commands
self.run_command(cmd)
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\dist.py", line 963, in run_command
super().run_command(command)
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
cmd_obj.run()
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\wheel\bdist_wheel.py", line 368, in run
self.run_command("build")
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\dist.py", line 963, in run_command
super().run_command(command)
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
cmd_obj.run()
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\_distutils\command\build.py", line 131, in run
self.run_command(cmd_name)
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\dist.py", line 963, in run_command
super().run_command(command)
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
cmd_obj.run()
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\command\build_ext.py", line 88, in run
_build_ext.run(self)
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 345, in run
self.build_extensions()
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 467, in build_extensions
self._build_extensions_serial()
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 493, in _build_extensions_serial
self.build_extension(ext)
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\overlay\Lib\site-packages\setuptools\command\build_ext.py", line 249, in build_extension
_build_ext.build_extension(self, ext)
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\normal\Lib\site-packages\Cython\Distutils\build_ext.py", line 130, in build_extension
new_ext = cythonize(
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\normal\Lib\site-packages\Cython\Build\Dependencies.py", line 1154, in cythonize
cythonize_one(*args)
File "C:\Users\zaida\AppData\Local\Temp\pip-build-env-w3s5dhxf\normal\Lib\site-packages\Cython\Build\Dependencies.py", line 1321, in cythonize_one
raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: ./sparse_dot_topn_for_blocks/sparse_dot_topn.pyx
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for sparse-dot-topn-for-blocks
Failed to build sparse-dot-topn-for-blocks
ERROR: Could not build wheels for sparse-dot-topn-for-blocks, which is required to install pyproject.toml-based projects



It was earlier asking me to upgrade Microsoft Build tools which I did. Now it is giving the above error.

I've read earlier posts of users facing a similar error but didn't find a solution.

Continue reading...