This question came up while I was saving a large number of model-inferred embeddings to plain text. To do so, I needed to convert lists of float embeddings into strings, and I found this conversion to be surprisingly time-consuming. Inspired by this discussion, I benchmarked four different methods for converting float arrays to strings. Surprisingly, orjson performed the best—even though it's a third-party JSON serialization library. This got me wondering: Is there a native Python method that can achieve performance comparable to orjson for converting lists of floats to strings? Below are the commands I used for profiling, along with the results: $ python -m pyperf timeit --fast -s 'x = [3141592653589793] * 100' 'str(x)' Mean +- std dev: 4.79 us +- 0.06 us $ python -m pyperf timeit --fast -s 'from orjson import dumps; x = [3141592653589793] * 100' 'dumps(x)' Mean +- std dev: 2.70 us +- 0.02 us $ python -m pyperf timeit --fast -s 'from json import dumps; x = [3141592653589793] * 100 ' 'dumps(x)' Mean +- std dev: 8.03 us +- 0.31 us $ python -m pyperf timeit --fast -s 'x = [3141592653589793] * 100' '"{}".format(x)' Mean +- std dev: 4.94 us +- 0.16 us Continue reading...
