The current pl.DataFrame(iterable, infer_schema_length=None) or pl.from_dicts(iterable, infer_schema_length=None) is unreliable and documentation is also ambiguous (reads: The maximum number of rows to scan for schema inference. If set to None, the full data may be scanned (this is slow).) For my example it's not scanning full data hence not including all the columns. Is there any way in polars which can reliably reads the iterable either full length or without inferring the data? Pandas is correctly infering (Reading all the rows) Currently I'm using return pl.from_dicts(iterable, infer_schema_length=999999999) but there should be a better way. Continue reading...
