I am working on a huge parquet file containing over 30 million rows. I only need a small portion of it and wish pick a number of randomly selected rows. When i inspect the file's metadata, there's only one row_group which means I can't use pyarrow's read_row_groups to select random groups. pf = ParquetFile('Flight_Delay.parquet') created_by: parquet-cpp-arrow version 6.0.1 num_columns: 30 num_rows: 30132672 num_row_groups: 1 format_version: 1.0 serialized_size: 15701 Continue reading...
