1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. Anuncie Aqui
    Anuncie aqui você Também: fdantas@4each.com.br

[Python] Transfer and write Parquet with python and pandas got timestamp error

Discussão em 'Python' iniciado por Stack, Setembro 18, 2025.

  1. Stack

    Stack Membro Participativo

    I tried to concat() two parquet file with pandas in python .
    It can work , but when I try to write and save the Data frame to a parquet file ,it display the error :

    ArrowInvalid: Casting from timestamp[ns] to timestamp[ms] would lose data:


    I checked the doc. of pandas, it default the timestamp syntax in ms when write the parquet file.
    How can I white the parquet file with used schema after concat?
    Here is my code:

    import pandas as pd

    table1 = pd.read_parquet(path= ('path.parquet'),engine='pyarrow')
    table2 = pd.read_parquet(path= ('path.parquet'),engine='pyarrow')

    table = pd.concat([table1, table2], ignore_index=True)
    table.to_parquet('./file.gzip', compression='gzip')

    Continue reading...

Compartilhe esta Página