[Python] BigQuery - Apache Avro library failed to parse the header with the following error:...

I'm messing with avro file for the first time.

I get a lot of csv files with the encoded data and just made the python treatment of the data that should go to BigQuery. In the end I got the schema beat I received in avsc with csv data. So far so good.

Now I understand that I need to create the table in BigQuery that will receive this data. I simply tried to create a blank table and insert the received schema and could not. I get the following information: "Error while reading data, error message: The Apache Avro library failed to parse the header with the following error: Invalid data file. Magic does not match: gs://xxx/xxx/test.avsc". I tried to convert avsc to json through this online page: https://json-schema-validator.herokuapp.com/avro.jsp however unsuccessfully.

Is this the process path? As I can generate the records after decoding I understand that the schema is not incorrect. So I doubt if I'm doing the right thing.

Continue reading...