[Python] Does bigquery.LoadJobConfig() and file loads method of apache beam write to bigquery...

Discussão em 'Python' iniciado por Stack, Abril 11, 2025.

  1. Stack

    Stack Membro Participativo

    Does bigquery.LoadJobConfig() and file loads method of apache beam write to bigquery method are same

    write_to_bq = (
    csv_data
    | "Write to BigQuery" >> WriteToBigQuery(
    table= fetch_table,
    schema='SCHEMA_AUTODETECT',
    write_disposition=beam.io.BigQueryDisposition.WRITE_APPEND,
    create_disposition=beam.io.BigQueryDisposition.CREATE_IF_NEEDED,
    method=beam.io.WriteToBigQuery.Method.FILE_LOADS,
    ignore_unknown_columns=True,
    )
    )


    And this -

    job_config = bigquery.LoadJobConfig(
    source_format=bigquery.SourceFormat.NEWLINE_DELIMITED_JSON,
    autodetect=True,
    )

    Continue reading...

Compartilhe esta Página