[Python] JSON DIctionary saved to S3 using S3FileSystem saved with "/ in data

Discussão em 'Python' iniciado por Stack, Agosto 14, 2025.

  1. Stack

    Stack Membro Participativo

    I am working on a project and trying to convert a csv into JSON Dictionary and saving that JSON data file to S3 bucket via S3FileSystem. If I save this JSON DIctionary to local machine, formate is correct but on S3 formate is like below

    S3 Bucket JSON Data

    "{\"Index\": \"GSPTSE\", \"Date\": \"1990-01-24\", \"Open\": 3760.399902, \"High\": 3763.399902, \"Low\": 3747.100098, \"Close\": 3761.5, \"Adj Close\": 3751.947998, \"Volume\": 44720000.0, \"CloseUSD\": 3122.045}"


    Local JSON File :-

    {"Index": "HSI", "Date": "12/31/1986", "Open": 2568.300049, "High": 2568.300049, "Low": 2568.300049, "Close": 2568.300049, "Adj Close": 2568.300049, "Volume": 0.0, "CloseUSD": 333.8790064}


    how to save JSON data in correct formate to S3?

    data saved localy as expected but JSON file on S3 has incorrect formate

    The Code used -

    Kafka Producer -

    while True:
    dict_stock = df.sample(1).to_dict(orient="records")[0]

    producer_2.send('demo_stock_market',value=dict_stock)


    Kafka Consumer to send JSON to S3

    for count, i in enumerate(consumer):
    with s3.open("s3://de-on-kafka-stock-raw-useast2-dev/kafka_stock_market_{}.json".format(count),'w',encoding='utf-8') as file:
    print(i.value)
    json.dump(i.value,file,indent=4)

    Continue reading...

Compartilhe esta Página