[Python] MongoDB-PySpark: StringType has no matching BsonValue

I'm trying to write data to Mongodb, and I'm having the following error:

StringType has no matching BsonValue. Error: For input string: "457133333333672364913"


while saving the dataframe to mongodb using pyspark.

To save the dataframe I'm using:

(df
.write
.format("mongodb")
.mode("append")
.option("convertJson", "any")
.option("database", "test")
.option("collection", "mongo_collection")
.save())


The column giving the error is an ArrayType(StringType())

Using Python3.9 and pyspark 3.4.0

I've searched online but I'm not finding anything related.

Continue reading...