[Python] Extract multiple strings from a column in a new column as a list in pyspark dataframe

Have a pyspark dataframe with a single column with below values (Only 2 records for example)-

+------------------------------------------------------------------------------------------------------+
|data |
+------------------------------------------------------------------------------------------------------+
{Job : "Job A", Condition: "Pass"},{Job : "Job B", Condition: "Pass"}
{Job : "Job C", Condition: "Pass"},{Job : "Job D", Condition: "Pass"},{Job : "Job E", Condition: "Pass"}
+------------------------------------------------------------------------------------------------------+


I am looking to extract the job names only in a list in a separate column.

Desired Outcome-

+-----------------------+
|data1 |
+-----------------------+
["Job A","Job B"]
["Job C","Job D","Job E"]
+-----------------------+


Any easy way i can do it in pyspark?

Continue reading...