[Python] Using Ray on vertex to for running ollama qwen2.5 model in current runtime

I have a dataset with 100 rows.I am taking inputs from each row and feeding it to local ollama runtime to get the desired result on vertex ai colab. All the files are stored in local runtime. Original code is running perfectly fine but when I try to implement ray into it it throws some errors.

My original code is as follows for normal execution. Note this is not the whole code but a part of it

import threading
import subprocess
import time
def run_ollama_serve():
subprocess.Popen(["ollama", "serve"])
thread = threading.Thread(target=run_ollama_serve)
thread.start()
time.sleep(5)

!ollama pull qwen2.5:7b
llm = OllamaLLM(model='qwen2.5:7b' , temperature=0.6)
chain = LLMChain(llm=llm , prompt=prompt)

import pandas as pd
data = pd.read_csv('/content/sample_data.csv')
data

results = []
import json
from tqdm import tqdm
for i , (index, row) in enumerate(tqdm(data[:10].iterrows())):
id = row["ID"]
url = row["url"]
keywords = row["keywords"]

predictions = chain({
"id": id,
"url": url,
"keywords": keywords,
"category_json": category_json
})




I converted it to

import ray
import json
from tqdm import tqdm



@ray.remote
def process_row(index, row, category_json):
id = row["ID"]
url = row["url"]
keywords = row["keywords"]

predictions = chain({
"id": id,
"url": url,
"keywords": keywords,
"category_json": category_json
})

return {"id": id, "predictions": predictions}

def run_parallel_processing(data, category_json):
futures = [process_row.remote(index, row, category_json) for index, row in data[:10].iterrows()]
results = ray.get(futures)
return results

results = run_parallel_processing(data, category_json)

ray.shutdown()


I am getting an error

---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-30-8fb1909cf58e> in <cell line: 27>()
25 return results
26
---> 27 results = run_parallel_processing(data, category_json)
28
29 ray.shutdown()

16 frames
/usr/local/lib/python3.10/dist-packages/ray/cloudpickle/cloudpickle.py in dump(self, obj)
1243 def dump(self, obj):
1244 try:
-> 1245 return super().dump(obj)
1246 except RuntimeError as e:
1247 if len(e.args) > 0 and "recursion" in e.args[0]:

TypeError: cannot pickle '_thread.RLock' object


I want to run my code properly on ray but facing these issues while using ollama. Details about the ray and python environ are


Ray: 2.33.0
Python: 3.10.16
This process on Ray Client was started with:
Ray: 2.33.0
Python: 3.10.12

Continue reading...