[Python] How to Resolve OSError: Read-only File System Error in AWS Lambda when Using CrewAI?

I am encountering an issue when trying to run a Lambda function via API Gateway in AWS, while using CrewAI. The error I am receiving is as follows:

[ERROR] OSError: [Errno 30] Read-only file system: '/home/sbx_user1051'
Traceback (most recent call last):
File "/var/lang/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/var/task/src/app.py", line 7, in <module>
from main import process_message
File "/var/task/src/main.py", line 5, in <module>
from tools import Tools
File "/var/task/src/tools.py", line 1, in <module>
from crewai_tools import WebsiteSearchTool, TXTSearchTool
File "/var/task/crewai_tools/__init__.py", line 1, in <module>
from .tools import (
File "/var/task/crewai_tools/tools/__init__.py", line 4, in <module>
from .code_docs_search_tool.code_docs_search_tool import CodeDocsSearchTool
File "/var/task/crewai_tools/tools/code_docs_search_tool/code_docs_search_tool.py", line 3, in <module>
from embedchain.models.data_type import DataType
File "/var/task/embedchain/__init__.py", line 5, in <module>
from embedchain.app import App # noqa: F401
File "/var/task/embedchain/app.py", line 42, in <module>
from mem0 import Memory
File "/var/task/mem0/__init__.py", line 5, in <module>
from mem0.client.main import MemoryClient, AsyncMemoryClient # noqa
File "/var/task/mem0/client/main.py", line 9, in <module>
from mem0.memory.setup import get_user_id, setup_config
File "/var/task/mem0/memory/setup.py", line 8, in <module>
os.makedirs(mem0_dir, exist_ok=True)
File "<frozen os>", line 215, in makedirs
File "<frozen os>", line 225, in makedirs


The issue arises because the system tries to create a directory and a .sqlite3 file in a path that is not writable due to AWS Lambda’s read-only file system. From my understanding, AWS Lambda only allows writing within the /tmp directory.

I followed guidance from an issue thread on GitHub here and set the environment variable CREWAI_STORAGE_DIR='/tmp' to point to the /tmp directory. Despite this, I am still facing the same issue.

I've tried multiple configurations and approaches, but nothing has worked so far. If anyone has encountered this problem before or has any advice on how to resolve this issue, your help would be greatly appreciated.

This is quite urgent, and I am looking forward to any guidance or suggestions.

I tried defining the directory to be used with the following code:

mem0_dir = os.getenv('CREWAI_STORAGE_DIR', '/tmp')
os.makedirs(mem0_dir, exist_ok=True)
setup_config(user_dir=mem0_dir)


I also attempted to set the environment variable as mentioned in the GitHub issue here:

CREWAI_STORAGE_DIR='/tmp'


My expectation is to be able to make the request to the API Gateway and ensure that anything that needs to be created is done inside the /tmp directory.

Continue reading...