[Python] Django/Celery SQLite database locked on concurrent access

I have a local Django 5.1/Celery 5.4 project that is using SQLite. I am the only user.

Certain model saves trigger a Celery task that queries (SELECT) for the updated record (using the Django ORM), then runs an API call to update a remote record based on the local data, and then runs another UPDATE locally. The task wraps all this inside of with transaction.atomic():.

(The Celery worker is configured to run tasks in serial.)

While this task is running, any attempts to write to the database result in a "database is locked" OperationalError.

I have configured Django/SQLite with the latest "production-ready" settings:

DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': DB_DIR / 'db.sqlite3',
'OPTIONS': {
'init_command': """
PRAGMA foreign_keys=ON;
PRAGMA journal_mode=WAL;
PRAGMA synchronous=NORMAL;
PRAGMA busy_timeout = 5000;
PRAGMA temp_store = MEMORY;
PRAGMA mmap_size=134217728;
PRAGMA journal_size_limit=67108864;
PRAGMA cache_size=2000;
""",
'transaction_mode': 'IMMEDIATE',
'timeout': 20,
},
},
}


I was under the impression that with these settings, concurrent access was possible. "SQLite in Production" is the latest hotness, and these settings, especially the new-to-Django 5.1 'transaction_mode': 'IMMEDIATE' in OPTIONS, would allow writes to queue. What am I missing?

Continue reading...