[Python] Setting up Alembic for a Kivy android app

I am currently trying to build a mobile app using Python, Kivy and SQLAlchemy. However, I am having difficulty implementing database migrations using Alembic.

Basically, the migrations work just fine on my computer but Alembic is not finding any migrations when called from my phone. I am using buildozer for the APK building.

My guess is that Alembic is not finding the migrations files from the versions folder when run on android, probably because of a path issue.

Additional notes on the app functioning on Android :

• 
  The database is created correctly when the app the launched for the first time
  
  
• 
  When the app is launched, Alembic starts to run but doesn't find any migrations to apply. Then the app exits without any error code.
  
  
• 
  If I replace manually the database with one already up-to-date, the app runs smootly.
  
  
• 
  If I replace the database with one initialized but not up-to-date (containing the initial migration but not the updates). I get this error using adb logcat :
  
  • 
    02-17 21:27:48.451 17547 21720 I python : Migration failed: Can't locate revision identified by '2c282135e834'
    02-17 21:27:48.933 17547 21720 I python : Python for android ended.

For reference here is my current code. I left all the debugging prints I added, hopefully it can help...

The Kivy App definition :

class AStatApp(MDApp):
def build(self):
self.theme_cls.theme_style = "Light"
self.theme_cls.primary_palette = "Darkred"

db_path = get_db_path()

# This is the function currently not working
try:
run_migrations()
except Exception as e:
print(f"Migration failed: {e}")

self.Session = self.init_db(db_path)

Builder.load_file("kv/selector.kv")
Builder.load_file("kv/ascent-list-screen.kv")
Builder.load_file("kv/ascent-screen.kv")
Builder.load_file("kv/settings-screen.kv")
Builder.load_file("kv/area-screen.kv")
Builder.load_file("kv/statistic-screen.kv")
Builder.load_file("kv/statistic-filter-screen.kv")
Builder.load_file("kv/to-do-list-screen.kv")
Builder.load_file("kv/screenmanager.kv")
return MainScreenManager()


The get_db_path():

def get_db_path():
db_filename = "astat.db"
if platform == "win":
data_dir = os.getcwd()
elif platform == "android":
# Get Android context
from jnius import autoclass, cast

PythonActivity = autoclass("org.kivy.android.PythonActivity")
context = cast("android.content.Context", PythonActivity.mActivity)

# Get external storage path for the app
file_p = cast("java.io.File", context.getExternalFilesDir(None))
data_dir = file_p.getAbsolutePath()

writable_db_path = os.path.join(data_dir, db_filename)

print(f"Final DB path: {writable_db_path}")
print(f"Path exists: {os.path.exists(writable_db_path)}")

return writable_db_path


The run_migrations()

def run_migrations():
"""Run migrations using Alembic's command API with proper configuration."""
db_path = get_db_path()

# Creates database file if it doesnt exist
if not os.path.exists(db_path):
open(db_path, "a").close()

base_dir = os.path.abspath(os.path.dirname(__file__))
script_location = os.path.join(base_dir, "migrations")

# Configure Alembic programmatically
alembic_cfg = Config("alembic.ini")
alembic_cfg.set_main_option("script_location", script_location)
alembic_cfg.set_main_option("sqlalchemy.url", f"sqlite:///{db_path}")

# Debug prints
db_url = alembic_cfg.get_main_option("sqlalchemy.url")
migrations = os.listdir(os.path.join(script_location, "versions"))
print(f"database url : {db_url}")
print(f"migration path: {script_location}")
print(f"Migration versions: {migrations}")

try:
command.upgrade(alembic_cfg, "head")
print("Database migrations applied successfully.")
except SystemExit as e:
if e.code != 0:
print(f"Migration failed with code {e.code}")
raise


the env.py:

config = context.config

if config.config_file_name is not None:
fileConfig(config.config_file_name)

target_metadata = Base.metadata

def run_migrations_offline() -> None:
"""Run migrations in 'offline' mode.

This configures the context with just a URL
and not an Engine, though an Engine is acceptable
here as well. By skipping the Engine creation
we don't even need a DBAPI to be available.

Calls to context.execute() here emit the given string to the
script output.

"""

url = config.get_main_option("sqlalchemy.url")

context.configure(
url=url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
)

with context.begin_transaction():
context.run_migrations()


def run_migrations_online() -> None:
"""Run migrations in 'online' mode.

In this scenario we need to create an Engine
and associate a connection with the context.

"""
from alembic.script import ScriptDirectory

script = ScriptDirectory.from_config(config)

# Debug prints
print(ScriptDirectory.versions)
print("Discovered migrations:")
for rev in script.walk_revisions():
print(f"- {rev.revision} ({rev.doc})")

db_path = get_db_path()
print(f"Android DB Path: {db_path}")
print(f"File exists: {os.path.exists(db_path)}")
print(f"Migration files: {os.listdir(os.path.dirname(__file__))}")


# Actual migration run
connectable = create_engine(f"sqlite:///{db_path}")

with connectable.connect() as connection:
context.configure(
connection=connection, target_metadata=target_metadata
)

with context.begin_transaction():
context.run_migrations()

if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()


So far, I have tried many things to reset the way Alembic is looking for migrations but nothing works (and I have a hard time remembering everything I tried...)

Also, with the current code, this is the adb logcat i get when I launch the app for the first time (I removed everything before that is just about the normal kivy app setup) :

02-17 21:19:43.422 12846 17556 I python : [INFO ] [Window ] auto add sdl2 input provider
02-17 21:19:43.423 12846 17556 I python : [INFO ] [Window ] virtual keyboard not allowed, single mode, not docked
02-17 21:19:43.672 12846 17556 I python : [INFO ] [Clipboard ] Provider: android
02-17 21:19:43.912 12846 17556 I python : Final DB path: /data/user/0/com.cmareau.astat/files/astat.db
02-17 21:19:43.912 12846 17556 I python : Path exists: False
02-17 21:19:43.913 12846 17556 I python : Final DB path: /data/user/0/com.cmareau.astat/files/astat.db
02-17 21:19:43.913 12846 17556 I python : Path exists: False
02-17 21:19:43.914 12846 17556 I python : database url : sqlite:////data/user/0/com.cmareau.astat/files/astat.db
02-17 21:19:43.914 12846 17556 I python : migration path: /data/data/com.cmareau.astat/files/app/migrations
02-17 21:19:43.914 12846 17556 I python : Migration path content: ['2c282135e834_initial_migration.pyc', '3b4994652668_populate_grade_table.pyc', '__pycache__', 'db33e18bf97f_adding_of_todolist_sector_and_climbtodo_.pyc']
02-17 21:19:43.920 12846 17556 I python : Final DB path: /data/user/0/com.cmareau.astat/files/astat.db
02-17 21:19:43.920 12846 17556 I python : Path exists: True
02-17 21:19:43.920 12846 17556 I python : Android DB Path: /data/user/0/com.cmareau.astat/files/astat.db
02-17 21:19:43.920 12846 17556 I python : File exists: True
02-17 21:19:43.920 12846 17556 I python : Migration files: ['README', '__pycache__', 'env.pyc', 'script.py.mako', 'versions']
02-17 21:19:43.936 12846 17556 I python : Database migrations applied successfully.
02-17 21:19:44.232 12846 17556 I python : Python for android ended.

Continue reading...