[Python] install.packages() not working inside of AWS Lambda function

I need to be able to download R packages from within a Python application and then run an R script as a subprocess that sources them. Due to the nature of the application I'm building, I have to install these dependencies at runtime inside of my Lambda function.

I'm using a custom lambda/Python:3.13 container with R installed and am having issues with downloading these packages. When I test locally, I have no issues, but for whatever reason, it doesn't work on Lambda. No matter what I've tried, the packages aren't being downloaded to the /tmp/Rlibs folder. I only get the packages that were downloaded from installing R in the Docker container.

Here is the Python function that downloads the packages:

def install_cran_package(package, lib_path=None):
"""
Installs a single CRAN package non-interactively
"""
if not package:
logger.info("No CRAN package dependency")
return

logger.info(f"Installing CRAN package: {package}")
lib_path = lib_path or "/tmp/Rlibs"
os.makedirs(lib_path, exist_ok=True)

command = [
"Rscript",
"-e",
f'.libPaths(c("{lib_path}", .libPaths())); '
f'install.packages("{package}", lib="{lib_path}", '
f'repos="https://cloud.r-project.org", dependencies=TRUE, verbose=TRUE)'
]

result = subprocess.run(command, text=True, capture_output=True)

if result.returncode != 0:
logger.error(
f"Failed to install {package}:\n"
f"std err: {result.stderr}\n"
f"std out: {result.stdout}"
)
raise RuntimeError(f"Install failed for {package}")
else:
logger.info(f"Successfully installed {package}")


and the R script:

.libPaths(c("/tmp/Rlibs", .libPaths()))
library("httr")
library("jsonlite")
source("r_client_stubs.R")
source("r_func_helper.R")

# Entry for R function process

args <- commandArgs(trailingOnly = TRUE)
func_name <- args[1]
user_args <- fromJSON(args[2])
invocation_id <- args[3]

source_r_files(file.path("/tmp/functions", invocation_id))

# Execute User function
result <- run_user_function(func_name, user_args)

func_return(result)


and the R portion of the Docker container:

ENV R_LIBS_USER=/tmp/Rlibs

#Install R
RUN dnf install -y \
R \
unzip \
gcc-c++ \
make \
cmake \
libsodium-devel \
git which && \
dnf clean all

Continue reading...