Using pip-tools for multiple environments
For couple of years I’ve been using pip-tools for managing requirements but recently I faced with a problem of splitting requirements for dev and production environments. The problem was that if we just split requirements and compile them separately we will end up with different versions.
So next step is to use an approach from this article. The idea is to compile requirements.prod.in file first and than compile development part with requirements.prod.txt
as constraints. It can be done like this:
-c requirements.prod.txt
# other requirements
This solution is also has drawbacks which I faced on the first try. I’v got unsolvable dependendencies between requirements.prod.txt and my development requirements. This conflict can be resolved manually, but it’s quite a tedious and not an elegant solution.
Next improvement is using intermediate file with all constraints, compile it and than build requirements. I found this solution in this comment to github issue. The author BTW is the same person, who wrote the article in the link above. Thank you, James Cooke.
Here my modified solution for Makefile
:
## pip-compile: compile all requirements
pip-compile: prepare-constraints
pip-compile constraints.in
pip-compile requirements.prod.in
pip-compile requirements.dev.in
## pip-upgrade: upgrade all requirements
pip-upgrade: prepare-constraints
rm -f constraints.txt requirements.prod.txt requirements.dev.txt
touch constraints.txt
pip-compile constraints.in
pip-compile requirements.prod.in
pip-compile requirements.dev.in
prepare-constraints: check-pip-compile
rm -f constraints.in
touch constraints.txt
cat requirements.*.in > constraints.in
## pip-sync: sync requirements in local environment
pip-sync: check-pip-compile
pip-sync requirements.prod.txt requirements.dev.txt
check-pip-compile:
@which pip-compile > /dev/null
And you need to add a following string to each .in
file:
-c constraints.txt
Also, here is an example of Dockerfile
to build an image based on ARG
:
ARG ENVIRONMENT="prod"
FROM python:3.9-buster AS app-prod
RUN pip3 install --upgrade pip
RUN useradd --no-create-home --gid root runner
WORKDIR /code
COPY requirements.prod.txt .
RUN pip3 install --no-cache-dir -r requirements.prod.txt
FROM app-prod AS app-dev
COPY requirements.dev.txt .
RUN pip3 install --no-cache-dir -r requirements.dev.txt
FROM app-${ENVIRONMENT} AS app
COPY . .
RUN chown -R runner:root /code && chmod -R g=u /code
USER runner
EXPOSE 8000
ENTRYPOINT [ "/code/docker-entrypoint.sh" ]
To create dev
image, build it with argument ENVIRONMENT="dev"