For a long time, when developing applications or data pipelines, I used a combination of requirements.txt and Dockerfile. However, since earlier this year, I changed my habit to use Poetry instead of a requirements.txt

What is Poetry?

According to the Poetry Official website, “Poetry is a tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on, and it will manage (install/update) them for you. Poetry offers a lockfile to ensure repeatable installs and can build your project for distribution.”

What I like the most is that it resolves dependency conflicts easily for me, and it’s way more flexible compared to requirements.txt

Setting Up Poetry

Prerequisites

  • Python 3.8+

1 - Installation

pip install poetry

2 - Create a pyproject.toml file on your dir, it will look something like this.

poetry init

[tool.poetry]
name = "app-v1"
version = "0.1.0"
description = ""
authors = ["name <name@email.com>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.11"
pydantic = "^1.10.7"
more-itertools = "^9.1.0"

pydantic = "^1.10.4"
requests = "^2.31.0"

google-api-core = "^2.10.2"


[build-system]
requires = ["poetry-core>=1.4.1"]
build-backend = "poetry.core.masonry.api"

3 - Create poetry lock file for your environment

poetry lock

4 - To update poetry lock file, we can adjust the pyproject.toml file , then run

poetry update

4.5 - If you are on Mac and encounter SSL issue when running poetry update

Go to Applications > Python folder > double click on “Install Certificates.command” file

5 - Using Poetry with Docker

FROM python:3.11-slim

WORKDIR /usr/src/app

ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    POETRY_VERSION=1.4.2

RUN apt-get update && apt-get install -y curl
RUN curl -sSL https://raw.githubusercontent.com/python-poetry/install.python-poetry.org/07ccfe459b6b5c3b3b8859f5cf643c62591d2ea6/install-poetry.py | python3 - ;

ENV PATH="/root/.local/bin:$PATH"
COPY ./poetry.lock pyproject.toml /usr/src/app/
RUN poetry install -nv --no-root

COPY app .
CMD ["poetry", "run", "python3", "main.py"]

Extra Section - Deployment

So after building our application, how can we deploy it?

I use skaffold for deployment most of the time.

Installation

brew install skaffold

Deployment

You will need a skaffold.yaml which looks something like this,

apiVersion: skaffold/v2beta2
kind: Config
metadata:
  name: app-name
build:
  artifacts:
    - image: us-central1-docker.pkg.dev/gcp-project-id/artifects-folder/app-name

deploy: {}

profiles:
  - name: production
    activation:
      - kubeContext: prod
    deploy:
      kustomize:
        paths:
          - ./deploy/production

  - name: staging
    activation:
      - kubeContext: stg
    deploy:
      kustomize:
        paths:
          - ./deploy/staging

  - name: development
    activation:
      - kubeContext: dev
    deploy:
      kustomize:
        paths:
          - ./deploy/development

Deploy using the long way,

k config use-context <k8s-context>

skaffold build

skaffold render -p production > ../a.yaml

cat ../a.yaml

k apply -f ../a.yaml

k get pods -n <k8s-ns>

Or simply run,

skaffold run -p production -v info

That’s it for now.

Thank you for reading and have a nice day!