I haven't ever done software in a professional setting with best practices and all so I don't have a mature development and deployment workflow. I can quickly hack something together but to package and deploy it in a production setting is something I haven't mastered yet. I have been spending a lot of time configuring environments for my little python web apps and docker has helped a lot. Now I want to take it to the next level. Here's how I envision it.
Develop either locally or on a remote host. I prefer not to have to configure development environments on different machines. I would much rather connect to a remote development environment most of the time and have everything done their. Coder.com's code-server and sail projects seem very good for my needs but I've been having trouble setting up sail in my environment.
I've also done a fair amount of research on what is the right flow. Some interesting tools that could serve my needs include GitLab with CI/CD, CapRover, Kubernetes etc. But they seem to be overkill given my current bandwidth.
The current workflow I'm tinkering with is as follows:
- Python installed
- Conda installed [Optional]
- Git installed
- Linux with miniconda installed
conda create -n template python=3.8
conda activate template
- Install packages using
pip install packagenot
python -m pip install packagebecause the later caused installation to host python install rather than the conda environment python install
conda env export > environment.yml
pip freeze > requirements.txt
- Create template of
docker-compose.ymlfile with the right names and routing configuration (to be automated as a template)
- Push the repository to github
- Pull the repository on server
- Run build
docker build . -t base-template
docker-compose up -d
During the performance of above, I found the package I was trying to run would not work with conda. Basically if there is a python command line tool that needs to be invoked by CMD in the Dockerfile, it doesn't run because docker doesn't find it in the PATH. Probably something to do with how conda creates the environment within docker. Hence, I had to move from a conda docker base image to a python docker base image.
I have setup GitLab and now migrating my projects to git. This also required re-familiarity with best practice git workflows. I found the following tutorial on Git Collaboration helpful. The rest of the sections are also a good crisp read for quickly reviewing the concepts.