How to connect to a running mysql service on host from a docker container on same host
Introduction I have a host running mysql (not on a container). I have to run an…
May 20, 2021
I was having a private artifactory, and has my custom modules deployed there. I wanted to build a docker image, where I was using my private artifacts. I needed to pass credentials to build process, so that I could install those private artifacts.
I tried using ARG
and ENV
in Dockerfile
, and it did successfully built. But, when I tried running docker image history --no-trunc <image>
, I could see the credentials naked.
So, the goal is to build docker image, by passing secrets. But, not exposing it in docker image history.
I will describe two solutions:
FROM YOUR_IMAGE:1.0 as intermediate
WORKDIR /var/www/html/
COPY . /var/www/html/
ARG ARTIFACTORY_USER
ARG ARTIFACTORY_TOKEN
RUN virtualenv env --python python3.9 \
&& source /var/www/html/env/bin/activate \
&& python3.9 -m pip install -r /var/www/html/requirements.txt -i https://${ARTIFACTORY_USER}:${ARTIFACTORY_TOKEN}@artifactory.MY.com/artifactory/api/pypi/MY/simple \
&& python3.9 -m pip install mod_wsgi -i https://${ARTIFACTORY_USER}:${ARTIFACTORY_TOKEN}@artifactory.MY.com/artifactory/api/pypi/MY/simple
RUN virtualenv env --python python3.9 \
&& source /var/www/html/env/bin/activate \
&& mod_wsgi-express install-module
COPY ./docker/httpd.conf /etc/httpd/conf/httpd.conf
COPY ./docker/sites-enabled /etc/httpd/sites-enabled
ENV CONFIG "/var/www/html"
EXPOSE 8080
USER root
CMD ["/usr/sbin/httpd","-D","FOREGROUND"]
Now build and run docker image history --no-trunc <YOUR_IMAGE>
You could see somewhere in between the step and which exposes secrets in plain text:
<missing> 13 seconds ago RUN |2 ARTIFACTORY_USER=USER ARTIFACTORY_TOKEN=TOKEN /bin/sh -c virtualenv env --python python3.9 && source /var/www/html/env/bin/activate && python3.9 -m pip install -r /var/www/html/requirements.txt -i https://${ARTIFACTORY_USER}:${ARTIFACTORY_TOKEN}@artifactory.MY.com/artifactory/api/pypi/MY/simple && python3.9 -m pip install mod_wsgi -i https://${ARTIFACTORY_USER}:${ARTIFACTORY_TOKEN}@artifactory.MY.com/artifactory/api/pypi/MY/simple # buildkit 102MB buildkit.dockerfile.v0
<missing> 13 seconds ago ARG ARTIFACTORY_TOKEN
Lets have a look at one of example I made,
FROM YOUR_IMAGE:1.0 as intermediate
WORKDIR /var/www/html/
COPY . /var/www/html/
ARG ARTIFACTORY_USER
ARG ARTIFACTORY_TOKEN
RUN virtualenv env --python python3.9 \
&& source /var/www/html/env/bin/activate \
&& python3.9 -m pip install -r /var/www/html/requirements.txt -i https://${ARTIFACTORY_USER}:${ARTIFACTORY_TOKEN}@artifactory.MY.com/artifactory/api/pypi/pypi-MY-release/simple \
&& python3.9 -m pip install mod_wsgi -i https://${ARTIFACTORY_USER}:${ARTIFACTORY_TOKEN}@artifactory.MY.com/artifactory/api/pypi/MY/simple
FROM YOUR_IMAGE:1.0
WORKDIR /var/www/html/
COPY --from=intermediate /var/www/html /var/www/html
RUN virtualenv env --python python3.9 \
&& source /var/www/html/env/bin/activate \
&& mod_wsgi-express install-module
COPY ./docker/httpd.conf /etc/httpd/conf/httpd.conf
COPY ./docker/sites-enabled /etc/httpd/sites-enabled
ENV CONFIG "/var/www/html"
EXPOSE 8080
USER root
CMD ["/usr/sbin/httpd","-D","FOREGROUND"]
Notice the two FROM
keyword in above Dockerfile
. The command to build this is:
docker build -t data-ws --build-arg ARTIFACTORY_USER=<USER> --build-arg ARTIFACTORY_TOKEN=<ARTIFACTORY_TOKEN> .
I’m passing two arguments to dockerfile: ARTIFACTORY_USER
, and ARTIFACTORY_TOKEN
.
Now, if you do: docker image history --no-trunc <image>
. You will not see the first stage command history. And, you are able to successfully hide your secrets from the build process.
In this solution, I will pass complete pip.conf
file to docker build
process. And, I will copy that file inside image before running pip install
. Also I need to set PIP_CONFIG_FILE
environment variable to set path of custom pip.conf
file.
FROM YOUR_IMAGE:1.0
WORKDIR /var/www/html/
COPY . /var/www/html/data
ENV PIP_CONFIG_FILE /root/.pip.conf
RUN --mount=type=secret,id=pip.conf,dst=/root/.pip.conf virtualenv env --python python3.9 \
&& source /var/www/html/env/bin/activate \
&& python3.9 -m pip install -r /var/www/html/data/requirements.txt \
&& python3.9 -m pip install mod_wsgi \
&& mod_wsgi-express install-module
COPY ./docker/httpd.conf /etc/httpd/conf/httpd.conf
COPY ./docker/sites-enabled /etc/httpd/sites-enabled
EXPOSE 80
USER root
CMD ["/usr/sbin/httpd","-D","FOREGROUND"]
Build command:
docker build -f Dockerfile --secret id=pip.conf,src=/path/to/pip.conf -t MY_IMAGE .
Note the --mount=type=secret,id=pip.conf,dst=/root/.pip.conf
.
From the build command, I’m passing source file information in src
, and I’m setting its id as id
.
In the Dockerfile
, I’m receiving it in same id
. And, copying it with dst
path inside the docker image.
Note, the file used in dst
will not be copied in the image produced. The magic :)
It will not be present even in the intermediate layers.
Hope you enjoy the post.
Introduction I have a host running mysql (not on a container). I have to run an…
Introduction In this post, we will see Python 3.7.9 patch for FIPS enabled…
While doing code review of one of junior, I encountered following code: What…
Agenda I will cover following in this post: Prepare Docker image for Next.js app…
Introduction In this post, we will see Python 3.9.x patch for FIPS enabled…
Introduction Lets assume we have a csv something similar to following: Python…
Introduction This post has the complete code to send email through smtp server…
Introduction In a normal email sending code from python, I’m getting following…
Introduction In one of my app, I was using to talk to . I have used some event…
Introduction So you have a Django project, and want to run it using docker image…
Introduction It is very important to introduce few process so that your code and…
Introduction In this post, we will see a sample Jenkin Pipeline Groovy script…