How To Use an External Docker Image as a Base Image in Domino

katie.shakman
katie.shakman Administrator, Domino Posts: 257 admin
edited October 2020 in Tutorials & Best Practices

When setting up a new compute environment in Domino, the best practice in Domino is to use and build off of our Domino base images. More about our Domino base images here: https://docs.dominodatalab.com/en/4.1/reference/environments/Domino_4_standard_environments.html

However, sometimes you may want to experiment with a third-party Docker image in Domino. One way to do this is to look at what packages are installed, and add them to your favorite Domino compute environment. But sometimes you might want to try to just import and use it with minimal modification. Your results may vary depending on the specifics of what's in your Docker image, but these guidelines will help you to import your desired image with the fewest changes possible:


  • Determine what operating system the image uses. Domino supports  Ubuntu, CentOS, and RHEL images.
  • Create a new compute environment and specify your 3rd-party Docker image using the Custom image option. Example shown below for the crisprlab/crisprdisco image:


Once you save, it will show up like this on the Environment Definition page:

  • Set up the ubuntu user. Domino requires userid host machine has match userid docker image. By convention, most ubuntu hosts use username ubuntu. You can enforce this adding following top Dockerfile instructions when you build your image:
USER root

# Create the Ubuntu user
RUN \
 groupadd -g 12574 ubuntu && \
 useradd -u 12574 -g 12574 -m -N -s /bin/bash ubuntu

Screenshot showing where to add this on the compute environment page:

  • Ensure that pip is installed in the environment. If needed, you can add the following to the Dockerfile instructions to install pip:
RUN apt-get update && apt-get install -y \
    python-pip
RUN pip install --upgrade pip
  • [If you just want to run scripts/bash jobs non-interactively, you can skip this step. ] If interactive workspaces are desired, such as Jupyter, set them up as described here: https://docs.dominodatalab.com/en/latest/reference/environments/Adding_new_workspaces_with_pluggable_notebooks.html. If you encounter any issues with this step, particularly if you are using an image with an older version of Python (<Py 3.x), you may need to install additional dependencies and a Python 3.x version (this won't uninstall your 2.7 version, so you can still use that in a terminal from your interactive workspace). Try adding the snippet below to your Dockerfile instead of the snippet from the docs:
USER root
RUN apt-get update -y

#### Jupyter dependencies ####
RUN apt-get install -y libzmq3-dev pandoc pandoc-citeproc libssl-dev libsqlite3-dev unzip && \
apt-get install zlib1g-dev && \
apt-get install python-setuptools -y && \
apt-get install python-tk -y

#### Install python 3.x ####
RUN cd /tmp && \
wget -q https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tgz && \
tar -xzf Python-3.6.8.tgz && \
cd Python-3.6.8 && ./configure && make && make install && rm -rf /tmp/*

#### Set python version ####
RUN rm /usr/bin/python && \
ln -s /usr/local/bin/python3 /usr/bin/python
RUN cd /tmp && curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && python3 get-pip.py
RUN rm -rf /opt/conda/bin/pip && \
ln -s /usr/bin/pip3 /opt/conda/bin/pip && \
pip install --upgrade pip

#### Installing Notebooks,Workspaces,IDEs,etc ####
# Clone in workspaces install scripts
# Add workspace configuration files
RUN \
  cd /tmp && \
    wget -q https://github.com/dominodatalab/workspace-configs/archive/2019q4-v1.zip && \
  unzip 2019q4-v1.zip && \
  cp -Rf workspace-configs-2019q4-v1/. /var/opt/workspaces && \
  rm -rf /var/opt/workspaces/workspace-logos && rm -rf /tmp/workspace-configs-2019q4-v1 && \

### Install Jupyter from workspaces 
  chmod +x /var/opt/workspaces/jupyter/install && \
  /var/opt/workspaces/jupyter/install && \

### Clear apt list to save space
rm -rf /var/lib/apt/lists/*

Then build and test your environment.

If everything works, you're good to go! If you run into issues, let us know in the comments (or if it's urgent/sensitive/you prefer, you can put in a support ticket).

Katie

Field Data Scientist @ Domino

Comments

  • katie.shakman
    katie.shakman Administrator, Domino Posts: 257 admin
    edited September 2020

    If you have trouble installing pip, and lsb_release -a shows that you are using Ubuntu 20.04, then follow the instructions here: https://stackoverflow.com/questions/61981156/unable-to-locate-package-python-pip-ubuntu-20-04

    Specifically, from user Expired Brain, try this:

    Pip for Python 2 is not included in the Ubuntu 20.04 repositories.

    You need to install pip for Python 2 using the get-pip.py script.

    1. Start by enabling the universe repository:

    sudo add-apt-repository universe
    

    2. Update the packages index and install Python 2:

    sudo apt update 
    sudo apt install python2
    

    3. Use curl to download the get-pip.py script:

    curl https://bootstrap.pypa.io/get-pip.py --output get-pip.py
    

    4. Once the repository is enabled, run the script as sudo user with python2 to install pip :

    sudo python2 get-pip.py
    


    Katie

    Field Data Scientist @ Domino

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!