Installing packages with conda

mmiron
mmiron Member Posts: 12

Does anyone know how to edit the docker files to install packages using conda?

Tagged:

Answers

  • katie.shakman
    katie.shakman Administrator, Domino Posts: 257 admin
    edited April 2020

    Hi @mmiron,

    In general you can just add the following to your dockerfile:

    RUN conda install <package-name>
    

    However because conda tries to solve the environment and install the package without any conflicts, you may see errors using conda to install into very complex environments. This may include compute environments built on top of the Domino Analytics Distribution, which we designed to install most python packages with pip due to the relatively large number and variety of packages this distribution contains. If you're having trouble setting up your conda packages due to conflict errors, you might try building off of a more minimal base image such as the Domino Minimal Distribution (DMD): https://docs.dominodatalab.com/en/4.1/reference/environments/Domino_4_standard_environments.html#domino-minimal-distribution-dmd

    I'll also check in with you over email about specifics for using this in your deployment of Domino.

    Another thing to keep in mind: It's often helpful to iterate on your installation commands in an interactive workspace. This helps confirm they run as expected and don't require interactive input before you move them over to your Dockerfile instructions.

    Please let us know if that helps or if you have further questions.

    Best,

    Katie

    Katie

    Field Data Scientist @ Domino

  • mmiron
    mmiron Member Posts: 12

    Hi Katie,

    Thanks for the tip. As you mentioned the package I was trying to install with conda was having trouble because of the existing conflicts with the environment. Instead I ended up installing the package directly without conda.

    Thanks for your help!

  • msterling
    msterling Member Posts: 5

    I'd like to customize an environment to use a specific conda environment (rather than just installing new libraries into the base conda envrionment). It works fine if you want to access it from an interactive workspace but I run into trouble when I want to run a python script as a job because it's not straightforward to activate a conda environment. What actually happens when you run "job.py" from the run command? Is there any way to modify it? Thanks!

    A couple of related links:


  • katie.shakman
    katie.shakman Administrator, Domino Posts: 257 admin

    Hi @msterling ,

    In general we recommend against using conda environments within compute environments, since the compute env is itself a virtual env so you get an "env inside an env" which can lead to confusion when sharing your setup with colleagues.

    However, if a conda environment makes the most sense for you, there are a few ways you can set that up in your compute environment:

    1. Add it into the Dockerfile instructions with a command like RUN /bin/bash -c "source activate myenv"
    2. Use the "Pre Run Script" and "Pre Setup Script" sections of the environment definition to specify how the virtual machine is set up and any commands you want to execution before your job is run. i.e. add  /bin/bash -c "source activate myenv or similar to your environment configuration, after your Dockerfile instructions. More info on the "Pre Run Script" and "Pre Setup Script" here: https://docs.dominodatalab.com/en/4.1/reference/environments/Environment_management.html?highlight=pre%20run%20script#environment-attributes


    Katie

    Field Data Scientist @ Domino

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!