Installing packages with conda

mmironmmiron Member Posts: 11

Does anyone know how to edit the docker files to install packages using conda?

Tagged:

Answers

  • katie.shakmankatie.shakman Administrator, Domino Posts: 182 admin
    edited April 1

    Hi @mmiron,

    In general you can just add the following to your dockerfile:

    RUN conda install <package-name>
    

    However because conda tries to solve the environment and install the package without any conflicts, you may see errors using conda to install into very complex environments. This may include compute environments built on top of the Domino Analytics Distribution, which we designed to install most python packages with pip due to the relatively large number and variety of packages this distribution contains. If you're having trouble setting up your conda packages due to conflict errors, you might try building off of a more minimal base image such as the Domino Minimal Distribution (DMD): https://docs.dominodatalab.com/en/4.1/reference/environments/Domino_4_standard_environments.html#domino-minimal-distribution-dmd

    I'll also check in with you over email about specifics for using this in your deployment of Domino.

    Another thing to keep in mind: It's often helpful to iterate on your installation commands in an interactive workspace. This helps confirm they run as expected and don't require interactive input before you move them over to your Dockerfile instructions.

    Please let us know if that helps or if you have further questions.

    Best,

    Katie

    Katie

    Customer Success Engineer @ Domino

  • mmironmmiron Member Posts: 11

    Hi Katie,

    Thanks for the tip. As you mentioned the package I was trying to install with conda was having trouble because of the existing conflicts with the environment. Instead I ended up installing the package directly without conda.

    Thanks for your help!

  • msterlingmsterling Member Posts: 5

    I'd like to customize an environment to use a specific conda environment (rather than just installing new libraries into the base conda envrionment). It works fine if you want to access it from an interactive workspace but I run into trouble when I want to run a python script as a job because it's not straightforward to activate a conda environment. What actually happens when you run "job.py" from the run command? Is there any way to modify it? Thanks!

    A couple of related links:


  • katie.shakmankatie.shakman Administrator, Domino Posts: 182 admin

    Hi @msterling ,

    In general we recommend against using conda environments within compute environments, since the compute env is itself a virtual env so you get an "env inside an env" which can lead to confusion when sharing your setup with colleagues.

    However, if a conda environment makes the most sense for you, there are a few ways you can set that up in your compute environment:

    1. Add it into the Dockerfile instructions with a command like RUN /bin/bash -c "source activate myenv"
    2. Use the "Pre Run Script" and "Pre Setup Script" sections of the environment definition to specify how the virtual machine is set up and any commands you want to execution before your job is run. i.e. add  /bin/bash -c "source activate myenv or similar to your environment configuration, after your Dockerfile instructions. More info on the "Pre Run Script" and "Pre Setup Script" here: https://docs.dominodatalab.com/en/4.1/reference/environments/Environment_management.html?highlight=pre%20run%20script#environment-attributes


    Katie

    Customer Success Engineer @ Domino

Sign In or Register to comment.