How do I ensure a Python package from a private repo is usable in both runs and models?

terry.stebbens
terry.stebbens Moderator, Domino Posts: 17 mod

When developing code for a model you may pull in modules from private Git repos using the Git Repositories setting in the project files

and adding the line

-e /repos/reponame

to requirements.txt to ensure the module is installed when the run starts. This works when running the code in a Workspace or normal Run but when you try to publish it as a model you'll find that you'll get a ModuleNotFoundError for the module as models do not read or process requirements.txt.

A solution to ensure the code works in both normal runs and a model is to load the module with the following code:

try:
  import packagename
except ImportError:
  import subprocess,sys,site as site
  subprocess.call([sys.executable, "-m", "pip", "install", "-e", "/repos/reponame"])
  site.main()
finally:
  import packagename

Using this, if the requirements.txt is not processed your code will use start a shell to run pip and install the package from the mounted repository location ensuring your code works in both situations.

Comments

  • katie.shakman
    katie.shakman Administrator, Domino Posts: 257 admin

    Nice, thanks for sharing this Terry!

    Katie

    Field Data Scientist @ Domino

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!