Sync files programatically

pankhuri.verma
pankhuri.verma Member, Administrator, Moderator, Domino Posts: 19 admin

How can we sync files programmatically in between our code runs.

3 votes

Active · Last Updated

Comments

  • john.joo
    john.joo Member, Moderator, Domino Posts: 6 mod

    Hi @pankhuri.verma , why would you want to sync files programmatically during your code runs?

    Is this to make sure you're backing up files so that you don't lose progress on some buggy code? Is this to monitor the progress of your run, say loss metrics, in the realtime way?

  • katie.shakman
    katie.shakman Administrator, Domino Posts: 257 admin

    We had a support ticket that sparked this request initially, and it seems the use case had to do with running multiple experimental configurations from the same job or workspace, and syncing after each iteration/configuration change.

    Katie

    Field Data Scientist @ Domino

  • joaquim.benseny
    joaquim.benseny Member Posts: 4

    We would also appreciate this. Use case:

    1. From workspace A we run some code and create some data that we store in the workspace.
    2. Trigger multiple workspaces that should use this data (i.e. workspace B, workspace C...)
    3. Wait from workspace A and wait until they finish.
    4. Bring files from workspace B, workspace C...
    5. Collate all results and carry on some calculations

    Although we can use blobs_get and files_upload this is not great when you have multiple files and we would like to use sync between step 1 and 2 and get latest domino files for step 4.

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!