Our community is getting a makeover! We will be migrating to a new community that integrates more closely with our support and knowledge base tools, as well as the core Domino product, to give you a more unified experience. Existing community articles have been added to our knowledge base, which you can preview here with your community credentials: https://tickets.dominodatalab.com/hc/en-us/community/topics Watch this space for further updates and send any feedback to [email protected] with subject "Community Feedback".

Datasets and app for storage: Can I store application persistent data into datastets?

aly
aly Member Posts: 7

Hi,

We would like to use datasets as shared storing system for a Shiny app.

We would to use datasets as an alternative to a database that we don't have yet. So far we used Scratch as an alternative but this is available only from the project owner so other users that are connecting to the app cannot access to potential sessions data.

We tried with domino.yaml but seems like the app is not writing data into datasets from the app (as if it needs a "commit" action as within the workspace).

Tagged:

Answers

  • jaclyn.patterson
    jaclyn.patterson Member, Administrator, Moderator, Domino Posts: 31 admin

    Hi Aly,

    There are a limited number of ways you can write to a Dataset: UI upload, the CLI, by running a script as a Domino Job, or using a Domino workspace (https://docs.dominodatalab.com/en/4.4/reference/data/datasets/Datasets_overview.html#writing-to-a-local-dataset). The only one that seems to fit your use case of writing to a Dataset from within an app is using the Domino CLI, specifically the upload-dataset command.

    For example, if the files you want to write to the Dataset are in /Users/myUser/data, you would run the following command:

    domino upload-dataset <username>/<project-name>/<dataset-name> /Users/myUser/data
    

    Instructions for installing the CLI can be found here: https://docs.dominodatalab.com/en/4.4/reference/cli/Installing_the_Domino_Command_Line_(CLI).html.

    If you have any questions let me know, or feel free to put in a Domino Support ticket at [email protected]

    Best,

    Jaclyn

  • aly
    aly Member Posts: 7
    edited February 23

    Hello,

    Thanks for the answer. The solution you suggest would mean we need to install CLI inside the environnement and set a kind of service account to run the command from the app container is that correct?

    CLI requires Domino login for me so it would mean we need to store the secret inside or with environment variables right? Or does it exist and embedded CLI version in the Domino env?

    It would be a good datasets improvement to better interact in read/write with app or with Model API I think.

    Antoine

  • jaclyn.patterson
    jaclyn.patterson Member, Administrator, Moderator, Domino Posts: 31 admin

    Hi Aly,

    You're correct about needing the CLI inside the environment. An alternative would be to use the Domino API to save the files to your project files and then convert them to a Dataset Snapshot from there via something like a scheduled job for example. Environment variables would be your best bet for storing those credentials, you're correct.

    As for your comment on improvements, I completely agree. There are plans to drastically improve Datasets in the near future, so I'll make sure to add this suggestion as well.

    Best,

    Jaclyn

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!