Our community is getting a makeover! We will be migrating to a new community that integrates more closely with our support and knowledge base tools, as well as the core Domino product, to give you a more unified experience. Existing community articles have been added to our knowledge base, which you can preview here with your community credentials: https://tickets.dominodatalab.com/hc/en-us/community/topics Watch this space for further updates and send any feedback to [email protected] with subject "Community Feedback".


cgalea11 Member Posts: 14

Hi All,

Has anyone used PyArrow on Domino. I'd like to speed up the conversion to pandas dataframes in PySpark using the .toPandas() command.

Thank you,



  • dan.stern
    dan.stern Member, Moderator, Domino Posts: 37 mod

    Hi Charles,

    It looks like pyarrow should be installable in your workspace with just a

    pip install pyarrow

    Have you tried this and run into some difficulties with Pyarrow?

  • cgalea11
    cgalea11 Member Posts: 14

    Hi Dan,

    All good thanks. It seems to be running fine (please see code fragment below) showing a substantial reduction in time taken to convert from a Spark dataframe to pandas dataframe.


Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!