How to Import Pandas DataFrame into Glue

100 views
Skip to first unread message

Phil Edwards

unread,
Oct 20, 2019, 5:50:09 AM10/20/19
to Glue users
Hi,

Glue looks like it can import a Pandas DataFrame in the "Import Data" menu option, but what file extension is it looking for to know it's a Pandas DataFrame?  My intent is to save a DataFrame as a file from Python and open Glue to import it.  Normally I  would save a Pandas DataFrame as a Pickle file, but Glue doesn't recognize the .pkl extension.
Thanks
Phil


Thomas Robitaille

unread,
Oct 20, 2019, 6:20:38 PM10/20/19
to glue...@googlegroups.com
Hi Phil,

The 'pandas' option in the 'import data' dialog works with .csv files
exported from DataFrame.to_csv. Could you try this and let me know if
it doesn't work?

Thanks!
Tom
> --
> You received this message because you are subscribed to the Google Groups "Glue users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to glue-viz+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/glue-viz/15e40fec-83b1-49ce-ab72-6cbb26b0b323%40googlegroups.com.

Phil Edwards

unread,
Oct 24, 2019, 6:59:32 PM10/24/19
to Glue users
Oh I see.  Yes csv works - thanks.  I assume that's the only/fastest way to load for large DataFrames (2 million rows)?
Phil

Thomas Robitaille

unread,
Oct 26, 2019, 11:57:49 AM10/26/19
to glue...@googlegroups.com
Hi Phil,

If you are working in a Python session where you have a DataFrame and want to visualize it in glue, the most efficient way is to launch glue using the qglue function (from glue import qglue) and pass the DataFrame with:

qglue(data=thedataframe)

The other option, if you want to work with e.g. pkl files (if they would be faster) is to create your own data loader function as described here:


Another option would be to convert the data to a Numpy structured array and save it to a file with np.save - then load the resulting file in glue, which should recognize it.

Cheers,
Tom


--
You received this message because you are subscribed to the Google Groups "Glue users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to glue-viz+u...@googlegroups.com.

Phil Edwards

unread,
Nov 1, 2019, 10:30:31 PM11/1/19
to Glue users
I realize what I want to do is launch Glue (from Python) and then in Glue click on "Open Session" to open a previously saved session with desired the charts already set up.  As long as I have replaced the source data file with an updated one from Python, my Glue session will now display the latest data in the format I want.  My programming level is not good enough to write my own Pickle file loader, and a Numpy file will lose the column titles, so I'll stick with CSV files for now.

Is it possible to launch Glue with a previously saved session, directly from Python?
To unsubscribe from this group and stop receiving emails from it, send an email to glue...@googlegroups.com.

Thomas Robitaille

unread,
Feb 9, 2020, 10:58:40 AM2/9/20
to glue...@googlegroups.com
Hi Phil,

Sorry for the long delay in replying - to launch a previously saved glue session from Python you can do:

from glue.app.qt import GlueApplication
app = GlueApplication.restore_session('saved_session.glu')

I hope this helps!

Cheers,
Tom

To unsubscribe from this group and stop receiving emails from it, send an email to glue-viz+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/glue-viz/ac1ff304-d3a8-4674-ac29-5a93a6f52e34%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages