newbie. apache spark python3 'Jupyter' data frame problem with auto completion and accessing documentation

[Andy Davidson]

Apache spark has a notion of a “data frame”. This object is like a PANDAS data frame or a data frame in R.

I wonder if the auto completion problem as todo with the way code is typically written in spark. I.E. By creating chains of function alls? Each function returns a data frame.

hashTagsDF.groupBy("tag").agg({"tag": "count"}).orderBy("count(tag)", ascending=False).show()

Any suggestions would be greatly appreciated. Bellow is my original post to the apache spark list.

Andy

From: Andrew Davidson <An...@SantaCruzIntegration.com>
Date: Monday, August 1, 2016 at 10:08 AM
To: "user @spark" <us...@spark.apache.org>
Subject: python 'Jupyter' data frame problem with autocompletion

I started using python3 and jupyter in a chrome browser. I seem to be having trouble with data frame code completion. Regular python functions seems to work correctly.

I wonder if I need to import something so the notebook knows about data frames?

Kind regards

Andy 

[Thomas Kluyver]

Hi Andy,

On 1 August 2016 at 22:46, Andy Davidson <An...@santacruzintegration.com> wrote:

I wonder if the auto completion problem as todo with the way code is typically written in spark. I.E. By creating chains of function alls? Each function returns a data frame.

hashTagsDF.groupBy("tag").agg({"tag": "count"}).orderBy("count(tag)", ascending=False).show()

At the moment, tab completion isn't smart enough to know what the result of calling a function or method is, so it won't complete anything after the first groupBy there. This is something we hope to improve for IPython 6.

Thomas

[Andy Davidson]

Thanks for the explanation 

Andy

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.
To post to this group, send email to jup...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CAOvn4qiGbH_m-TqsSKA6ou-DenwJXJ9Z%2BxNXSQzV2u7Q5-6HhA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.