[JIRA] (SPARKR-189) Add support for selecting and operating on columns in DataFrames

14 views
Skip to first unread message

Shivaram Venkataraman (JIRA)

unread,
Feb 3, 2015, 1:56:02 PM2/3/15
to spark...@googlegroups.com
Shivaram Venkataraman created an issue
 
SparkR / Sub-task SPARKR-189
Add support for selecting and operating on columns in DataFrames
Issue Type: Sub-task Sub-task
Assignee: Unassigned
Created: 03/Feb/15 10:55 AM
Priority: Major Major
Reporter: Shivaram Venkataraman

This will involve adding a new class `Column` and supporting access of columns in a dataframe using `$` syntax (example: df$A to refer to column A)

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011-sha1:fa13a3e)
Atlassian logo

Shivaram Venkataraman (JIRA)

unread,
Feb 6, 2015, 5:02:01 PM2/6/15
to spark...@googlegroups.com

Shivaram Venkataraman (JIRA)

unread,
Feb 15, 2015, 9:48:01 PM2/15/15
to spark...@googlegroups.com
Shivaram Venkataraman updated an issue
Change By: Shivaram Venkataraman
Priority: Major Critical
This message was sent by Atlassian JIRA (v6.4-OD-14-082#64012-sha1:d36fff0)
Atlassian logo

Chris Freeman (JIRA)

unread,
Feb 25, 2015, 11:02:01 AM2/25/15
to spark...@googlegroups.com
Chris Freeman commented on Sub-task SPARKR-189
 
Re: Add support for selecting and operating on columns in DataFrames

Working on this. Starting with base DataFrame methods (filter, groupby, etc.).

Next step is building the Column class and its associated methods.

This message was sent by Atlassian JIRA (v6.4-OD-15-055#64014-sha1:797c660)
Atlassian logo

Chris Freeman (JIRA)

unread,
Feb 25, 2015, 11:02:02 AM2/25/15
to spark...@googlegroups.com

Davies (JIRA)

unread,
Feb 25, 2015, 8:25:01 PM2/25/15
to spark...@googlegroups.com
Davies assigned an issue to Davies
Change By: Davies
Assignee: Chris Freeman Davies

Shivaram Venkataraman (JIRA)

unread,
Mar 4, 2015, 8:30:02 PM3/4/15
to spark...@googlegroups.com
Change By: Shivaram Venkataraman
Assignee: Davies Shivaram Venkataraman
Status: To Do In Progress

Shivaram Venkataraman (JIRA)

unread,
Mar 4, 2015, 8:30:02 PM3/4/15
to spark...@googlegroups.com

Shivaram Venkataraman (JIRA)

unread,
Mar 4, 2015, 10:13:02 PM3/4/15
to spark...@googlegroups.com
Shivaram Venkataraman commented on Sub-task SPARKR-189
 
Re: Add support for selecting and operating on columns in DataFrames

The first cut of the Column API was merged in https://github.com/amplab-extras/SparkR-pkg/pull/188

Some other APIs which would be good to support include

Column selection:

  # Selecting the first column
  df[[1]]
  df[["V1"]]
  df[,1]

  # Selecting the first two columns
  df[,1:2]
  df[,c("V1", "V2")]

   # Column assignment, mutating the DF
   df$V3 <- df$V1 + df$V2

I think we can implement this by overriding the `[`, `[[` and assignment operators in the DataFrame S4 class (http://stackoverflow.com/questions/24253048/whats-the-difference-between-setmethod-and-set-setreplacemethod might be useful)

Shivaram Venkataraman (JIRA)

unread,
Mar 13, 2015, 2:35:02 AM3/13/15
to spark...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages