[JIRA] (SPARKR-223) Make API documentation accessible outside of R installation

21 views
Skip to first unread message

Felix Cheung (JIRA)

unread,
Mar 12, 2015, 2:46:02 AM3/12/15
to spark...@googlegroups.com
Felix Cheung created an issue
 
SparkR / Improvement SPARKR-223
Make API documentation accessible outside of R installation
Issue Type: Improvement Improvement
Assignee: Unassigned
Created: 11/Mar/15 11:45 PM
Priority: Minor Minor
Reporter: Felix Cheung

We have .Rd for API documentation and they get converted into HTML when the SparkR package is installed.
Is it possible to get these up on a site and available on the internet a la PySpark: http://spark.apache.org/docs/latest/api/python/

Existing API HTML pages:

00Index.html maximum.html
aggregateRDD.html minimum.html
broadcast-class.html name.html
broadcast.html numPartitions.html
broadcast-internal.html objectFile.html
cache-methods.html parallelize.html
checkpoint-methods.html partitionBy.html
cogroup.html persist.html
collect-methods.html print.jobj.html
combineByKey.html R.css
countByKey.html RDD.html
countByValue.html reduceByKey.html
count.html reduceByKeyLocally.html
distinct.html reduce.html
filterRDD.html sampleRDD.html
flatMap.html saveAsObjectFile.html
flatMapValues.html saveAsTextFile.html
fold.html setCheckpointDir.html
foreach.html setName.html
groupByKey.html sortBy.html
hashCode.html sortByKey.html
includePackage.html sparkR.init.html
join-methods.html take.html
keyBy.html takeOrdered.html
keys.html takeSample.html
lapply.html textFile.html
lapplyPartition.html top.html
lapplyPartitionsWithIndex.html unionRDD.html
lookup.html unpersist-methods.html
mapValues.html values.html

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4-OD-15-055#64014-sha1:797c660)
Atlassian logo

Shivaram Venkataraman (JIRA)

unread,
Mar 12, 2015, 2:13:02 PM3/12/15
to spark...@googlegroups.com
Shivaram Venkataraman commented on Improvement SPARKR-223
 
Re: Make API documentation accessible outside of R installation

Felix Cheung This is a good idea and we can definitely host this out of the SparkR webpage http://amplab-extras.github.io/SparkR-pkg/.
It'll be especially cool if we used knitr or something like that to get pretty HTML. (http://yihui.name/en/2012/10/build-static-html-help/ has some details)

BTW the code for the website lives in the gh-pages branch on the same repo – so you if you want to pick this up you can just send a PR to that branch.

Felix Cheung (JIRA)

unread,
Mar 12, 2015, 7:06:03 PM3/12/15
to spark...@googlegroups.com

Felix Cheung (JIRA)

unread,
Mar 12, 2015, 7:06:03 PM3/12/15
to spark...@googlegroups.com
Felix Cheung commented on Improvement SPARKR-223
 
Re: Make API documentation accessible outside of R installation

Cool, knit_rd would be what I was looking for. Rdconv also does the conversation. https://stat.ethz.ch/R-manual/R-devel/library/base/html/RdUtils.html

The catch is this requires R and R packages to build the HTML. I suppose that's ok since we would update the public doc on each release only.

I'll look into this.

Felix Cheung (JIRA)

unread,
Mar 19, 2015, 3:42:01 PM3/19/15
to spark...@googlegroups.com

Shivaram Venkataraman Seems like you have implement this in amplab-extras/spark

Shivaram Venkataraman (JIRA)

unread,
Mar 19, 2015, 4:25:01 PM3/19/15
to spark...@googlegroups.com

So I did write a script to generate the html files. They are still not hosted anywhere though as the plan is that they will be hosted as a part of the Spark docs from Spark 1.4. We can still host docs for Spark 1.3 in the SparkR webpage if you think that is helpful.

Felix Cheung (JIRA)

unread,
Mar 20, 2015, 12:36:02 AM3/20/15
to spark...@googlegroups.com

Cool. I think it's a good idea to add this to the main github page for now.
My understanding is

amplab-extras/SparkR-pkg = Spark 1.2.1
amplab-extras/SparkR-pkg/tree/sparkr-sql = Spark 1.3.0
amplab-extras/spark/R = Spark 1.4.0
?

Created these PR. Thought?
https://github.com/amplab-extras/SparkR-pkg/pull/227
https://github.com/amplab-extras/SparkR-pkg/pull/228

Shivaram Venkataraman (JIRA)

unread,
Mar 20, 2015, 1:07:02 AM3/20/15
to spark...@googlegroups.com

Shivaram Venkataraman (JIRA)

unread,
Mar 20, 2015, 10:03:01 PM3/20/15
to spark...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages