Spark caching across REST calls

24 views

Skip to first unread message

Stephen Petschulat

unread,

Sep 15, 2016, 9:35:44 PM9/15/16

to CDAP User

Is it possible to access a cached RDD across REST requests? Or put another way, can I use the same SparkContext across HTTP requests, thereby taking advantage of caching?

This Spark Revamp wiki doc discusses related issues but it isn't clear if it has already been implemented or is in the works.

cheers,

- steve

Terence Yim

unread,

Sep 15, 2016, 11:47:37 PM9/15/16

to CDAP User

Hi Steve,

You are right that it was part of the revamp design but it is not yet implemented. You can watch the ticket https://issues.cask.co/browse/CDAP-6033 for updates.