[GSoC] Typed Transient Proposal

[Sean Laguna]

Hi,

I am a third year computer science PhD student at the University of Chicago, and am interested in submitting a proposal for the typed transients project. I am very interested, in general, about presistency, transience, and the interaction between the two models for operating on data. Persistency is actually influence some of my current work on eliminating race conditions on data structures common in scientific computing (ghost nodes and other regions allocated to overlapping processors in distributed memory programs, etc). I've poked around in the Clojure source code to get some inspiration for my own implementations (I've done some work on that in C++ and in Nim), and of course have read the canonical set of blog posts on the implementation of persistent vectors in Clojure. (Which, if I recall correctly, Rich Hickey actually cited in on of his papers on Clojure!)

I would be happy to work on typing transient data structures in core.clojure, and to work out a means of typing transient that interacts efficiently and elegantly with other types, especially persistent data structures. I'm working on a proposal for this now and can send it in a couple hours, but I wonder if there's a good way of identifying the lowest hanging fruit for typing a crucial transient data structure in the core Clojure codebase where I could do a sort of trial run? It would help immensely I think with coming up with a laundry list of tasks for performing this, instead of it being more of a casual perusal of the code.

One endgame for this work that I'm interested in is automatic parallelization, which Clojure's persistent data structures, in my opinion, do inspirationally well. Something I'm very interested in is applying the shared memory concurrency model to a distributed memory environment. Persistency has natural distributed memory implications with regard to especially parallelism, because modifications to the "same" value that are made, at the same logical time, in different, distributed places in memory, can potentially be reconciled when the values are eventually synced (perhaps even lazily). Transients could be used in the distributed locations, and syncing could be done using persistent logic. My thoughts on this are a bit more fleshed out than as presented here, but I see real potential for nearly free (from a syntactic point of view) distributed memory parallelism. I've read about avout but I'm a bit disappointed by their somewhat unleveraged use of persistency and transience, and about their seeming lack of support for data structures.

I know that this endgame is a bit separate from the goal of typing transients, but I believe that the extra information gained from typing transients could lend itself to a more painless implementation of something like the above. So, I have a few questions:

1. would a very simple distributed memory parallelism implementation that relies on persistence and transients be desirable as a test-case for this project? The desire for distributed memory parallelism could itself be inferred through a type annotation.
   
2. would a project that leverages persistent/transients for race-free distributed memory parallelism be desirable in general, separate from this project?
• do you agree that there may be some viability here?
3. for this particular project proposal: is there a simple example of an instance of a transient data structure in the core Clojure codebase upon which I can base an itemized procedure for carrying out this proposal?
4. is it within the scope of this project to also type persistency that may be missing in the core Clojure codebase, or to convert non-transient data stuctures that may benefit from transience?

Thanks! Hope to hear back soon (and sorry for the late correspondence about this)!

Sean Laguna

[Ambrose Bonnaire-Sergeant]

Hi Sean,

Sounds like you have greater ambitions than simply supporting transients. Please feel free to disregard any suggestions

in the project template and make the *you* would like to implement over the summer. Please post it here or on Melange then we 

can discuss further.

Thanks!

Ambrose

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sean Laguna]

Hi,

I did end up taking your advice and submitting a bit of an ambitious proposal! You can find it here: https://gist.github.com/seanlaguna/c2003b52cc197119bdec as well as submitted.

I made a bit of a blunder though -- I didn't think about the fact that by submitting my proposal as a gist (an external link from the submission), I would be (in some sense) circumventing the deadline. I really hope this doesn't affect my ability to potentially be accepted, and am definitely willing to do whatever might be necessary to correct for this. Of course, the gist has full revision history, so you can see the only thing I did past 2:00pm Central Time (where I am) was add a link to my resume (since I realized I could not attach two files within melange itself).

I also do apologize for cutting this so close; I got a bit carried away delving into Clojure, though I had a great time. To comment on the way that I work, I would say that I do get very engrossed in what I'm doing, and benefit a lot from frequent communication that keeps me on-track. I do think that summer of code would facilitate my style of work well in that regard. I will also say that I note that my open-source contributions are not particularly strong (they barely exist, actually). I have focused more on the academic side of work, but I love the ideals of open source and have always been meaning to make direct contributions. I really do hope that this is a way to get my foot in the door and stay there, and I hope that whether or not I'm accepted that you have some interest in my ideas and can perhaps give me feedback of any sort. 

This is another reason I took a while to submit my proposal: I wanted to provide some code in the background section of my proposal which would show that I did have some chops for parallel programming in Clojure to indicate that I could pick up languages quickly and that this is a feasible project for me. I have done some programming in Clojure before but getting some of the reducer/atom/future/do* syntax exactly right was a fun challenge!

Again, I'd love to hear comments on my proposal, and let me know if there's anything else I can do in the meantime.

Best,

Sean

[Ambrose Bonnaire-Sergeant]

Hi Sean,

I'm afraid student applications are a hard deadline, so you will have to try again next year.

I will go through your application over the weekend.

Thanks,

Ambrose

[Sean Laguna]

Hi,

I did get the melange submission in on time, it just points to the gist. Presumably that means I'm technically on time?

Best,
Sean

You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/U3uaNq0Jt4c/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.

[Ambrose Bonnaire-Sergeant]

Ah yes. Then we're all good!

[Ambrose Bonnaire-Sergeant]

You should be able to edit the proposal now. Please copy the gist over.

[Sean Laguna]

Done, thanks for bearing with me. I foolishly didn't even think at first that an attachment would be insufficient -- and it's a bit too bad, since markdown definitely served me better than the provided text box. I realized just a bit too late that it really wanted those boxes filled.