F# application checkpointing

[Eric]

I know this could be asked at StackOverflow, but this may need a discussion and I prefer answer(s) from those who know F#. If the info of here is of value then we can post to StackOverflow

 

There is a need for application checkpointing, (Wikipedia) for a project some of us are working on and my search for such with F#, or even Windows is turning up nothing.

 

Note: The link from Wikipedia should be http://checkpointing.org/, (Thanks Jack)

 

If you check http://checkpointing.org/ you will see that this is common in the Linux world.

 

What I am looking for is a way to save and restore the state when using F# Interactive. Think checkpoint (save) an F# interactive session, shut down computer. Restart computer, restore checkpoint, keep working. Later on I would like this to work even when F# Interactive is in the middle of an evaluation. I had one eval run for almost 48 hours which turned up a bug with the #time directive.

 

I know one solution would be to write some code unique to the application, but I don't even know what to save other than the mutables and bindings and how to access and restore what needs to be saved.

 

Does anyone know of any possible solution or have any suggestions?

 

 

Thanks, Eric

[Dave Thomas]

So you want to save everything, all the bindings and current values etc?

In essence it's just a dynamic assembly so it has the same scope of limitations as that. Compiling to an assembly on the fly with source code services writes to the disk before returning a memory loaded assembly. I think you would be looking at a lot of reflection unless you were thinking of a serialisation library that can deal with all F# types.

What's the scope and purpose here?

[Eric]

Hi Dave,

 

Thanks, I wasn't thinking along those lines.

 

The project is https://github.com/jack-pappas/NHol which is the logical continuation of https://github.com/jack-pappas/fsharp-logic-examples.

 

The checkpointing comes about from http://www.cl.cam.ac.uk/~jrh13/hol-light/tutorial_220.pdf section 1.4 Checkpointing.

 

Yes Phan is working on this when you are not working with him on Fantomas. Jack is also obviously on this one.

 

Regards, Eric

 

 

 

 

--
--
You received this message because you are subscribed to the Google
Groups "FSharp Open Source Community" group.
To post to this group, send email to fsharp-o...@googlegroups.com
To unsubscribe from this group, send email to
fsharp-opensou...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/fsharp-opensource?hl=en

---
You received this message because you are subscribed to the Google Groups "FSharp Open Source Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fsharp-opensou...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Robert Pickering]

Interesting, I found some discussion of checkpoint tools on Windows on the sysinternals forum, and it seems a number of tools where developed in the WinNT 4.0/Win 2000 era, but development on them seems to have been discontinued. If you do a search for WinNTckpt, Winckp, ChaRM-NT you'll find academic papers about these tools, but no source.  It's unclear why development discontinued, maybe they exploited some security hole in Windows that has since been plugged.

It would be an interesting project to try and port one of the Linux tools to Windows, but also quite tricky and with no guarantee of success. You'd be more like to find people interested in doing this on a forum discussing low level programming in Windows, the sysinternals forums might be a good place to start.

The sysinternals forums also seem to suggest that application level checkpointing maybe a better approach in many case, this is pretty much what Dave suggests and would be a useful addition to FSI.

[Eric]

Hi Robert,

 

Thanks, your suggestions are something that I will look into.

 

The other current working option is to use desktop system virtual machines such as  VMware, VirtualBox or Virtual PC. Most of my experience is with VMware Workstation which requires one to dedicate processors and memory to the app when running and the dedicated processors and memory cannot be shared when running. I can't speak for the other virtual machines applications that run in server farms, but don't believe them to have that restriction, but they do have prohibitive price tags for an individual. The use of such applications might be why they stopped talking about system chceckpointing.

 

This would be of benefit to FSI and I am sure the rest of us on the project will have no problem making this open source and available on Github if we get it to work.

 

Regards, Eric

[Eric]

I did some looking into these suggestions. The dynamic assembly looks promising.

 

Along the way I ran into Workflow Persistence. As I have never used a Workflow and before I spend time researching it more for this, does anyone know if this is a path I should look down or is there a reason it won't work. Based on what I see, using a Workflow is less work than writing reflection and/or serialization code which I know to be a lot of work. 

 

Thinking out loud it seems that FSI is a fence I have to stay on one side or the other to make this work, or make/modify FSI so that I can communicate across.

 

 

 

[Eric]

Found my answer with regards to Workflow Persistence.

 

From Persistence Best Practices

 

"A workflow can only be persisted if all of the data types used by the workflow are serializable"

[Jack Fox]

I might be wrong about this, but possibly Tsumani IDE supports checkpointing? (i.e. saving off IDE state)

[Eric]

I looked for Tsumani IDE but the only thing I find is

 

https://github.com/Tsunami-ide/public

 

which doesn't have much.

 

Any links to learn more?

--

[Jack Fox]

http://tsunami.io/

I don't know why it is so bad at showing up on searches. They have a real SEO problem with their site.

You received this message because you are subscribed to a topic in the Google Groups "FSharp Open Source Community" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/fsharp-opensource/IgUhkn-aFMk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to fsharp-opensou...@googlegroups.com.

[Eric]

FYI

 

If I am understanding this correctly (https://ocaml.janestreet.com/ocaml-core/latest/doc/core_extended/Cache.html), Jane Street is saving function results already with OCaml.