Ce: a successor of Céu (?)

[Francisco Sant'anna]

Hello,

I'm working on a new language prototype:

https://github.com/fsantanna/ce0

Ce is a simple language with algebraic data types, ownership semantics, and scoped memory management.

Céu focuses on the synchronous semantics but overlooks other important aspects, particularly type systems. I'm restarting from scratch with the focus on a modern type system and aiming to incorporate all concurrency characteristics of Céu as time goes by.

The ultimate goal is still to ensure memory/execution safety/predictability.

So, Ce will be something in between C and Céu.

I'm also open for suggestions and contributions.

Francisco

[Johnicholas Hines]

It sounds great! I am excited to take a look at it!

--

---
You received this message because you are subscribed to the Google Groups "The Programming Language Céu" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceu-lang+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceu-lang/CAD4QiZsczGWudHSnzDS6%3D9na-oyO5E6qKrqa3A3hrCouVt8KXg%40mail.gmail.com.

[Job van der Zwan]

A fresh start? Well, given that your interest is mainly in language design *research*, a from-scratch approach to solve design problems doesn't seem like a bad idea I guess :)

> So, Ce will be something in between C and Céu.

Hehehe, punny :)

So I guess you're taking the developments from Rust as a starting point for your own designs. What I was wondering is whether it would be interesting to explore the GALS paradigm that you mentioned by mixing the synchronous (local) concurrency semantics with some actor-based model. The latter need not be part of the language, but maybe it could be something to keep in mind in designing the language so that (for example) writing a library that gives actor-capabilities would be possible.

Also, have you looked at Pony and its type system? What is your opinion on that? To be clear: I have no experience with writing either Rust or Pony, so I have no personal stake here, just curious what your opinions are.

Anyway, looking forward to see what comes out of this new project!

Cheers,

Job

[Luke Whittlesey]

Sounds neat. Thanks for all of the great work.

I like that you included "simple" in the description. I imagine a
simple language is much harder to design, but pays off in the long
run.

I have not had time to look at Ce yet, so this is a generic comment,
and just my opinion.. I think strong type systems can be useful, but
should always come with an escape button. Some way to annotate, cast,
or otherwise tell the compiler to continue even when it can't prove
type safety. I think there will always be valid programs that you
can't get through a type checker, and it can be really tiresome to
have to fight the compiler.

I really like the semantics of Ceu, so I'm excited to hear about you
continuing with the concurrency characteristics of Ceu.

I've used Ceu now in a number of projects and here are some thoughts
(not criticisms) that have come up:

- The interop with C is really useful. If you are going to have a
vector type again it would be useful to make it available to/from C.
This would be used for sharing data between the two domains without
having to fall back to some user defined C type.

- I wonder if not using a stack for emit might be a reasonable idea.
Maybe turn emit into a yield that will resume on the next event loop.
I know it would change the semantics, but it would get rid of the
possibility of a stack overflow. I've inadvertently written some nasty
emit hairballs and it can be hard to debug on something like an
arduino. Or maybe have an emit/yield statement to opt out if the stack
semantics are necessary. Or be able to detect a stack overflow and
somehow flag it through something like a PANIC event.

- Adding a yield statement would make it easier to break up async
code. Right now only loop can do that.

- I think the organism idea (I forget what you call them now) is
neat. It would be quite useful to be able to await methods as well.

- I've needed flow control on a number of occasions and emits are
easily lost if an await is not ready. Maybe have an emit/await
statement that will block the emit until a matching await is ready to
accept. I've come up with manual approaches, but it's easy to get
wrong.

I look forward to learning something new from Ce!

On Wed, Jan 20, 2021 at 1:36 PM Francisco Sant'anna
<francisco...@gmail.com> wrote:
>

[Francisco Sant'anna]

On Wed, Jan 20, 2021 at 5:23 PM Job van der Zwan <j.l.van...@gmail.com> wrote:

So I guess you're taking the developments from Rust as a starting point for your own designs. What I was wondering is whether it would be interesting to explore the GALS paradigm that you mentioned by mixing the synchronous (local) concurrency semantics with some actor-based model. The latter need not be part of the language, but maybe it could be something to keep in mind in designing the language so that (for example) writing a library that gives actor-capabilities would be possible.

In Céu this is possible to some extent with "async/thread". I'm not sure to what extent this should be taken, which leads to the other question. 

Also, have you looked at Pony and its type system? What is your opinion on that? To be clear: I have no experience with writing either Rust or Pony, so I have no personal stake here, just curious what your opinions are.

I don't have any practical experience with Pony and Rust, so my opinions are superficial.

I read an early paper about Pony (I was in the program committee) and it was very good. The capability modifiers are very well designed.

But my feeling is that both Pony and Rust type systems are too complex considering the well-behaved synchronous semantics of Céu: For example, deterministic execution and "lexical parallelism" (which decreases aliasing significantly).

Sharing memory is not a big issue in Céu and it's already very common (the default) to share with mutation, multiple readers, etc, which would lead to a lot of annotations in those type systems.

When you have asynchronous sharing, which is the case of Rust and Pony, then these type systems are justified.

So, providing more and more support for actors would move towards complexity. That's my feeling.

[Francisco Sant'anna]

On Thu, Jan 21, 2021 at 1:03 PM Luke Whittlesey <luke.wh...@gmail.com> wrote:

Sounds neat. Thanks for all of the great work.

I like that you included "simple" in the description. I imagine a
simple language is much harder to design, but pays off in the long
run.

Well, simple there is more in the sense of academic focus exclusively on those features. For example, no vectors, strings, generics, etc, etc, which real languages must support.

That's also true as an overall goal towards meeting Céu, but I guess everyone claims to design as simple as possible.

 

I think strong type systems can be useful, but
should always come with an escape button. Some way to annotate, cast,
or otherwise tell the compiler to continue even when it can't prove
type safety. I think there will always be valid programs that you
can't get through a type checker, and it can be really tiresome to
have to fight the compiler.

Yes. This is very aligned with my goals.

You can have very complex type systems in which you can encode 99% of your problems.

Even there, 20% are still very hard to encode in the name of soundness and you spend hours just to realise a few days later that you actually won't use that code.

For the remaining 1% it might either be impossible or very hard to encode, may using a second language.

I aim for 80% soundness and the remaining 20% should be very easy to workaround.

This is one of the reasons why I believe source compatibility with C is so important. The types, expressions and statements bridges between C->Céu and Céu->C should be a few keystrokes away with zero conversions.

 

  - The interop with C is really useful. If you are going to have a
vector type again it would be useful to make it available to/from C.
This would be used for sharing data between the two domains without
having to fall back to some user defined C type.

Sure. That will probably be vectors+slices and sharing will be like today.

 

  - I wonder if not using a stack for emit might be a reasonable idea.
Maybe turn emit into a yield that will resume on the next event loop.
I know it would change the semantics, but it would get rid of the
possibility of a stack overflow. I've inadvertently written some nasty
emit hairballs and it can be hard to debug on something like an
arduino. Or maybe have an emit/yield statement to opt out if the stack
semantics are necessary. Or be able to detect a stack overflow and
somehow flag it through something like a PANIC event.

That's not the first time I see people questioning the stack semantics.

I'd be happy to see these hairy examples simplified to discuss.

It should only be possible to have a stack overflow if you have a loop w/o an external await.

You can always use an "async do end" as an yield, but it's a code smell.

In some situations, internal emit/await should be substituted by "spawn" of some "code/await".

  - Adding a yield statement would make it easier to break up async
code. Right now only loop can do that.

I didn't understand this one.

 

  - I think the organism idea (I forget what you call them now) is
neat. It would be quite useful to be able to await methods as well.

It's called "code/await" and will probably be renamed to "task".

If you substitute "code/tight" (method) to "code/await" you can await a method. This is already supported.

  - I've needed flow control on a number of occasions and emits are
easily lost if an await is not ready. Maybe have an emit/await
statement that will block the emit until a matching await is ready to
accept. I've come up with manual approaches, but it's easy to get
wrong.

Again, this seems like you should "spawn" something instead of "emit".

Thanks for taking your time!

[Luke Whittlesey]

Thank you for your responses.

Yes. This is very aligned with my goals.

You can have very complex type systems in which you can encode 99% of your problems.

Even there, 20% are still very hard to encode in the name of soundness and you spend hours just to realise a few days later that you actually won't use that code.

For the remaining 1% it might either be impossible or very hard to encode, may using a second language.

I aim for 80% soundness and the remaining 20% should be very easy to workaround.

This is one of the reasons why I believe source compatibility with C is so important. The types, expressions and statements bridges between C->Céu and Céu->C should be a few keystrokes away with zero conversions.

Well said.

That's not the first time I see people questioning the stack semantics.

I'd be happy to see these hairy examples simplified to discuss.

It should only be possible to have a stack overflow if you have a loop w/o an external await.

You can always use an "async do end" as an yield, but it's a code smell.

In some situations, internal emit/await should be substituted by "spawn" of some "code/await".

Unfortunately it's been difficult for me to pinpoint some of the issues and it only seems to catch me in complex code. Sometimes I'll add an `await 1ms` somewhere and the problem goes away. This makes me think of a stack problem, but it's just a guess really. It's probably my own fault somewhere. I'll try to find a simple example if I can. Thanks for the "async do end" trick; it's better than "await 1ms".

  - Adding a yield statement would make it easier to break up async
code. Right now only loop can do that.

I didn't understand this one.

An example:

async do

    // loop will send control back to the main event loop

    loop do

        // some code here

        ...takes 0.5 seconds...

        // a yield might be used to return control to the main loop

        yield

        // more code here

       ...takes 0.5 seconds...

    end

end

I've written some code that is fairly cpu intensive, not so much that I want a separate thread, but enough that I still worry about staying responsive to an external event. Async is a good fit, but having a yield statement that I might call from within an async block might be useful. Would the "async do end" trick work here; i.e. nested?

It's called "code/await" and will probably be renamed to "task".

If you substitute "code/tight" (method) to "code/await" you can await a method. This is already supported.

I forgot that it was in the roadplan, but I don't think it's currently supported by the ceu compiler right now. It's nice to know Ce will have it.

  - I've needed flow control on a number of occasions and emits are
easily lost if an await is not ready. Maybe have an emit/await
statement that will block the emit until a matching await is ready to
accept. I've come up with manual approaches, but it's easy to get
wrong.

Again, this seems like you should "spawn" something instead of "emit".

I'll have to think on this.

I have some other half thought out (but not necessarily good) ideas for a ceu successor:

  - Is there value in turning par/or and maybe even par/and into expressions that return values?

  - Is there value in being able to pause and resume a running process? Maybe I have a deeply nested par statement and I want to pause but not cancel the whole tree.

--

---
You received this message because you are subscribed to the Google Groups "The Programming Language Céu" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceu-lang+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceu-lang/CAD4QiZt3n%2Br_ZxWJi2HZ%2BK0dk4dHstdk0ysnQcVRYmgD3qx-YA%40mail.gmail.com.

[Francisco Sant'anna]

On Mon, Jan 25, 2021 at 5:59 PM Luke Whittlesey <luke.wh...@gmail.com> wrote:

  - Adding a yield statement would make it easier to break up async
code. Right now only loop can do that.

I didn't understand this one.

An example:

async do

    // loop will send control back to the main event loop

    loop do

        // some code here

        ...takes 0.5 seconds...

        // a yield might be used to return control to the main loop

        yield

        // more code here

       ...takes 0.5 seconds...

    end

end

I've written some code that is fairly cpu intensive, not so much that I want a separate thread, but enough that I still worry about staying responsive to an external event. Async is a good fit, but having a yield statement that I might call from within an async block might be useful. Would the "async do end" trick work here; i.e. nested?

I see. So these codes that take 0.5 seconds are not loops

I'm asking this because Céu inserts an yield automatically at the end of every loop iteration.

If they are not loops, are they native calls? This is the only situation I can imagine in which you would require an yield in the middle of an async loop.

It's called "code/await" and will probably be renamed to "task".

If you substitute "code/tight" (method) to "code/await" you can await a method. This is already supported.

I forgot that it was in the roadplan, but I don't think it's currently supported by the ceu compiler right now. It's nice to know Ce will have it.

Céu already supports this:

https://fsantanna.github.io/pingus/#continuation-passing-2

The "Worldmap", "Story" and "Credits" are methods (code/awaits) that execute sequentially and are awaited in the call site.

  - Is there value in turning par/or and maybe even par/and into expressions that return values?

In terms of implementation, turning statements into expressions is harder than I initially thought.

My plan is to keep them separate.

However, what you can do is to enclose everything with a "do-end", which can be assigned with an "escape" that terminates everything nested:

var int ret = do

    par do

      ...

    with

      ... escape 10; ...   // this assigns 10 to "ret" and aborts the whole block nested below the "do-end"

    end

end;

 

  - Is there value in being able to pause and resume a running process? Maybe I have a deeply nested par statement and I want to pause but not cancel the whole tree.

Yes! This is very interesting and is supported with "pause/if":

https://github.com/fsantanna/ceu-sdl-birds/blob/master/src/birds-12.ceu#L71

This mechanism was stolen from Esterel as usual (they call it "suspend").

[Luke Whittlesey]

I see. So these codes that take 0.5 seconds are not loops

I'm asking this because Céu inserts an yield automatically at the end of every loop iteration.

If they are not loops, are they native calls? This is the only situation I can imagine in which you would require an yield in the middle of an async loop.

Yes, a series of native calls that take some cpu time, but aren't necessarily in a nested loop.

Céu already supports this:

https://fsantanna.github.io/pingus/#continuation-passing-2

The "Worldmap", "Story" and "Credits" are methods (code/awaits) that execute sequentially and are awaited in the call site.

This was an interesting read. I think I wasn't clear on what I meant with the organism method.

Something like this, but it raises a compiler `not implemented error`.

````

code/await Org (var int x) -> NEVER do
    code/await Method (var int v) -> int do
        await 1s;
        escape outer.x + v;
    end
    await FOREVER;
end

do

    var& Org org = spawn Org(1);
    var int result = await org.Method(10);

end

````

I've also tried this approach, but something goes wrong in the runtime.

````

code/await Org (var int init_x) -> (var int x) -> NEVER do
    x = init_x;
    await FOREVER;
end

code/await OrgMethod (var& Org org, var int v) -> int do
    await 1s;
    escape org.x + v;
end

do
    var& Org org = spawn Org(1);

    var int result = await OrgMethod(&org, 10);

end

````

A use case might be managing a serial port where multiple users can write datagrams, but the datagrams are not interleaved. The main organism would manage the actual writing in a non-blocking way and an inner code/await "method" would manage access among multiple users. Anyway, I don't mean to distract with this. There are other ways to go about this. I guess it's more of a syntax sugar thing.

Yes! This is very interesting and is supported with "pause/if":

Ah, I see it in the documentation, but somehow I missed it before. Thanks!

--

---
You received this message because you are subscribed to the Google Groups "The Programming Language Céu" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceu-lang+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceu-lang/CAD4QiZvOOsF0NpDEg%2Br73_v97K6hZg1MEX506itp_nXqTE7zbA%40mail.gmail.com.

[Francisco Sant'anna]

On Wed, Jan 27, 2021 at 2:35 PM Luke Whittlesey <luke.wh...@gmail.com> wrote:

I see. So these codes that take 0.5 seconds are not loops

I'm asking this because Céu inserts an yield automatically at the end of every loop iteration.

If they are not loops, are they native calls? This is the only situation I can imagine in which you would require an yield in the middle of an async loop.

Yes, a series of native calls that take some cpu time, but aren't necessarily in a nested loop.

Ok. Makes sense then.

 

Céu already supports this:

https://fsantanna.github.io/pingus/#continuation-passing-2

The "Worldmap", "Story" and "Credits" are methods (code/awaits) that execute sequentially and are awaited in the call site.

This was an interesting read. I think I wasn't clear on what I meant with the organism method.

Something like this, but it raises a compiler `not implemented error`.

Oh, ok. Nested code/awaits are not supported. It's been a while, but I remember that it was somewhat tricky to implement and I knew I would rewrite the language, so I didn't bother.

[Alan Guedes]

Hi Francisco.

It sounds interesting.

Just a curiosity. Why implement it using kotlin instead of c/lua like ceu?

Best regards.
-- Alan Guedes.

[Francisco Sant'anna]

On Sat, Jan 15, 2022 at 3:12 PM Alan Guedes <alan...@gmail.com> wrote:

Just a curiosity. Why implement it using kotlin instead of c/lua like ceu?

Kotlin is typed and has good library support to work with data and functional programming.

It is also more portable to execute since its Java.

For a compiler the other languages are not the best options.

I used Lua in the past because I wanted to learn LPeg.

[Francisco Sant'anna]

This is a follow up after 1 year! Unfortunately, I could not work on the language during most part of the year.

1. After a lot of exploration an ponderation, I gave up on the ownership semantics.

    - It's too complex and does solve the main problem of dynamic allocation and aliasing across callbacks.

    - So, the main mechanism is now memory regions. Fine-grained allocation should be manual.

As some evidence, Rust programmers xxx the type system  use numeric indexes and/or reference counting, which are both poor solution in comparison to synchronous concurrency enables.

2. The language implementation is split in two.

    - An extremely verbose core: https://github.com/fsantanna/ce0

    - An extension that compiles to the core: https://github.com/fsantanna/ce1

    - Everything that ce1 can do ce0 can also do but with tons of annotations.

3. This is the new description:

Ce is a simple language with algebraic data types, pointers, first-class functions, and region-based memory management. The main goal of Ce is to support safe memory management for dynamically allocated data structures.
An allocated data is always attached to a specific block and cannot move. When a block terminates, all attached allocations are automatically released. This prevents memory leaks. A pointer is also attached to a specific block and cannot point to data allocated in nested blocks. This prevents dangling pointer dereferencing. These ideas have been successfully adopted in Cyclone: https://cyclone.thelanguage.org/

4. Tasks:

The next step is to add a task construct that I would describe as a "lexical coroutine".

They have a similar API to coroutines but a semantics that is closer to Céu trails (broadcast, finalization, bounded memory).

For instance, the par constructs (par, par/and, par/or, watching, etc) would go to ce1 and translate to tasks.

Hope to have something equivalent in functionality to Céu this year.

Regards,

Francisco

[Francisco Sant'anna]

On Tue, Jan 18, 2022 at 2:49 PM Francisco Sant'anna <francisco...@gmail.com> wrote:

As some evidence, Rust programmers xxx the type system  use numeric indexes and/or reference counting, which are both poor solution in comparison to synchronous concurrency enables.

 As some evidence, Rust programmers trick their own type system with numeric indexes to deceive the borrow checker and/or rely on reference counting, which are both poor solutions in comparison to what synchronous concurrency enables.