Modelling domains declaratively with Eve (blog post)

[Joe Geldart]

Hello,

I have started work on a first blog post based on my earlier thoughts around Eve and requirements engineering. This post is a quick look at translating domain requirements, like business rules, into Eve code in such a way that the mapping is obvious and inspectable by humans. I'd love constructive feedback and comments about the post before I publish it properly. It is on Medium, so you can comment there if you have a Medium account, or here if you don't.

https://medium.com/@arnia/modelling-domains-declaratively-with-eve-a1eb93b80942

If there is interest, I intend for this to be the first in a series looking at different ways of thinking about Eve programs to reduce the burden of mapping between the problem and solution domains. The next post would likely look at translating behavioural requirements.

Thanks,

Joe

[Chris Granger]

This is really awesome, thanks for sharing!

Everything looks great. Your conditional example isn't quite right though. As it's written if there are no administrators, the block won't fire.

```

search

  subject = [#team-plan]

  team-members = subject.team-member

  member-count = count[given: team-members, per: subject]

  member-count > 5

  not(subject.administrator)

bind @error

  [#error subject member-count administrator-count message: "{{subject.name}} must have at least 1 administrator if {{subject.name}} has more than 5 members. It has {{administrator-count}} administrators and {{member-count}} members"]

```

That will give you the case where the subject has no administrator. If you wanted to change the condition to must have at least two you'd do

not(administrators = subject.administrator

  administrator-count = count[given: administrators, per: subject]

  administrator-count >= 2)

This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. If you are not the intended recipient of this email, you must neither take action based upon its contents, nor copy or show it anyone. Please contact the sender if you believe you have received this email in error.

--
You received this message because you are subscribed to the Google Groups "Eve talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eve-talk+u...@googlegroups.com.
To post to this group, send email to eve-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/eve-talk/0b5f3317-e37f-4443-8b20-06bd2f82a4af%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Chris Granger]

whoops, left out the explanation. The original block won't fire because since the query is for admins and there are none we'll bail. By using not, we success when the conditions fail, which is exactly what we want in this case; the block should succeed if there are no admins.

[Joe Geldart]

Thanks! I've updated the code.

[Joe Geldart]

I have now published the post, so feel free to share with whoever you think would find it interesting :)

[Ravil Bayramgalin]

Hi, Joe.

I've found the same type of mistake in your post as Chris but in another block:

```

  member-count = count[given: subject.team-member, per: subject]

  member-count < 2

```

If there is no subject.team-member than the block won't fire.

Therefore in its current state it will execute only when member-count = 1. 

A fix:

```

  not(

  member-count = count[given: subject.team-member, per: subject]

  member-count >= 2

  )

```

Thanks for the post.

This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. If you are not the intended recipient of this email, you must neither take action based upon its contents, nor copy or show it anyone. Please contact the sender if you believe you have received this email in error.

--
You received this message because you are subscribed to the Google Groups "Eve talk" group.

To unsubscribe from this group and stop receiving emails from it, send an email to eve-talk+unsubscribe@googlegroups.com.

To post to this group, send email to eve-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/eve-talk/6270381c-d72d-466a-80db-182829fefbbe%40googlegroups.com.

[Joe Geldart]

Thanks for the correction! I've fixed the block. This is making me wonder whether cardinality needs a somewhat special treatment in the semantics. This seems like a 'natural' error that will happen a lot, and be a bit hard to trace. That said, any exceptions to the simplicity of the semantics should be very strongly justified, so maybe we just need to include it as 'something you must remember' in any introductory tutorials.

To unsubscribe from this group and stop receiving emails from it, send an email to eve-talk+u...@googlegroups.com.

To post to this group, send email to eve-...@googlegroups.com.

[Chris Granger]

It's a tough problem and has pretty far reaching consequences. Allowing something like a left join to enable counting the empty set would force us into introducing null to the language, which we've tried very hard to keep out. Null is disastrously bad for programming semantics in general and it also has serious implications on the kind of optimizations and analysis we can do. So we'd have to be talking about a pretty fundamental issue before the tradeoffs might work out.

That being said, I agree with you. One typically thinks of count as being able to produce 0, but in our case it really can't. There are two ways to deal with that, one is to invert and use negation like we did above. The other is to use if to check for the absence in one branch and the present condition in the other. To aid with this, we want to make this statement false: "and be a bit hard to trace." We actually have complete information there and can tell you exactly why a thing isn't firing, so while I don't have a good solution for removing that edge, I think we can address it when it comes up.

To view this discussion on the web visit https://groups.google.com/d/msgid/eve-talk/0fcf6c24-40ad-4b8f-a682-fe2701246e63%40googlegroups.com.

[Nick Smith]

It's a tough problem and has pretty far reaching consequences. Allowing something like a left join to enable counting the empty set would force us into introducing null to the language, which we've tried very hard to keep out. Null is disastrously bad for programming semantics in general and it also has serious implications on the kind of optimizations and analysis we can do. So we'd have to be talking about a pretty fundamental issue before the tradeoffs might work out.

I'm not familiar with the underlying machinery, but I don't understand why aggregation can't be treated specially. At the user-facing level, inputs to aggregation operators are like a sub-query that returns a list of results.

If I've got:

count[given: x]

then the system should ideally query for x, return a (possibly empty) list of results, and then let the aggregate operator do what it needs to do with that list. This also opens the door for a user to write their own aggregates, and any other function that naturally accepts an enumerable set of objects as input. Correct me if I'm wrong on this, but we probably need an interface between "the land of sets" and "the land of collections" anyway to implement many fundamental algorithms.

[Chris Granger]

That would either still require null, or require having actual subqueries which are a rats nest of complexity. Here's why it would introduce null:

```

search

  class = [#class student]

  student-count = count[given: student, per: class]

bind @browser

  [#div text: "{{class.name}} - {{student.name}} - {{student-count}}"]

```

In order for this to report 0, it also needs to report a student's name, but there are no students. We could say well then this just doesn't return a result for that class, because it also references student, but now we're getting into rules that become very hard to reason about. What are the boundaries? What affects what here? If I put student in a different div, would the class div then show up? If we allowed that, what happens if instead of drawing divs, this was purchasing food? Is it ok that some of my commits went through while others didn't? How would I reason about that partial failure?

A valid solution to those questions would be explicit subqueries, which is what we had for several versions of Eve, but they trade all of this for a different kind of complexity. For one, you end up with nested scopes, which we found to be one of the most confusing things to novices. Worse than that though, is that you now have to constantly consider cardinality. Every subquery is a cardinality island and you have to think about how adding or removing records from a search might cause your cardinalities to change. In the current semantics, we take the trade of making absence a little more awkward for not having to care about any of that. Instead, you explicitly tell us what unique things to aggregate over and regardless of how many times they may have been crossed (e.g. you look for both shoes and fruits) you will still only count what you asked for. We could keep those same semantics and also allow subqueries, but then the question is whether or not the gain is worth the complexity. Given the issues it brings up with scope and the complexity it would add to the syntax, it doesn't seem like counting 0 is worth it.

To view this discussion on the web visit https://groups.google.com/d/msgid/eve-talk/1dd790d7-6434-4336-aefe-a14396707066%40googlegroups.com.

[Nick Smith]

We could say well then this just doesn't return a result for that class, because it also references student, but now we're getting into rules that become very hard to reason about. What are the boundaries? What affects what here? If I put student in a different div, would the class div then show up?

I think the semantics are quite simple there. In fact, they're almost identical to what they are now, namely:

• The number of instances of a bound/committed record is equal to the product of the cardinalities of the sets (variables) it references.

The difference from the current behaviour being that sets that aren't put into a record could have cardinality zero without everything falling apart. I understand this as removing edge cases, not adding them. And I think the reasoning becomes simpler too: to determine how many instances of a record will be committed, you just need to reason about the sets (variables) it references. You don't need to worry about any unrelated sets that appear in the same block and what their cardinality might be.

If we allowed that, what happens if instead of drawing divs, this was purchasing food? Is it ok that some of my commits went through while others didn't? How would I reason about that partial failure?

 

In certain situations it might make sense for certain records to be "missing". It would depend on the details of the particular use-case. If you want to ensure that records are committed, you can add an assertion/error check that the cardinality of a certain record is 1. This is no different from how you'd have to handle situations under the current semantics where an entire (as opposed to partial) search/commit unexpectedly finds/creates zero results.

As far as reasoning about the failure, a visual tool for inspecting block (non-)execution would make it obvious what happened. Input has cardinality zero => output has cardinality zero. Just annotate each variable with its cardinality and follow the trail of zeros.

A valid solution to those questions would be explicit subqueries

r.e. this, I don't think explicitly nesting subqueries is necessary, and I agree it would add too much complexity. 

[Chris Granger]

It becomes very hard to talk about constraints that don't contribute to an output in that scheme. E.g. how do I get a list of people who don't have dogs?

```

search

  p = [#person]

  not([#dog owner: p])

bind

  [#div text: "{{p.name}} doesn't have a dog"]

```

To view this discussion on the web visit https://groups.google.com/d/msgid/eve-talk/068a328b-4efc-408b-9587-3deb15efee06%40googlegroups.com.

[Nick Smith]

In that example, the not-dog-owner constraint affects the set p, from which "p.name" comes, and thus most definitely contributes to the output. So it still makes sense. Just highlight the relevant constraints when you hover over the div record, so the user doesn't have to work it out themselves.

Here is a different example which highlights the issue I think you were getting at:

```

search

  p = [#person]

  not(p = [#person name])

bind @browser

  [#div text: "Warning: some people do not have names!"]

```

This can be solved without too much trouble, if we tweak the semantics a bit:

```

search

  p = [#person]

  not(p = [#person name])

bind @browser

  if p then [#div text: "Warning: some people do not have names!"]

```

Or alternately:

```

search

  a = [#person]

  b = [#person name]

bind @browser

  if a != b then [#div text: "Warning: some people do not have names!"]

```

The distinction between the search and bind actions is starting to blur a bit now. This actually starts linking back to some thoughts I brought up a few days ago about not needing to cordon off behaviours into blocks. If we descend deeper into this rabbit hole, we can start to think about things a little differently.

Warning: the implications of these ideas are not fully understood. Feedback is welcomed. If I say anything stupid, it's because I'm not an expert on Eve and/or database implementation. 

Also, ignore the code highlighting in the below examples. I just used code blocks for readability purposes.

Beyond blocks

Let us switch to using the  ←  symbol to denote binding a record to a particular database. We can use it like this:

 a = [#person]

 b = [#person name]

 if a != b then

   browser ← [#div text: "Warning: some people do not have names!"]

Note that we don't need explicit search and bind/commit phases anymore (we'd need a variant of the arrow for commit too). But (to play devil's advocate) that implies we could write one huge chunk of code that intersperses searching and binding, which would make it difficult to see what constraints affect what bind actions. To resolve this I'd propose that only constraints that appear before a bind should apply to it. I think this makes the most intuitive sense to a reader anyway.  A second essential requirement will be a basic notion of scope, which I referred to in my earlier post as contexts, to stop all our code from polluting a single shared namespace. Let's use the syntax { } to represent a context.

A database is a context. Here's how we'd initialise one:

browser = {}

We can make our block from earlier a context too:

{
  a = [#person]
  b = [#person name]
 
  if a != b then browser ← [#div text: "Warning: some people do not have names!"]
}

This just means that variables a and b aren't accessible from outside. Searching for a [#person] here would implicitly be searching in the current and parent contexts. 

To search for records in a particular "customers" context:

{
  c = [#customer] ← customers
  number = count[given: c]

  browser ← [#div text: "Found {{number}} customers."]
}

What happened to block atomicity?

We seem to have lost the ability to reason about when records will be committed and when changes will be seen by the world. That can be regained by treating contexts as the new unit of atomicity, i.e. a context's effect on the world (i.e. other contexts) is only seen after all computations within it have finished.


Representing processes

To revisit the thread about how to represent processes, we now have a way to do it:

{
  browser ← [#label text: "Enter your name: ", children: [#input type: "text"]]
  [#input value] ← browser
  browser ← [#div text: "Hello {{value}}!"]

}

Forgive me if the actual DOM stuff is erroneous. When this context first runs, a label and input box will be committed the browser. There initially won't be a filled input box in the browser, so the div, which relies on the value, won't be committed. Thus at the end of the first execution we have:

• Label and input box appear in the browser

Now that the user can see the input box, they can type something in. Once they do, this triggers the context to run again. The input and the value now exist, so the div is committed. Thus at the end of the second execution we have:

• Label and input box are still present
• The div appears

Those are my rough, initial thoughts on how processes can be implemented. The advantage over how it must be done in Eve today is that the logic doesn't have to be broken up into a bunch of separate blocks. I think it should be able to extend to more complex scenarios, but there are assuredly many more details to consider.

Relationship between contexts, records, and attributes

So far we've been looking at contexts for records. But, we can also consider a record to be a context for attributes, and an attribute can be considered a context for values. Under these semantics the ← operator naturally obviates a few other operators presently in Eve.

Take for example the merge operator (<-), which combines two records. We can interpret it as merging one set of attributes (contained in a record) into another.

Before:

search

  celia = [#Celia]

bind

  celia <- [#student grade: 10, school: "East"]

After:

celia = [#Celia]

celia ← [#student grade: 10, school: "East"]

Or more succinctly:

[#Celia] ← [#student grade: 10, school: "East"]

Consider also the add operator (+=), which we can interpret as merging one set of values (contained in an attribute) into another.

Before:

search
  [#friends person1 person2]
bind
  person1.friend += person2
  person2.friend += person1

After:

[#friends person1 person2]
person1.friend ← person2
person2.friend ← person1

Functions

I'm still working out syntax, but functions can be represented using contexts too, if we treat them as attribute contexts (i.e. good old records). That means we're switching to square [ ] brackets:

square(x) = [
  value: x * x
]

x = 5
s = value ← square ← x

browser ← [#div text: "Squaring {{x}} yields {{s.value}}."]

Here the x attribute can be interpreted as a missing attribute in the context "square" which is provided by the caller.

Therefore:

square ← x

evaluates to:

square = [
  x: 5
  value: 25
]

Which is why we can read the "value" attribute to get the result.

How is this any better than Eve as it stands today?

To summarise, this is aiming to solve a few problems that have been identified in Eve today:

• Problems with zero cardinalities (see earlier posts in the thread). I consider these problems baked into Eve's block-based execution model.
• The lack of a means to express sequential and iterative processes elegantly and without polluting a database with temporary records.

But I think there are other substantial benefits to be gained:

• Databases, records, and attributes can be understood uniformly as "contexts".
• Iterative/recursive functions can be implemented as contexts which commit values back to themselves for the subsequent iteration. This means we potentially have an elegant way to express any computation in Eve.

Just to reiterate, there many be some substantial issues with some of the things that I have said. Looking forward to getting feedback on it all.

[Chris Granger]

Hi Nick,

Thanks for the detailed proposal! I wanted to think about a lot of the ideas here before I jumped into a response. Talking about semantics is tough, because we have to be ridiculously precise - small differences in understanding can equate to vastly different outcomes. As such, I'm going to err on the side of being overly exact so that we can suss out what some of this would mean.

> Just highlight the relevant constraints when you hover over the div record, so the user doesn't have to work it out themselves.

This is an appealing idea, but it has its downsides. It means that reading *needs* to be tool assisted, since it's very easy to miss complex interactions between variables. It also requires direct interaction on the part of the user, which is a pretty high price to pay. In general, our goal is to minimize reasoning costs as much as possible, which while we can create prosthetics to help with still leaves the complexity on the table.

The current all-or-nothing semantics reduce the failure mode to "this block didn't match" as opposed to "something in this block related to this kind of record didn't match". While that may not seem like a big difference, reasoning about what does or does not affect a set adds a lot of cognitive overhead. It requires arbitrary rules around how aspects of the language like not and aggregates play a part. Each rule like that is something that the user has to internalize to reason about their program. Right now, the only thing you have to be cognizant of is that you can't count nothing.

```

search

  a = [#person]

  b = [#person name]

bind @browser

  if a != b then [#div text: "Warning: some people do not have names!"]

```

FWIW, this example actually means something else. It says find all the pairs of records tagged #person where at least one of them has a name. `a != b` enforces that the two person records can't be the same record. That's different than saying find all the people without a name. Given the next statement about removing the distinction between searching and updating though, we could make that work out like so:

```

search

  a = [#person]

bind @browser

  if not(a.name) then [#div text: "Warning: some people do not have names!"]

```

As you're showing in the beyond blocks section though, there's nothing that technically prevents us from removing the `search` and `commit` fences. For what it's worth though, we intentionally designed the syntax to make search and update distinct. The current blocks in Eve can be read as "when I see this, do this" and because it is enforced at the language level, that remains entirely consistent. Once you've seen one block, you've seen them all. Similar to the bit above, the goal here is to reduce the cost of reasoning. The separation makes it easy to reason about "what does this block look for?" and "what does this block do?". Imagine we're scanning the codebase for blocks that add todos. Because updates are separate from searches, we can ignore all of the search sections as we go. If they weren't separated thought, we'd have to look at everything and piece together which bits are updates and which parts are just looking for todos. This is perhaps more daunting in Eve than in other languages since updating and searching look very similar. So to foster that ability to reason about what's going on, we formalized the pattern: "when Eve sees these things, Eve's going to do these things."

> A second essential requirement will be a basic notion of scope, which I referred to in my earlier post as contexts, to stop all our code from polluting a single shared namespace.

For the most part this is reintroducing blocks with a different syntax. The notion that everything is a context is exciting though. If you think of facts in the database as themselves being declarative statements, then a context is a container for declarative statements related to some shared information. E.g. facts in the browser database are declarative statements about the browser and "code" is a set of declarative statements that refine or produce other statements about the browser. I don't think there's anything wrong with that conceptually. It does introduce either arbitrary nesting of contexts or somewhat arbitrary rules around nesting though. Both of which lead to a lot of complexity. When we first embarked on our journey for Eve, one of the big things we found wrong with programming was scope. Nested scopes are very confusing, especially to non-programmers, and they introduce a forced structure to the program, which makes it much harder to do literate programming correctly. I don't know if there are ways around that, but it's something that would need to be considered pretty carefully.

> a context's effect on the world (i.e. other contexts) is only seen after all computations within it have finished

This adds global and local ordering to the semantics which we'd need to be able to reason about. Currently, Eve blocks execute without order which means they can be composed arbitrarily. If we impose an ordering, we lose a lot - from the ability to place blocks anywhere in the narrative to guarantees around distributed evaluation. Most importantly, we now have to always reason about the order of our code as opposed to being able to make declarative statements about the system. We're used to doing that in imperative and functional code, but it comes at a cost and with some difficult semantics. Async/await, promises, CSP, and so on are all attempts at addressing that cost.

```

{
  browser ← [#label text: "Enter your name: ", children: [#input type: "text"]]
  [#input value] ← browser
  browser ← [#div text: "Hello {{value}}!"]

}

```

This example more or less boils down to the semantics of async/await, but I'm not sure how it would behave in a reactive language. Let's pretend that we join the value on something else in our system and by the time the value is available, that something has been removed. What happens? How do I reason about where execution is in that pipeline? How do I compose things into that pipeline? If we're waiting on an asynchronous thing to return, do all the other blocks continue forward? If so, how does that play into our previous statement that a context has to finish before anyone can see the results? What happens if I type really fast and we're still waiting on web requests - do more instances of the block run and wait? How do I manage those?

There are lots of weird edges to dealing with asynchrony in the context of an ordered language. If things are allowed to stall a pipe, we have to consider what that means for the rest of the program and both of the straightforward answers (block or continue) lead to very difficult to reason about situations. At the cost of writing multiple blocks, the current semantics remove those from consideration. Blocks react only to the presence or absence of data. How or when that data comes to be doesn't really matter. It seems like if we could keep that property and find a way to reduce the overhead of writing multiple blocks, we'd be in a great place.

> Functions

In terms of functions, I think we're thinking about things mostly in the same way. :) The one implication here that we haven't really talked about is the need to be able to say something "instantly". Right now, commit (which is what bind also resolves down to) only allows you to say what the next state of something will be. That doesn't work if we're trying to react to a click which disappears after one tick. We need to be able to say something about "now" - e.g. sqr[value: 5] = 25. This needs more thought as there may be some other way to talk about that, but it is at least one way forward and is the heart of what you're showing here.

> The lack of a means to express sequential and iterative processes elegantly

FWIW, the current semantics allow for arbitrary recursion, so there shouldn't be anything fundamentally preventing this. Like the pipelining stuff, we need to write more code to see what the cost is here and then how we might address that cost.

> Looking forward to getting feedback on it all.

This was absolutely fantastic and thought provoking. Thank you so much for taking the time to write it out! It's really encouraging to see people engaging so deeply with us.

Cheers,

Chris.

To view this discussion on the web visit https://groups.google.com/d/msgid/eve-talk/2b77f7c0-1d9b-4524-a47b-39b1786e6dc0%40googlegroups.com.

[Zubair Quraishi]

I like the idea of adding the context to the expressions, as it is clean:

search
  [#input value] ← browser
bind

  browser ← [#div text: "Hello {{value}}!"]

  browser ← [#label text: "Enter your name: ", children: [#input type: "text"]]