Thoughts and Questions (and Thanks!)

[Satshabad Khalsa]

First off, thanks Zach for writing an elegant and cogent book that really does succeed in putting into words what I feel like I already knew. 

As I was reading I had some questions, would love to hear anyone's thoughts as to some answers.

Pg. 111

The book says we can only do three things to data, "accrete, reduce, and reshape it". 

What about a function that takes a word in spanish and returns a word in english? What operation are we performing on our input in this case? Is this just a reduction of our spanish word into an english word?

Pg. 118

"It is useful and often necessary for the completion of a task to be acknowledged back up the chain ... This state should exist in a single place at the edge of the system so that other processes can safely forget a command once they've passed it along."

This actually applies to a real world example at my day job. 

Proc A communicates to Proc B via PubSub in Redis. Proc A sets a key, and doesn't worry about Proc B acknowledging the completion of the command, or even the receiving of the command. It's nice because Proc A doesn't have to wait for Proc B at all. However there are times when Proc B doesn't receive the command for whatever reason. 

How can that failure be recorded in any process in the system without introducing a step where Proc A waits to know if Proc B received the command? Or is this just exactly what queues are made for? If you're answer is "queue", I don't see how a queue can't just be Proc B in this example. 

Pg 90

Questions about "principled component" vs "principled system" and my guesses at answers.

What's the relation between a "component", "system" and "module"?

> A component can be a process, module, or a single function. A system is made of components. A component can not be stretched across more than one process.

What's a concrete example of a "central design principle"?

> "In the geocoding component, all latitude and longitude data is represented as a 2-vector of [lat lon]"

Is a principle component internally structured as a hierarchy, or is that just a principled system?

> A principle component can be structured as a hierarchy if the component is large enough

Pg 113

"The semantics of our data are defined by the effects it produces when passed into our functions. ... The push phase begins wherever our functions perform effects. Put another way, it begins where the meaning of our data is defined."

In that quote, is the place "the meaning of our data defined" just the functions that will interpret the data?

Would you consider a pull from out of process "performing an effect"?

Pg 114

"A process is composed of operational phases at the edges and a functional phase at the middle"

^ These are my favorite programs to write, but too often I have to write a program that looks like:

- Pull data

- Transform data

- Pull data

- Transform data

- Pull data

- Transform data

- Push data

For example, making a series of web requests who's results depend on each other. Where are the edges and middle of the system here? I guess the only thing to do is keep the transforms free of side effects?

Pg 65

The book says that we can't create an "open" conditional by putting the predicates and clauses into a data structure because of the possible overlapping cases and non determinant order.

But what if we also encode the order in this data structure? Then the code that interprets the data structure could just use the encoded order. What am I missing?

Loved the model for modules. I work in python now so I struggle between OOM patterns and my favored clojure functional patterns. Thinking at the module level (a collection of functions and data, but most importantly names and interfaces) really clicks for me. It doesn't matter if the module exposes objects or dicts or functions, but getting clear on the invariants and assumptions is super helpful.

I really enjoyed the read, thanks for putting so much into it Zach!

[Zach Tellman]

On Sun, Mar 10, 2019 at 9:30 PM, Satshabad Khalsa <satshab...@gmail.com> wrote:

First off, thanks Zach for writing an elegant and cogent book that really does succeed in putting into words what I feel like I already knew. 

As I was reading I had some questions, would love to hear anyone's thoughts as to some answers.

Pg. 111

The book says we can only do three things to data, "accrete, reduce, and reshape it". 

What about a function that takes a word in spanish and returns a word in english? What operation are we performing on our input in this case? Is this just a reduction of our spanish word into an english word?

If translation between languages were lossless (we could translate from Spanish → English → Spanish without any changes to the text), that would qualify as reshaping, but since translation almost always changes the connotation of a word or phrase (losing some, adding others), we're very much in the land of abstraction where we're treating different things as if they're the same.  

So in short, I think translation aspires to reshape, but in practice it reduces.

Pg. 118

"It is useful and often necessary for the completion of a task to be acknowledged back up the chain ... This state should exist in a single place at the edge of the system so that other processes can safely forget a command once they've passed it along."

This actually applies to a real world example at my day job. 

Proc A communicates to Proc B via PubSub in Redis. Proc A sets a key, and doesn't worry about Proc B acknowledging the completion of the command, or even the receiving of the command. It's nice because Proc A doesn't have to wait for Proc B at all. However there are times when Proc B doesn't receive the command for whatever reason. 

How can that failure be recorded in any process in the system without introducing a step where Proc A waits to know if Proc B received the command? Or is this just exactly what queues are made for? If you're answer is "queue", I don't see how a queue can't just be Proc B in this example. 

A protocol is a means of ensuring a task is completed, even if there are faulty processes or channels of communication.  For TCP or any other queue, the "task" is simply conveying messages, but in most applications the task is actually *doing* the thing the messages describe.  

You probably want some way for Proc B to acknowledge it's received *and* acted upon the signal, and some way for Proc A to resignal if no acknowledgement comes within a given time.  Of course, this assumes the work done by Proc B is idempotent.

Pg 90

Questions about "principled component" vs "principled system" and my guesses at answers.

What's the relation between a "component", "system" and "module"?

> A component can be a process, module, or a single function. A system is made of components. A component can not be stretched across more than one process.

A "component" is a piece of a larger part of software, it can be used interchangeably with "module" and "process".  A "module" is a piece of software separated from the surrounding software by an interface.  A "process" is a kind of module that has execution and data isolation.  A "system" is a collection of processes which communicate with each other.

What's a concrete example of a "central design principle"?

> "In the geocoding component, all latitude and longitude data is represented as a 2-vector of [lat lon]"

"Every user-id in our backend system represents a valid user", as enforced by validation at the edge of the system.

Is a principle component internally structured as a hierarchy, or is that just a principled system?

> A principle component can be structured as a hierarchy if the component is large enough

Any tree with more than two layers has sub-trees, so yes.

Pg 113

"The semantics of our data are defined by the effects it produces when passed into our functions. ... The push phase begins wherever our functions perform effects. Put another way, it begins where the meaning of our data is defined."

In that quote, is the place "the meaning of our data defined" just the functions that will interpret the data?

Would you consider a pull from out of process "performing an effect"?

A process "interprets" its input data as a series of commands, describing effects, that it needs to follow.  Since pulls affect the world outside a process, they are definitely effects, but often less important than push effects when it comes to understanding what a system is trying to accomplish.

Pg 114

"A process is composed of operational phases at the edges and a functional phase at the middle"

^ These are my favorite programs to write, but too often I have to write a program that looks like:

- Pull data

- Transform data

- Pull data

- Transform data

- Pull data

- Transform data

- Push data

For example, making a series of web requests who's results depend on each other. Where are the edges and middle of the system here? I guess the only thing to do is keep the transforms free of side effects?

The operational edges of the process invoke the functional core of a process, not the other way around.  The "transform" phases of your process should not perform effects, but also shouldn't presume a fixed relationship to other parts of that process.  Adding or removing a request should only effect the edges, not the core.  

Pg 65

The book says that we can't create an "open" conditional by putting the predicates and clauses into a data structure because of the possible overlapping cases and non determinant order.

But what if we also encode the order in this data structure? Then the code that interprets the data structure could just use the encoded order. What am I missing?

If two different parts of the codebase say they want their predicate to go first, who wins?  Either there is a single arbiter of ordering (Clojure's `cond` macro), or the winner is unpredictable/arbitrary (colliding protocol or multimethod definitions for a single type/value), or collisions are structurally impossible (a class/defrecord implementing a Java interface).

Loved the model for modules. I work in python now so I struggle between OOM patterns and my favored clojure functional patterns. Thinking at the module level (a collection of functions and data, but most importantly names and interfaces) really clicks for me. It doesn't matter if the module exposes objects or dicts or functions, but getting clear on the invariants and assumptions is super helpful.

I'm glad to hear that, I'm hoping that my next book will be able to reach a much larger audience, so it's good to know you're finding them applicable outside of Clojure.

I really enjoyed the read, thanks for putting so much into it Zach!

Thank you, detailed questions like these are always welcome.

--
You received this message because you are subscribed to the Google Groups "Elements of Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elements-of-clojure+unsubscribe@googlegroups.com.
To post to this group, send email to elements-of-clojure@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elements-of-clojure/2fe98fae-5f40-4db8-9c93-bcbde6217b14%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[satshabad khalsa]

Thanks, that all makes sense! Looking forward to the next book!

To unsubscribe from this group and stop receiving emails from it, send an email to elements-of-clo...@googlegroups.com.
To post to this group, send email to elements-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elements-of-clojure/2fe98fae-5f40-4db8-9c93-bcbde6217b14%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Elements of Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elements-of-clo...@googlegroups.com.
To post to this group, send email to elements-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elements-of-clojure/jt4u63lh.51a1a823-1c15-4804-97d7-f37038c57476%40we.are.superhuman.com.