Trouble Verifying Sorting Property on A Simple Spec

[Jeremy Wright]

Hello,

I am trying to write a simple spec for sorting. However, I've expressed a temporal property that I feel shouldn't be valid, yet TLC permits it.

Here is my Spec: https://github.com/JeremyLWright/specs/blob/master/algorithm/sort/sort.tla

Here is a video walkthrough of my question: https://www.loom.com/share/25f4eaaecd914d158551cbc061104a06

This spec is intended to be a simple abstraction of what sorting is. A sequence is (potentially) unsorted, then in one magical step it's sorted. I'm applying a refinement to it in another spec to show that, for example, bubble sort implements "sort by magic".

However, I have two temporal properties, which I feel conflict.

IsSortedAsc(seq) == \A a,b \in 1..N: a < b => seq[a] <= seq[b]
IsSortedDes(seq) == \A a,b \in 1..N: a < b => seq[a] >= seq[b]
EventuallySortedDes ==  <>[]IsSortedDes(A)
EventuallySortedAsc ==   <>[]IsSortedAsc(A)

I'm trying to express that eventually, the sequence is sorted In Ascending order or Descending Order. When I apply a model that checks if both are simultaneously true, TLC says this is successful.

Model Screenshot:

Thank you for any potential help. I fully expect I'm doing something incorrect, but I'm not quite sure where.

Sincerely,

Jeremy Wright

[Hillel Wayne]

The problem is the line

Spec == Init /\ [][Next]_vars /\ EventuallySortedAsc /\ EventuallySortedDes

Since there are no models where EventuallySortedAsc /\ EventuallySortedDes, Spec = FALSE. If you change it to

Spec == Init /\ [][Next]_vars

Then it should work as expected.

H

--
You received this message because you are subscribed to the Google Groups "tlaplus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tlaplus+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tlaplus/CAC2JkJsjSGQ5ZpYiBKLGzw0p7QimMmVTmgvCMzK3JFHKJi1e0w%40mail.gmail.com.

[Jeremy Wright]

Thank you for the response Hillel, but I still think there is something I don't understand. I

t seems that if I setup a Spec with no behaviors, I can ask TLC anything and it will tell me it's okay. This bothers me.

The spec's I'm using here are "obvious", however if I design a more complex system, and I make a mistake such that my system system has no valid behaviors, TLC won't tell me there is a problem. Not only will TLC not compain, it tells me there are no problems for ANY property I ask it to check.

For example:

Code: https://github.com/JeremyLWright/specs/blob/master/algorithm/nonsense/nonsense.tla

Video Demo: https://www.loom.com/share/36be2872d50043ca9875cc305692450d

I'll restate the video, for those that prefer to read.

I define a spec with 1 and only 1 state.

------------------------------ MODULE nonsense ------------------------------

EXTENDS TLC, Integers

VARIABLES A

vars == A

Init == A = 1

Next == A' = 1

EventuallyRubbish == <>(A = 999)

Spec == Init /\ [][Next]_vars /\ <>[](A = 0)

=============================================================================

The Spec includes a temporal property which is not in the state space. There is no state where A=0, but TLC never warns me that my system does nothing. Here it is obviously not in the state space, but a more complex system that might not be so obvious. I recall the phrase, "beware of success", but this feels fragile to me. Am I missing something fundamental about how I express the Spec line?

Thank you for your time.

Sincerely,

Jeremy

To view this discussion on the web visit https://groups.google.com/d/msgid/tlaplus/11632bd3-f810-730b-d093-8e37b9f97c70%40gmail.com.

[Alex Weisberger]

Are you familiar with the concept of vacuous truth? It’s the idea that, if you can’t prove something false, then it’s true by default. This is the principle behind implication: P => Q is true (vacuously) even when P is false. 

Remember that “correctness” generally means that your Spec implies some property. You are constructing purposefully contradictory Specs which actually evaluate to false since they cannot be satisfied by any state space, and false implies the vacuous truth of any and all other logical statements. 

Maybe TLC could throw a warning here, I think that’s fair. But, it’s just a case of a formal system doing exactly what we tell to.

To view this discussion on the web visit https://groups.google.com/d/msgid/tlaplus/CAC2JkJv5u0%3D%2BS4%3Dq-f4aGbFKiXkkvnUgvwtwEnJec6_Q%2BL2TRQ%40mail.gmail.com.

[Leslie Lamport]

Jeremy,

You have discovered what makes the TLA logic better than any other

temporal logic for dealing with liveness.  The practical way to

specify safety properties is with a state machine.  In TLA, that's the

Init /\ [][Next]_vars part of the spec.  Hundreds (probably thousands)

of ways of specifying a state machines have been proposed.  They all

more or less work for specifying safety.  But safety just says what

may happen.  It's nice to be able to say that your system must do

something, which means that you have to add a liveness property.

Temporal logic is the best way I know to describe liveness properties.

However, the only way to add a liveness property to a state machine

that makes the state machine understandable is to add a fairness

property.  If S is a safety property, adding a fairness property means

writing S /\ L where L is a liveness property that doesn't add any

safety property.  What that means is the S /\ L does not rule out

anything that may happen that S allows to happen.  More precisely,

note that Init /\ [][Next]_var is false of a behavior if and only if

either Init is false on the initial state or [Next]_var is false on

some pair of successive states.  So, we can talk about it being

true on a finite behavior.  L is a fairness property for S if

any finite behavior satisfying S can be extended to an infinite behavior

satisfying S /\ L.  (Instead of saying that L is a liveness property

for S, we usually say that the pair S,L is machine closed.)

The problem with your spec is that the liveness property <>[](A=0) is

not a fairness property for your safety spec Init /\ [][Next]_vars.

It's an egregious form of non-fairness that rules out all finite

behaviors satisfying the safety property.  But you can get into

trouble by writing less extreme examples.  You almost never want to

add a liveness property that is not fairness.  (I've written one spec

during my career that did it.)  But I know of no way to tell if an

arbitrary liveness property is a fairness property for your spec.

And, I don't know any easy way to make sure you're writing a fairness

property in any temporal logic except TLA.

In TLA it's very simple.  You just write your liveness condition 

as the conjunction of formulas of the form WF_vars(Act) or

SF_vars(Act), where Next can be written as Act \/ SomethingElse.

If you don't follow that rule, you're asking for trouble.  

Leslie

P.S. The one spec I've written that used a non-fairness liveness

property was for the condition that database people call

serializability.  The fact that I had to do that to get a simple spec

was a sign that serializability, although it looks like a simple,

reasonable condition, is actually very weird when you examine it

closely.

L.

[Jeremy Wright]

Thank you Alex, and Leslie for your responses. This was very beneficial. 

Alex, I did not know the term "vacuous truth". Putting a term on this construct helped me understand logical-implication a bit more clearly.

A member of my study group pointed out that Pressler's post https://pron.github.io/posts/tlaplus_part3#machine-closure-and-fairness helped them understand Leslie's comments a bit better. Thank you all, for engaging with my question. It's wonderful to have a helpful community behind our tool.

Thank you again,

Jeremy

To view this discussion on the web visit https://groups.google.com/d/msgid/tlaplus/b883fa15-b77f-4b3c-a5cd-8323efe90f76n%40googlegroups.com.

[Jeremy Wright]

Hello!

I turned these questions into a 4 part training series: https://youtube.com/playlist?list=PLacslU3Fwm5sJx5fTcYgOt1oOj0JMKKc5

I'd appreciate any feedback this community could offer :-)

Jeremy