Deep comparison of dicts - cmp() versus ==?

[Victor Hooi]

Hi,

What is the currently most Pythonic way for doing deep comparisons between dicts?

For example, say you have the following two dictionaries

a = {
'bob': { 'full_name': 'bob jones', 'age': 4, 'hobbies': ['hockey', 'tennis'], 'parents': { 'mother': 'mary', 'father', 'mike'}},
'james': { 'full_name': 'james joyce', 'age': 6, 'hobbies': [],}
}

b = {
'bob': { 'full_name': 'bob jones', 'age': 4, 'hobbies': ['hockey', 'tennis']},
'james': { 'full_name': 'james joyce', 'age': 5, 'hobbies': []}
}

Previously, I though you could do a cmp():

cmp(a, b)

However, this page seems to imply that cmp() is deprecated?

https://docs.python.org/3/whatsnew/3.0.html#ordering-comparisons

Should we just be using the equality operator ("==") instead then? E.g.:

a == b

What is the reason for this?

Or is there a better way to do this?

Regards,
Victor

[Ben Finney]

Victor Hooi <victo...@gmail.com> writes:

> What is the currently most Pythonic way for doing deep comparisons
> between dicts?

What distinction do you intend by saying “deep comparison”? As
contrasted with what?

> For example, say you have the following two dictionaries
>
> a = {
> 'bob': { 'full_name': 'bob jones', 'age': 4, 'hobbies': ['hockey', 'tennis'], 'parents': { 'mother': 'mary', 'father', 'mike'}},
> 'james': { 'full_name': 'james joyce', 'age': 6, 'hobbies': [],}
> }
>
> b = {
> 'bob': { 'full_name': 'bob jones', 'age': 4, 'hobbies': ['hockey', 'tennis']},
> 'james': { 'full_name': 'james joyce', 'age': 5, 'hobbies': []}
> }

Those two dicts are not equal. How would your intended “deep comparison”
behave for those two values?

> However, this page seems to imply that cmp() is deprecated?
> https://docs.python.org/3/whatsnew/3.0.html#ordering-comparisons

It is, yes.

> Should we just be using the equality operator ("==") instead then? E.g.:
>
> a == b

Yes. That is a comparison that would return False for comparing the
above two values. Would you expect different behaviour?

> What is the reason for this?

I don't really understand. ‘cmp’ is deprecated, and you can compare two
dicts with the built-in operators. That's the reason; are you expecting
some other reason?

> Or is there a better way to do this?

I don't really know what it is you want to do. What behaviour different
from the built-in comparison operators do you want?

--
\ “I went over to the neighbor's and asked to borrow a cup of |
`\ salt. ‘What are you making?’ ‘A salt lick.’” —Steven Wright |
_o__) |
Ben Finney

[Victor Hooi]

Hi Ben,

When I said "deep", I meant, as in, to an arbitrary level of nesting (i.e. dicts, containing dicts, containing dicts etc) - sorry if I got the terminology wrong.

The two dicts weren't equal by intention - the idea was that a comparison operator would return "False" for those two.

I was just curious why cmp() was phased out (as in, were there cases where "==" was better) - but if functionality they're the same, and it's just a nomenclature thing, that's also fine =).

Finally, so cmp()/== return true/false for comparison - just noticed this which actually prints out diff-style comparisons:

https://pypi.python.org/pypi/datadiff

Cheers,
Victor

> `\ salt. 'What are you making?' 'A salt lick.'" --Steven Wright |
> _o__) |
> Ben Finney

[Rustom Mody]

On Friday, March 20, 2015 at 8:27:26 AM UTC+5:30, Victor Hooi wrote:
> Hi Ben,
>
> When I said "deep", I meant, as in, to an arbitrary level of nesting (i.e. dicts, containing dicts, containing dicts etc) - sorry if I got the terminology wrong.
>
> The two dicts weren't equal by intention - the idea was that a comparison operator would return "False" for those two.
>
> I was just curious why cmp() was phased out (as in, were there cases where "==" was better) - but if functionality they're the same, and it's just a nomenclature thing, that's also fine =).
>

> Finally, so cmp()/== return true/false for comparison...

I guess you are confusing two orthogonal questions:
1. deep vs shallow comparison
2. binary vs trivalent comparison

Numbers (not complex) satisfy the trichotomy law: ie for any 2 numbers x,y:
x < y or x > y o x = y

Unfortunately most logic in programming languages is binary -- true or false
(well Fortran had an arithmetic if -- very unfashionable even in Fortran --
but thats because of the need to goto)

Likewise C's strcmp is trivalent. But a trivalent value is clumsy in a binary logic. Unfortunately no switch statement for <, =, > -- so we are stuck with
if-else-if-else.

All this is unrelated to deep-vs=shallow: cmp is deep, == is deep
In fact what would/could a shallow comparison mean??
Shallow/deep only make sense for copy not comparison
[If at all: Languages like Haskell are explicitly constructed so that the
programmers is disallowed from making that distinction at all]

[Chris Angelico]

On Fri, Mar 20, 2015 at 2:27 PM, Rustom Mody <rusto...@gmail.com> wrote:
> Numbers (not complex) satisfy the trichotomy law: ie for any 2 numbers x,y:
> x < y or x > y o x = y

Real numbers, yes, and integers in most computer representations, but
not IEEE floating point. Be careful of that distinction; we're talking
about computers here, not mythical numbers.

ChrisA

[Rustom Mody]

On Friday, March 20, 2015 at 9:05:19 AM UTC+5:30, Chris Angelico wrote:

> On Fri, Mar 20, 2015 at 2:27 PM, Rustom Mody wrote:
> > Numbers (not complex) satisfy the trichotomy law: ie for any 2 numbers x,y:
> > x < y or x > y o x = y
>
> Real numbers, yes, and integers in most computer representations, but
> not IEEE floating point. Be careful of that distinction; we're talking
> about computers here, not mythical numbers.

A putative set that claims to be some-kind-of-numbers and has an element
that is Not-A-Number looks more mythical/mystical/Zen-ish than classical
math numbers.

IOW: float is a poor approximation to ℝ.
Most of us laymen use it because its an approximation that works (kinda).
Numerical analysts earn their keep because of the 'poor'
Not too many people are interested in float independent of ℝ except perhaps
hardware designers who need to design respecting the IEEE standard.

[Chris Angelico]

On Fri, Mar 20, 2015 at 2:47 PM, Rustom Mody <rusto...@gmail.com> wrote:
> IOW: float is a poor approximation to ℝ.

Of course it's a poor approximation. That's because real numbers are
uncountably infinite, so it's rather tricky to represent and
manipulate them efficiently. IEEE 754 gives us practicality rather
than purity.

ChrisA

[Ben Finney]

Chris Angelico <ros...@gmail.com> writes:

> Real numbers, yes […] but not IEEE floating point. Be careful of that

> distinction; we're talking about computers here, not mythical numbers.

So real numbers are mythical? IEEE floating point values are more real
than real numbers? Just what are you saying, man?

--
\ “Pinky, are you pondering what I'm pondering?” “Wuh, I think |
`\ so, Brain, but how will we get three pink flamingos into one |
_o__) pair of Capri pants?” —_Pinky and The Brain_ |
Ben Finney

[Chris Angelico]

On Fri, Mar 20, 2015 at 4:09 PM, Ben Finney <ben+p...@benfinney.id.au> wrote:
> Chris Angelico <ros...@gmail.com> writes:
>
>> Real numbers, yes […] but not IEEE floating point. Be careful of that
>> distinction; we're talking about computers here, not mythical numbers.
>
> So real numbers are mythical? IEEE floating point values are more real
> than real numbers? Just what are you saying, man?

Poor choice of words. Not "mythical", but "theoretical". Comparisons
in Python are between actual representable numbers, not some
theoretical entire range of reals. Python operators and operations are
not defined in terms of real numbers, but objects.

ChrisA

[Steven D'Aprano]

On Friday 20 March 2015 14:47, Rustom Mody wrote:

> On Friday, March 20, 2015 at 9:05:19 AM UTC+5:30, Chris Angelico wrote:
>> On Fri, Mar 20, 2015 at 2:27 PM, Rustom Mody wrote:
>> > Numbers (not complex) satisfy the trichotomy law: ie for any 2 numbers
>> > x,y: x < y or x > y o x = y
>>
>> Real numbers, yes, and integers in most computer representations, but
>> not IEEE floating point. Be careful of that distinction; we're talking
>> about computers here, not mythical numbers.
>
> A putative set that claims to be some-kind-of-numbers and has an element
> that is Not-A-Number looks more mythical/mystical/Zen-ish than classical
> math numbers.

Forget NANs. The trichotomy of floating point values was violated before the
IEEE-754 standard was invented. In fact, IEEE-754 was invented in order to
bring some semblance of order to the unspeakable mess of floating point
arithmetic prior to that. With IEEE-754 floats, at least you can guarantee
that the trichotomy applies to all values apart from NANs.

I cannot remember the details, and I don't have my copy of the Apple
Standard Numerics manual here to look it up, but in the 1970s there was
hardware where the following could fail with Divide By Zero error for
certain values of x:

if x > 0.0:
print 1/x # Divide By Zero here


If you find that hard to believe, it's because you're spoiled by the
astonishing success of IEEE-754 floating point arithmetic.

> IOW: float is a poor approximation to ℝ.

True, but not because of NANs.

Even *finite* floats violate some of the usual properties of Real numbers:

# Associativity
py> 0.1 + (0.2 + 0.3) == (0.1 + 0.2) + 0.3
False

# Distributivity
py> 1.3*(0.3+0.4) == 1.3*0.3 + 1.3*0.4
False

[Aside: why is my spell checker flagging plus signs as misspelled words?]

And then there's the whole thing where there is an uncountable infinity[1]
of Real numbers, and only a finite number of floats.

> Not too many people are interested in float independent of ℝ except
> perhaps hardware designers who need to design respecting the IEEE
> standard.

I don't understand what you are trying to say here.



[1] Uncountable infinity doesn't just mean that you cannot count them all
because there's an infinite number of them. It means that you cannot
enumerate them all, even in an infinite amount of time.

http://en.wikipedia.org/wiki/Cantor%27s_diagonal_argument


--
Steve

[Roy Smith]

In article <550bbfc1$0$13010$c3e8da3$5496...@news.astraweb.com>,

Steven D'Aprano <steve+comp....@pearwood.info> wrote:

> I cannot remember the details, and I don't have my copy of the Apple
> Standard Numerics manual here to look it up

Amongst the details you don't remember is the correct name :-) It was
Standard Apple Numerics Environment (SANE). I think I still have my
copy of the manual somewhere.

> If you find that hard to believe, it's because you're spoiled by the
> astonishing success of IEEE-754 floating point arithmetic.

There's two things we're spoiled by. IEEE-754 is, of course, one of
them. But, more than that, are good math libraries built on top of it.

One of the truly unappreciated things to come out of the early BSD Unix
releases was a C math library written by people who understood (and
worried about) things like accurate FP calculations, rounding, corner
cases, and all that gunk. Sure, I could sit down with a C compiler and
the Wikipedia Gamma function article and code up gamma(). It would work
well enough to get a decent grade as a homework assignment for
Introduction To Programming 101. But, fundamentally, it would be a
piece of crap because I know just enough about numerical programming to
know that I should leave it to the experts.

[Rustom Mody]

On Friday, March 20, 2015 at 10:39:48 AM UTC+5:30, Ben Finney wrote:

> Chris Angelico writes:
>
> > Real numbers, yes […] but not IEEE floating point. Be careful of that
> > distinction; we're talking about computers here, not mythical numbers.
>
> So real numbers are mythical? IEEE floating point values are more real
> than real numbers? Just what are you saying, man?

In all fairness to Chris, there have been notable mathematicians in the last 100
years who have said more or less exactly what Chris is saying: "The set ℝ is
nonsense." In fact these fights are the very reason that CS came
into existence.

1880s: Kronecker and his student Cantor were working together when Cantor
started off on 'set-theory'
For a while Cantor was bemused with the direction of his (favorite!) student.
But he soon found it unacceptable and started disagreeing with Cantor, first
privately then publicly.

His famous statement dates from this disagreement:
"The Good Lord made the Natural numbers. All the rest is the work of man"
He was specifically implying that things like ℝ are just nonsense.
At that time almost all mathematicians were Kroneckerians.
Cantor ended up in an 'institution.'
Other famous statements of that time were from Gordon who said:
"This sir, is not mathematics, its theology"
[referring to a standard construction of set theory called
Axiom of choice]

A couple of generations later things had turned.

Mainstream mathematicians all accepted set theory, except a few trouble-makers
called constructivists.
The constructivists said that the other mathematicians who they called
'Platonist' which derisively signified mystic after the cave allegory of Plato
http://dwildepress.net/critica/mystic4.html were being ridiculously loose and
sloppy in talking about infinity as though it exists.

Hilbert -- the head of the classicists/platonists -- was angry with this charge,
made the famous statement: "Nobody will drive us out of the paradise created by Cantor."

And (in what he thought was super-cleverness) formulated the Entscheidung problem:

If the questions about mathematics could be finitistically (ie mechanically)
solved, then the constructivists' charge (of mysticism) could be kicked out

But then Gödel showed that the dream of a complete and consistent math was a pipe-dream.
A job that Turing completed by showing the unsolvability of the Entscheidung problem.
http://www.philocomp.net/home/hilbert.htm

From a math/logic pov these were all disastrous results.
But a side-effect of this disaster was that something called a 'universal-
machine' got invented…
viz a computer

An entertaining and longer account:
http://research.microsoft.com/en-us/um/people/gurevich/opera/123.pdf
Particularly note the Polya-Weyl bet
==================================
tl;dr "Real numbers are not real" is an argument at the creation of CS.
To find and work with a set that
- Does not have the nonsensical behaviors of floats
- Is not nonsensical in the sense of the constructivists' objections

is really a bit of an open problem:
http://www.encyclopediaofmath.org/index.php/Constructive_real_number

[Rustom Mody]

On Friday, March 20, 2015 at 12:06:00 PM UTC+5:30, Steven D'Aprano wrote:
> On Friday 20 March 2015 14:47, Rustom Mody wrote:
> > Not too many people are interested in float independent of ℝ except
> > perhaps hardware designers who need to design respecting the IEEE
> > standard.
>
> I don't understand what you are trying to say here.

The reason for creating float (IEEE or previous hacks) is that people want to do
ℝ-ish stuff.
The only justification for float is that it is consistent with ℝ
[Is it? Aint it? Depends on how you squint]¹
Nobody wants to do float for the sake of float.
The only parties that (presumably) want to do "pure" float (ie ignore ℝ completely)
are the h/w-guys who need to get an "IEEE-compliant" stamp.

=============
¹ Even the IEEE standard writers need to do a tight-rope walk between
- respecting ℝ
- making a spec that is 'hardware-able"
- And will not have us software-guys screaming

Ie There is no one (I can think of) except hw engineers who would care to think
float without minding ℝ.

[Marko Rauhamaa]

Rustom Mody <rusto...@gmail.com>:

> In all fairness to Chris, there have been notable mathematicians in
> the last 100 years who have said more or less exactly what Chris is
> saying: "The set ℝ is nonsense."

Intuitionism is nonsense.

> In fact these fights are the very reason that CS came into existence.

I trace it to Hilbert's program.


Marko

[Rustom Mody]

On Friday, March 20, 2015 at 7:40:31 PM UTC+5:30, Marko Rauhamaa wrote:
> Rustom Mody:

>
> > In all fairness to Chris, there have been notable mathematicians in
> > the last 100 years who have said more or less exactly what Chris is
> > saying: "The set ℝ is nonsense."
>
> Intuitionism is nonsense.

You are challenging me?!?! Come on lets have it out!!

On second thoughts Ive misplaced my boxing gloves.
[And my legs are more reliable than my arms..........
............
]

>
> > In fact these fights are the very reason that CS came into existence.
>
> I trace it to Hilbert's program.

And on third thoughts:
What was Hilbert's program in defence against?

Seriously:

I dont take a side on these matters [Or which side do you see me on?]

However I am talking some historical facts, viz:
Because some nuts did the 20th century equivalent of:
"Break each others' heads about how many angels can dance on the head of a pin"
therefore much of the world around us exists as it does (for better or worse)
including my (and I guess your) degree and job.

[Marko Rauhamaa]

Rustom Mody <rusto...@gmail.com>:

> However I am talking some historical facts, viz: Because some nuts did
> the 20th century equivalent of: "Break each others' heads about how
> many angels can dance on the head of a pin" therefore much of the
> world around us exists as it does (for better or worse) including my
> (and I guess your) degree and job.

The deep philosophical questions around the year 1900 in both
mathematics and physics were left embarrassingly open. The way out was
to stop caring.

Mathematics gave up on defining numbers and settled with a standard
chain of beads:

0 = {}
1 = { 0 }
2 = { 0, 1 }
3 = { 0, 1, 2 }

<URL: http://en.wikipedia.org/wiki/Set-theoretic_definition_o
f_natural_numbers#Contemporary_standard>


Marko

[Rustom Mody]

On Saturday, March 21, 2015 at 11:58:43 AM UTC+5:30, Marko Rauhamaa wrote:
> Rustom Mody :
>

> > However I am talking some historical facts, viz: Because some nuts did
> > the 20th century equivalent of: "Break each others' heads about how
> > many angels can dance on the head of a pin" therefore much of the
> > world around us exists as it does (for better or worse) including my
> > (and I guess your) degree and job.
>
> The deep philosophical questions around the year 1900 in both
> mathematics and physics were left embarrassingly open. The way out was
> to stop caring.

Ive collected some of the above points and some from earlier threads here:
http://blog.languager.org/2015/03/cs-history-0.html

Anyone un-mentioned there that wants to figure -- or conversely :-) --
just give me the say-so.