JSH: The will to lie
jst...@msn.com
basic things.
Some of you may think you understand probability and statistics, but
consider a question like, if someone flips a coin 10 times and gets 10
heads, and THEN flips the coin 10 times and get 10 tails, is the coin
flawed?
If you think the answer is yes, you're wrong.
The correct answer is, not enough information given.
If you think there is no way the coin can do that given the odds, then
you are wrong.
But no trained scientist (properly trained) would get the question
wrong.
So then, can a chi square test or any other statistical test prove that
p mod 3 with p a prime greater than 3 does NOT gives a random sequence?
Nope.
It can give you a degree of confidence and be very likely to be right,
but then again, any such test would take 10 heads in a row as proof of
non-randomness.
So what is the resolution? Is there none?
YES!!!
Mathematicians may get used to non-practical results to the extent that
they forget that in the real world, rules mean PREDICTION!!!
Our understanding of the laws of physics allows us to do things, well,
like type up missives on wonderful devices that many people don't fully
understand, or fly around the world, or chat up all those minutes on
cellphones BECAUSE WE KNOW THE RULES WELL ENOUGH.
So, the answer is not a fight over statistical tests, but prediction.
If mathematicians wish to maintain that p mod 3 with a prime greater
than 3 is not random, then give me the freaking rules.
Give me the laws of the behavior.
Consider in contrast that with the prime distribution itself--the count
of primes--it is well-known that the probability a number x is prime is
roughly 1/(ln x).
So you have an equation. You can say, for instance, that the
probability of primeness up to 100 is about 1/(ln 100) which is 0.217
to three digits, saying roughly 22 primes and there are 25, so you have
prediction and a test showing the validity of that prediction.
So then, if p mod 3 is not random, then there would be rules.
Consider an unknown hypothesized function PMOD3RULE(x), which like
1/(ln x), would tell you that the probability at x=1098340 that p mod 3
in that area is 1 is 70%.
Then you could test the rule, and see if you got more 1's than 2's.
People, our technological world rests on the truth within the debate
that non-random things are predictable, to some extent, or we would not
have clothes, would not have computers, and would not have nice big
houses to sit in and type up dumb messages to each other.
So the debate is at the foundations of our technology, at the
foundations of our science, at the foundations of our intellectual
pursuits about how we know what we know in this world.
Non-random means prediction. So posters arguing with me can't just
trot out a test.
Unfortunately many modern mathematicians behave like they are not part
of the real world, so the possibility of prediction as my counter to
claims against what I know is random behavior might have escaped them.
They are used to playing word games and not being challenged.
They are used to bulling their way through with enough math-ese to win
debates in a political arena.
But I SAY SHOW ME.
I don't care if these nincompoops think they win because they convince
a lot of you who don't know any better. Sure they may think that's all
that matters, but I know it's not.
Most of you may never get it, like you may never quite comprehend how
in the hell that big box with the green light works, or care as long as
you can download your music files.
Most of you will probably never understand much of anything in this
world, so yes, mathematicians can convince you until the day you die,
and you will die in ignorance.
But damn them, I am going to ask them once again to SHOW ME!!!
So it is a reality test people. Or better yet, a reality check and
mate.
James Harris
jshs...@yahoo.com
You don't understand something, so you win?????
Yet again, you prove the title you have been given. You earn it with
each post.
Eric Gisse
jst...@msn.com wrote:
> I find it fascinating watching math people lie on newsgroups about
> basic things.
[...]
Like how your 'research' is important to sci.physics and sci.skeptic?
Knows Everything
> Some of you may think you understand probability and statistics, but
> consider a question like, if someone flips a coin 10 times and gets 10
> heads, and THEN flips the coin 10 times and get 10 tails, is the coin
> flawed?
>
> If you think the answer is yes, you're wrong.
But nobody is thinking "yes" you silly twit -
> Mathematicians may get used to non-practical results to the extent that
> they forget that in the real world, rules mean PREDICTION!!!
>
> Our understanding of the laws of physics allows us to do things, well,
> like type up missives on wonderful devices that many people don't fully
> understand, or fly around the world, or chat up all those minutes on
> cellphones BECAUSE WE KNOW THE RULES WELL ENOUGH.
>
> So, the answer is not a fight over statistical tests, but prediction.
>
> If mathematicians wish to maintain that p mod 3 with a prime greater
> than 3 is not random, then give me the freaking rules.
How ever do you type while wearing a straitjacket ?
> Give me the laws of the behavior.
>
> Non-random means prediction. So posters arguing with me can't just
> trot out a test.
Sillytwit. Non-random means deterministic. Determinism IMPLIES
predictability.
Ass. The very assumption that you can even test for a property which is
presumed nondeterministic in the first place is inherently contradictory.
Why you fail to argue on those grounds really baffles me. Except to conclude
that you prefer to argue pointlessly and groundlessly, even when better
arguments exist - if only perhaps you understood them ?
Absolute disorder is trivial you mannequin -
http://sciphysicsopenmanuscript.blogspot.com/
http://sciphysicsopenmanuscript.blogspot.com/
http://sciphysicsopenmanuscript.blogspot.com/
http://sciphysicsopenmanuscript.blogspot.com/
http://sciphysicsopenmanuscript.blogspot.com/
Justin
: If mathematicians wish to maintain that p mod 3 with a prime greater
: than 3 is not random, then give me the freaking rules.
You dumb idiot - you just gave the rule. The rule IS p mod 3.
: But I SAY SHOW ME.
Show me yours first.
Justin
William Hughes
jst...@msn.com wrote:
<a long diatribe complaining among other things that the only
disproof of randomness was a chi_squared test and that
no one had shown any method of prediction>
I am baffled. In many,many replies it was pointed out
that a trivial method (predict the opposite of the last residue)
would provide prediction at better than 50%. In light of this
how can you claim that no-one considered the problem of
prediction?
-William Hughes
Frank J. Lhota
Why don't you get it over with and just give us "the hammer" :)
Or maybe I can do it this time:
http://imagescommerce.bcentral.com/merchantfiles/4844397/pluspucca006.jpg
Tim Peters
> I find it fascinating watching math people lie on newsgroups about
> basic things.
Fascinating indeed ;-)
> Some of you may think you understand probability and statistics, but
> consider a question like, if someone flips a coin 10 times and gets 10
> heads, and THEN flips the coin 10 times and get 10 tails, is the coin
> flawed?
>
> If you think the answer is yes, you're wrong.
Wow -- an /extended/ form of the counterfactual hypothetical (in this case,
nobody thought the answer was "yes").
> The correct answer is, not enough information given.
Which is why a chi-square test requires that the expected count for each bin
be "reasonably large" (I like at least 10, but that's conservative). If you
set up a 1024-bin test to check sequences of 10 coin tosses, the expected
count in each bin after one trial would be 1/1024, and that's way too small.
This is a way of quantifying "not enough information". There are sane ways
to proceed here. You don't /need/ to guess at every step -- take the
opportunity to learn something.
> If you think there is no way the coin can do that given the odds, then
> you are wrong.
>
> But no trained scientist (properly trained) would get the question
> wrong.
>
> So then, can a chi square test or any other statistical test prove that
> p mod 3 with p a prime greater than 3 does NOT gives a random sequence?
>
> Nope.
>
> It can give you a degree of confidence and be very likely to be right,
> but then again, any such test would take 10 heads in a row as proof of
> non-randomness.
Do your homework just once? Try it! It's actually nice to know something
correct.
The probability that 10 heads come up in a row for a fair coin is exactly
the same as for any other specific 10-toss outcome, and a properly performed
chi-square test will detect non-randomness if 10 heads in a row occurs too
often, /or/ if 10 heads in a row doesn't occur often /enough/. If the coin
is fair, 10 heads in a row should occur about 1 in each 2^10 = 1024 tries,
and a chi-square test will notice if the observed frequency is significantly
higher /or lower/ than 1/1024.
Did you notice the squares in the chi-square formula? It's a sum of terms
of the form:
(number_observed - number_expected)^2 / number_expected
Because of the square, what it cares about is the /absolute value/ of the
difference between the numbers observed and expected, not at all about
/which/ is the larger.
So (and as everyone else already knew), it's dead wrong that "any such test
would take 10 heads in a row as proof of non-randomness". Any such test
would take /deviation/, high or low, from the expected 1/1024 frequency as
raw evidence.
Even worse for your ignorance of all this, deviations aren't even
necessarily evidence of "non-randomness": it's also unlikely that a truly
random sequence will /exactly/ match, or even very closely match, the
expected frequency in each bin. The chi-square test knows all about that
too. Deviations are expected, and if there aren't enough deviations, the
computed chi-square statistic will have a low probability of being so
/small/.
That doesn't happen for your sequence -- your sequence's chi-square
statistic is jaw-droppingly huge. It would happen if, e.g., your sequence
looked like this instead:
1, 2, 2, 1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 1, 2, 2,
1, 2, 2, 1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 1, 2, 2,
1, 2, 2, 1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 1, 2, 2,
1, 2, 2, 1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 1, 2, 2,
1, 2, 2, 1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 1, 2, 2,
1, 2, 2, 1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 1, 2, 2,
1, 2, 2, 1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 1, 2, 2,
1, 2, 2, 1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 1, 2, 2,
1, 2, 2, 1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 1, 2, 2,
1, 2, 2, 1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 1, 2, 1,
...
Exercise: Compute the 4-bin chi-square statistic for the 80 non-overlapping
adjacent pairs of the 160 listed elements. Work out that for a uniformly
random sequence of 160 1s and 2s, the probability of seeing a distribution
so closely matching the expected counts in the 4 bins is about 0.008.
> So what is the resolution?
There's no problem here to resolve -- except for, perhaps, your inability to
learn anything about the topic you're going on about. Try learning
something here. You wouldn't regret it.
> Is there none?
>
> YES!!!
Huh? "Yes, there is none"?
> Mathematicians may get used to non-practical results to the extent that
> they forget that in the real world, rules mean PREDICTION!!!
LOL. Jeez Louise, bubba, this has already been covered so often it's not
funny. Yes, prediction. Your guess lost. Move on.
> ... [tired arguments already addressed] ...
donsto...@hotmail.com
Are you any relation to the James Harris of Red Rock, Texas, USA?
David C. Ullrich
>I find it fascinating watching math people lie on newsgroups about
>basic things.
And we find it fascinating to watch you complain about people
lying when what they've given incredibly simple (and correct)
demonstrations that they're telling the truth.
So everyone's fascinated. Great.
>[...]
>
>But damn them, I am going to ask them once again to SHOW ME!!!
Why? They've already SHOWN YOU many times.
>So it is a reality test people. Or better yet, a reality check and
>mate.
>
>
>James Harris
************************
David C. Ullrich
David Moran
You're never going to get anywhere in a field you are clearly uneducated
about. One thing we have on you is formal education in math. However, I
doubt any of us would be qualified to argue with you about physics. That
is if you really are a physicist.
Dave
Mark VandeWettering
It's hard to believe that someone as obviously uneducable as James appears
to be in terms of mathematics would be any better educated on a subject like
physics which relies on a decent grasp of mathematics.
> Dave
Gib Bogle
> You're never going to get anywhere in a field you are clearly uneducated
> about. One thing we have on you is formal education in math. However, I
> doubt any of us would be qualified to argue with you about physics. That
> is if you really are a physicist.
It is significant that, although supposedly educated in physics, he
chooses to post on the subject of mathematics. Perhaps he learned
enough physics to appreciate his limitations there, but this never
happened with mathematics.
jst...@msn.com
Prediction in science is more than just about some rough rule but also
about REASONS behind the rules.
As an example, in the past, people thought all kinds of common sense
and reasonable things like that feathers fall slower than a brick
because one is heavier than the other.
But the reality has to do with wind resistance, so remove the wind, and
you can use a simple equation that tells you how fast EACH will fall to
earth.
So the problem here isn't to look at some particular lists of something
that may be random and think you see a pattern and declare a rule, but
to give the laws of the supposed rule as then they can be tested.
The requirement then is a formula for p mod 3, where you plug something
in, and get a probability that you will have 1 at a higher than 50%
chance or 2.
What you suggest is not at all the way science is done, but it is the
way many people commonly think, like they see something happening a lot
and conclude that it will always tend to happen, which is a common
problem in the way people think.
Science is not just about finding patterns, but finding rules that
govern those patterns, and then using those rules--in this case
formulas--to make those predictions.
The example I gave for an area where rules DO work with primes is with
the prime distribution where 1/(ln x) is approximately the probability
of primesness for a natural number x, so you can use it to roughly
count primes.
So I have explained in detail and given you a concrete example.
I emphasize that and may sound a bit pedantic, but people need to
understand that the problem here is a dedicated resistance to the truth
from intelligent people willing to work to convince you of things that
are false, so I need to step out basic things--over and over and over
and over again--and still it usually does not work!!!
Human beings are troublingly weak against other human beings who are
intelligent and actively working to lie, who are part of a group that
is trusted.
The point here is to drop the automatic trust of mathematicians.
Modern mathematicians quite simply, cannot all be trusted.
You have to at least consider that these people are willfully lying to
you.
And I have to state basics over and over and over again and watch it
not matter, unfortunately because too many of you still are naive about
our modern world.
So you are taken in, by people who wouldn't surprise me if they would
skin you alive if it would suit their own needs.
James Harris
Jesse F. Hughes
> The requirement then is a formula for p mod 3, where you plug
> something in, and get a probability that you will have 1 at a higher
> than 50% chance or 2.
But the rule is clear. I've already offered a little wager. If you
think the rule doesn't work, then you have no reason not to accept the
wager.
Once again, here's the proposal: We let a pseudorandom number
generator pick some n between 1 and, say, 512 (or higher if you prefer
and it's feasible). I'll make fifty wagers: I'll bet on the value of
your sequence in positions n+1 to n+50. Each wager is as follows:
If position i has a 1, I wager a buck that position i+1 has a 2. If
so, you owe me a dollar and otherwise I ow you a dollar.
If position i has a 2, then I wager that position i+1 has a 1.
If you are correct and the sequence really is similar to coin flipping
(so that the value in i+1 is independent of the value in i), then you
should be willing to enter this wager. Your expected payoff is $0.
But if Tim is correct, then I expect to make more than I lose.
Of course, I'd be happy to do this with plain ol' mock wagers if you
prefer. The money isn't really all that relevant. And if 50 wagers
isn't enough, then name the total.
--
"I suggest to those who listen that they enjoy the world, whatever
their piece of it may be, as much as they can over the next few days,
as soon enough, it will pass away, thanks to people who call
themselves 'mathematicians'." -- JSH envisions geek Ragnarok
david petry
jst...@msn.com wrote:
> I find it fascinating watching math people lie on newsgroups about
> basic things.
I've never quite understood what the point of your participation in
this newsgroup is, but I think you might find the following article
interesting:
http://groups.google.com/group/sci.math/msg/40cc4610018d67de?hl=en&
marcus_b
> William Hughes wrote:
> > jst...@msn.com wrote:
> >
> > <a long diatribe complaining among other things that the only
> > disproof of randomness was a chi_squared test and that
> > no one had shown any method of prediction>
> >
> > I am baffled. In many,many replies it was pointed out
> > that a trivial method (predict the opposite of the last residue)
> > would provide prediction at better than 50%. In light of this
> > how can you claim that no-one considered the problem of
> > prediction?
> >
> > -William Hughes
>
> Prediction in science is more than just about some rough rule but also
> about REASONS behind the rules.
>
What you are asking for is a PROOF that in the sequence of primes
mod 3, pairs of the form '12' and '21' are more common that '11' and
'22'.
No one here has given such a proof. It could be that no such proof
is known.
What people have done is produce some very strong statistical
evidence that '12' is more common than '11', etc.. Statistics however
is not proof. The chi-square test does not give proof. It just gives
evidence.
So you're right. There's no proof.
On the other hand, you have no proof of the opposite. Do you?
It's known - a difficult theorem, actually - that p mod 3 is just
as likely to be '1' as it is to be '2'.
But that does NOT prove that the sequence of primes mod 3 is random.
To do that, you would have to look at the probabilities of
sequences, like '11', '12', '111', '121', etc.
Because there are no theorems about the probability of sequences,
people here collected actual data.
The evidence is overwhelming. If you assign a probability to
it, it's on the order of 0.000001 - much less, actually. It's a lot
less than the probability that a fair coin will come up heads
10 times in a row. It's more like coming up heads 1000 times
in a row.
This still isn't proof. There is still a tiny sliver of a chance
that you are right.
So the situation is: you don't have a proof for your side.
Your critics have no proof for their side. This is very likely an
unsolved problem. But the numerical evidence is overwhelming.
It's comparable to the evidence that lawyers use, based on
DNA, to get someone acquitted or convicted. It would very
definitely meet any LEGAL requirement for proof.
You can deny it if you want, but given that YOU don't have
even a hint of a proof for your side of the argument, you look
pretty silly. And calling people liars is mean and dirty and
makes you look small. No one is lying. They are just honestly
presenting statistical evidence. Sure, people can use statistical
evidence to lie, but that doesn't mean statistical evidence is
ALWAYS a lie. That would be one of your logical fallacies.
Consider the statement: women tend to be shorter than men.
That's based entirely on statistical evidence. Is it a lie?
We all accept statistical evidence every day with nary a blink.
It is of great value. You want to say it's a lie - ALWAYS a lie -
because that's the only defense you can have.
Bottom line: your critics don't have a proof. It's probably an
unsolved problem. Your critics have very strong numerical
evidence. You don't have a proof either. And you have NO
numerical evidence, except for singletons. The proof for
singletons (which you have not given, incidentally) does not
carry over for pairs.
> As an example, in the past, people thought all kinds of common sense
> and reasonable things like that feathers fall slower than a brick
> because one is heavier than the other.
>
> But the reality has to do with wind resistance, so remove the wind, and
> you can use a simple equation that tells you how fast EACH will fall to
> earth.
>
> So the problem here isn't to look at some particular lists of something
> that may be random and think you see a pattern and declare a rule, but
> to give the laws of the supposed rule as then they can be tested.
>
> The requirement then is a formula for p mod 3, where you plug something
> in, and get a probability that you will have 1 at a higher than 50%
> chance or 2.
>
Plug in the value of p. You will know with complete certainty
what p mod 3 is. With 100% accuracy. The sequence of
primes is not random at all. It is just as predictable as the
sequence 1, 4, 9, 16, .... We are actually not talking about
true randomness here anyway, but pseudo-randomness, the
kind that a computer-based pseudo-random number generator
produces.
> What you suggest is not at all the way science is done, but it is the
> way many people commonly think, like they see something happening a lot
> and conclude that it will always tend to happen, which is a common
> problem in the way people think.
>
> Science is not just about finding patterns, but finding rules that
> govern those patterns, and then using those rules--in this case
> formulas--to make those predictions.
>
> The example I gave for an area where rules DO work with primes is with
> the prime distribution where 1/(ln x) is approximately the probability
> of primesness for a natural number x, so you can use it to roughly
> count primes.
>
Yes. For this question about singletons, there is a proof. It was
discovered by a far, far better mathematician than you. But it is not
sufficient to prove "randomness" of the sequence of primes.
> So I have explained in detail and given you a concrete example.
>
No - you have only quoted a known theorem - you haven't
explained it at all. And that theorem is only about singletons. It
does not predict what you get for pairs, triplets, etc. There are
no theorems for those. There is only statistical evidence. But it's
very strong evidence. You are a fool to deny it.
> I emphasize that and may sound a bit pedantic, but people need to
> understand that the problem here is a dedicated resistance to the truth
> from intelligent people willing to work to convince you of things that
> are false, so I need to step out basic things--over and over and over
> and over again--and still it usually does not work!!!
>
That's because, in this case, you're ignoring the data.
> Human beings are troublingly weak against other human beings who are
> intelligent and actively working to lie, who are part of a group that
> is trusted.
>
NO ONE IS LYING. If you think the data that have been presented
here are wrong, present your own data for pairs, triplets, etc.. Put
your money where your mouth is.
> The point here is to drop the automatic trust of mathematicians.
>
> Modern mathematicians quite simply, cannot all be trusted.
>
In contrast to yourself ?? Do you think anyone who has read
your contributions to sci.math would trust you on any topic what-
soever???
> You have to at least consider that these people are willfully lying to
> you.
>
I don't think they are. I can check their numerical evidence. It
seems to be right. You, on the other hand, have no proof and
no evidence regarding pairs, triplets, etc. Why should anyone
believe you? It seems far more likely that YOU are wrong - I'm
not saying you're lying, just wrong - as you have been so very
often in the past.
> And I have to state basics over and over and over again and watch it
> not matter, unfortunately because too many of you still are naive about
> our modern world.
>
No one disagrees that p mod 3 is just as likely to be 1 as 2.
That's what you mean by 'basics'. It's a known proven fact. But
it does NOT prove that '11' is just as likely as '12'. This is what
people are trying to tell you. You just keep saying the same
simplistic thing over and over again. When people point out that
it isn't sufficient, you say they are lying. Then you get on the
soapbox and play to the grandstand with the simplistic message.
You look like a braying idiot.
Here's an analogy. Consider the sequence 1, 2, 3, 4, 5, ... Just
the sequence of integers. Pick one at random. There is a 50-50
chance it is even. Does that prove the sequence is random?
Does that prove that you are just as likely to get two consecutive
numbers both even as you are to get even-odd?
> So you are taken in, by people who wouldn't surprise me if they would
> skin you alive if it would suit their own needs.
>
And your evidence for that is ???
Marcus.
>
> James Harris
Knows Everything
> > I find it fascinating watching math people lie on newsgroups about
> > basic things.
>
> I've never quite understood what the point of your participation in
His participation can be compared to a persistent fly which lands on your
potato salad and then flies away before you can kill it. Flies shit wherever
they land, including on your food, and they hatch from rotting fleshy shit.
http://sciphysicsopenmanuscript.blogspot.com/
Tim Peters
> What you are asking for is a PROOF that in the sequence of primes
> mod 3, pairs of the form '12' and '21' are more common that '11' and
> '22'.
>
> No one here has given such a proof. It could be that no such proof
> is known.
As usual, due to his incessant creation of new & inappropriately
cross-posted threads, James has done all he can to ensure the story is
nearly impossible to follow.
Anyway, the distribution of pairs in this case is entirely determined by the
distribution of prime gaps. Nothing relevant about the latter has been
proved (not even for gaps of size 2), so it's certain that nobody has a
proof or disproof.
Maybe surprisingly, taking Hardy & Littlewood's /conjectures/ about
asymptotic prime-gap densities as truths, I believe asymptotic
equidistribution of the possible pairs here probably /does/ follow. This is
hard to see because the conjectured asymptotic formulas for gaps > 4 give
very poor approximations when applied to small integers, and 10^9 definitely
counts as "small" for this purpose -- prime gaps greater than 4 don't work
at all "in the small" in the quantitative ways they're conjectured to work
in the limit.
But, in some other pointlessly new JSH thread, I recently demonstrated that
the distribution was in fact very much smoother in a run using (probable)
primes > 2^300 ~= 10^90. That's probably a lot larger than needed to make
it look "obviously pretty even" instead of "obviously nowhere close to
even", but I'd be surprised if gross imbalance didn't persist at least
through 20-digit primes.
> What people have done is produce some very strong statistical
> evidence that '12' is more common than '11', etc..
Across the ranges tested, yes, and extremely strong.
> Statistics however is not proof. The chi-square test does not give
> proof. It just gives evidence.
Right X 3.
> So you're right. There's no proof.
Yes and no. The obvious proof that his sequence isn't random is trivial:
the i'th term of his sequence is mod(p_i, 3). It's completely determined.
But there isn't (and probably won't be) a proof either way that adjacent
terms "act randomly" in the limit.
> On the other hand, you have no proof of the opposite. Do you?
He believe he does. It's a "JSH proof", though, so you don't want to ask
;-)
> It's known - a difficult theorem, actually - that p mod 3 is just
> as likely to be '1' as it is to be '2'.
>
> But that does NOT prove that the sequence of primes mod 3 is random.
I think that's the 48th time someone tried to tell him that :-(.
> To do that, you would have to look at the probabilities of
> sequences, like '11', '12', '111', '121', etc.
And a lot more, if we're willing to overlook the obvious truth that it's not
random and settle for a weaker claim, such as that the sequence can't be
distinguished from a random sequence via some fixed battery of standard
tests.
> Because there are no theorems about the probability of sequences,
> people here collected actual data.
>
> The evidence is overwhelming. If you assign a probability to
> it, it's on the order of 0.000001 - much less, actually.
In another of these threads, I showed that the chi-square statistic for
non-overlapping adjacent pairs across the primes < 10^9 had a probability of
about 10^-66510 of being so large. So add about 66500 more zeroes before
the "1" in your "0.000001" :-)
> It's a lot less than the probability that a fair coin will come up heads
> 10 times in a row. It's more like coming up heads 1000 times
> in a row.
10^-65510 ~= 2^-220941, so it's like coming up heads more than 200000 times
in a row. Although that now /appears/ likely to me to be a measure of just
how poorly the asymptotics match reality in this range, it's a fair measure
of how badly someone could get burned if they tried to /use/ this sequence
in lieu of a random one in a context where randomness mattered.
> This still isn't proof. There is still a tiny sliver of a chance
> that you are right.
Well, the chance is exactly 0 that the sequence through 10^9 could sanely be
used as, e.g., a random bit source. The sequence is entirely fixed, for all
time, and the randomness tests it failed through that range will always fail
exactly as badly.
What's left to salvage here? /Maybe/ the mod(p, 3) sequence could be used
as a reasonably pseudo-random source if we stuck to primes with a very large
number of digits, but it would take a world of effort to prove that, and
there are already many ways to get such sequences more easily and
efficiently.
There's a bit of potentially interesting math concerning the asymptotic
distribution of k-tuples of adjacent primes modulo a fixed prime, and I
don't know whether that's been studied /as such/ before. It reduces to
reasoning about prime-gap densities (e.g., adjacent terms in James's
sequence have the same value iff the gap between the associated primes is
divisible by 6). I've enjoyed playing with it, anyway :-)
> ...
noshellswill
DM:
I believe it was Brower(?) who insisted all set-definitions must be
"constructive". Granted, not a popular position. JSTEVHs' point seems
close enough (to this) to require a response more serious than
name_calling. Surely there are heuristics for dealing with set-properties
that are evident, but not manifest.
This lurker would appreciate such an ( FAPP ) argument, to which a
"reasonable man" might assent.
nss
***********
Frank J. Lhota
news:pan.2006.09.03....@hotmail.com...
L.E.J. Brouwer was a Dutch mathematician and philospher whom many regard as
the father of intuitionism. See
http://en.wikipedia.org/wiki/Luitzen_Egbertus_Jan_Brouwer
I'm pretty sure he's the constructivist that you were thinking of. I'm not
sure what the connection is between Brouwer's work and Harris's latest
posts.
I will grant you that the ideas of Brouwer deserve a serious discussion.
Certainly such discussions have occurred at the sci.math newsgroup. The
issue of what constitutes a random sequence is also worthwhile, and these
newsgroups have had some interesting posts on this topic.
So why did this thread descend into name calling? A large part of the
problem is James Harris's rather boorish behavior. Tim Peters initially
wrote respectful and informative replies to the initial JHS posts on the
randomness of primes. To this, James Harris responded with his usual screed
against the math community, calling them liars and cheaters who are
unconcerned with the truth. In particular, Harris falsely accused Peters of
cheating. After being the butt of James Harris's vast stream of vitriol,
some posters have responded in kind. That is unfortunate, but James is the
*last* person in the world to have any right to complain about the tone of
this conversation.
Anyone who has any doubts about Mr. Harris's culpability in this matters
should:
1) Look at the subject line for this thread; and
2) Consider who composed this subject line.
I rest my case.
noshellswill
FH:
I'll certainly defer to those who respond harshly should Mr Harris
have crudely attacked them. 'What's good for the goose ..."
Otherwise, applying Brouwers ( thanks for the spelling correction )
constraint to the assertion < This sequence is a member of the set
of non-random sequences > requires the statement of a rule for the
construction of "this" sequence. I'm looking for a recipe!
But now I'm really bugged, because in the sizzling-olive-oil I smell
evil remnants of " ALL CRETINS ARE LIARS " ... and like any
self-respecting physics guy sit back, pour another vino and wait for
mathematicians to find a pork-chop ... er ... %^[ ... or suchlike.
nss
*************
jst...@msn.com
Hmmm...let's consider that position.
The sequence of natural numbers is perfectly ordered such that you have
1, 2, 3 followed by 4, 5, 6 followed by 7, 8, 9
and on out to infinity where each residue modulo 3 is perfectly
balanced when you count out by 3's in that way, so that you have 1/3
with 0, 1/3 with 1, and 1/3 with 2 as a residue modulo 3.
But the naturals are just primes and the product of primes along with
1, so consider the possibility that residues modulo 3 show a preference
for 1 then their products would also show that preference, so the
naturals would show it as well in contradiction to the ordering noted
above, therefore there is NO PREFERENCE POSSIBLE.
But if there is no preference then no rules for a preference can exist,
meaning that the only rule is an equal probability for each to occur.
> It's known - a difficult theorem, actually - that p mod 3 is just
> as likely to be '1' as it is to be '2'.
>
Yet I just stepped through a simple logical argument relying on the
triviality that the naturals follow a highly rigid order, which to me
gets back to an easy way to fool MOST people in an area widely believed
to be difficult, which is to claim that it's difficult!!!
Naturals are either primes, products of primes or 1.
Prime preferences push outward to their products, so that a prime
preference becomes a natural number preference.
But natural numbers are simple to most people, as you have
1, 2, 3, 4, 5 and so on out to infinity
so I connect a supposedly difficult area to trivialities.
> But that does NOT prove that the sequence of primes mod 3 is random.
>
The logical proof is that if there are no rules for the primes beyond
having a roughly equal number that have 1 as a residue modulo 3 and 2
as a residue modulo 3, then they behave as if it does not matter, and
switch randomly between the two.
So why do I say "roughly"?
Because there is no rule possible to generate a sequence like 1, 2, 1,
2, 1, 2, 1, 2 or any other such perfectly balanced sequence, as you can
just look--do experiments you might say with actual primes and the data
shows there is no such behavior--to see the actual rule is to be just
roughly equal to make the naturals PERFECTLY balanced.
For an example from physics, in the double-slit experiment from quantum
mechanical theory the hypothesis is that the electron DOES NOT CARE
where it ends up, so it has an equal probability of going in either
direction which gives you an interference pattern.
See http://physicsweb.org/articles/world/15/9/1
So one way to understand is to consider the a prime p greater than 3
does not care whether or not its residue is 1 or 2 modulo 3, so you get
a random process.
It may be that this is the basis of randomness in our natural world.
> To do that, you would have to look at the probabilities of
> sequences, like '11', '12', '111', '121', etc.
>
> Because there are no theorems about the probability of sequences,
> people here collected actual data.
>
> The evidence is overwhelming. If you assign a probability to
> it, it's on the order of 0.000001 - much less, actually. It's a lot
> less than the probability that a fair coin will come up heads
> 10 times in a row. It's more like coming up heads 1000 times
> in a row.
>
> This still isn't proof. There is still a tiny sliver of a chance
> that you are right.
>
> So the situation is: you don't have a proof for your side.
> Your critics have no proof for their side. This is very likely an
> unsolved problem. But the numerical evidence is overwhelming.
> It's comparable to the evidence that lawyers use, based on
> DNA, to get someone acquitted or convicted. It would very
> definitely meet any LEGAL requirement for proof.
>
> You can deny it if you want, but given that YOU don't have
> even a hint of a proof for your side of the argument, you look
> pretty silly. And calling people liars is mean and dirty and
> makes you look small. No one is lying. They are just honestly
> presenting statistical evidence. Sure, people can use statistical
> evidence to lie, but that doesn't mean statistical evidence is
> ALWAYS a lie. That would be one of your logical fallacies.
>
Well I've talked about how the ordering of the natural numbers--the
counting numbers familiar to many--requires a perfect balance which
restricts the primes, but only so far so that random behavior is
possible.
If there are additional rules, then those can be logically considered
or even empirically derived, and again I give the example of 1/(ln x)
as a case in point.
So there are examples with primes with non-random behavior, so you have
the prime number theorem.
See http://en.wikipedia.org/wiki/Prime_number_theorem
And as this IS such a well-known area where there are HUGE lists of
primes, it seems reasonable to suppose that if such rules existed they
would have been found.
Yet if you push the idea of randomness between primes with this residue
thing, you come up with the theory that (p-2)/(p-1) should feature
prominently, and not surprisingly to me, it does--
See: http://mathworld.wolfram.com/TwinPrimesConstant.html
For readers wondering why (p-2)/(p-1) should feature prominently, note
that with p=3, you just have 1/2, and in general given a prime p that
probability that p+2 does NOT have p as a factor is given by 1 -
1/(p-1) = (p-2)/(p-1).
So there is direct evidence as well for the randomness position.
So I have a logical argument. A prediction about the significance of a
key mathematical expression based on the logical position, and
demonstration within the mathematical literature of that very
expression.
So there is overwhelming evidence in support of the position of
randomness here, which comes even from the camp of mathematicians
themselves.
The will to deny that evidence is the problem.
And notice, all most mathematicians have to do is--nothing.
They simply do not acknowledge the debate, while a few argue endlessly,
always at the end maintaining that they are right, I am wrong, and that
the evidence is in their favor.
There is no intellectual discourse when one side refuses to follow
basic rules.
Mathematicians have enough power and enough trust from the world
community that they feel impervious to the truth, even when
dramatically it can be shown using their own research.
They behave as if they have total power, and the world lets them, so as
long as it does, they do, and part of the consequence is that you do
not know what you may think you know about prime numbers.
James Harris
Mark VandeWettering
You've missed a step, and that step is the difficult one: you have to show
that PRIMES mod 3 are equally likely to have a residue of 1 or 2. It is
*that* part which is difficult.
> But if there is no preference then no rules for a preference can exist,
> meaning that the only rule is an equal probability for each to occur.
>
>> It's known - a difficult theorem, actually - that p mod 3 is just
>> as likely to be '1' as it is to be '2'.
> Yet I just stepped through a simple logical argument relying on the
> triviality that the naturals follow a highly rigid order, which to me
> gets back to an easy way to fool MOST people in an area widely believed
> to be difficult, which is to claim that it's difficult!!!
It is difficult, because you didn't actually address the problem. Nobody
argues that the naturals are equally likely to be 0, 1, or 2 mod 3. But
we weren't talking about naturals, we were talking about primes. You
haven't actually shown anything about this rather limited subset of the
natural numbers.
> Naturals are either primes, products of primes or 1.
Yes.
> Prime preferences push outward to their products, so that a prime
> preference becomes a natural number preference.
What you haven't established is the actually likelihood that a given
prime mod 3 is equally likely to be 1 or 2.
> But natural numbers are simple to most people, as you have
>
> 1, 2, 3, 4, 5 and so on out to infinity
>
> so I connect a supposedly difficult area to trivialities.
You haven't solved the problem at all.
>> But that does NOT prove that the sequence of primes mod 3 is random.
>>
>
> The logical proof is that if there are no rules for the primes beyond
> having a roughly equal number that have 1 as a residue modulo 3 and 2
> as a residue modulo 3, then they behave as if it does not matter, and
> switch randomly between the two.
You have not shown that such a rule doesn't exist.
> So why do I say "roughly"?
>
> Because there is no rule possible to generate a sequence like 1, 2, 1,
> 2, 1, 2, 1, 2 or any other such perfectly balanced sequence, as you can
> just look--do experiments you might say with actual primes and the data
> shows there is no such behavior--to see the actual rule is to be just
> roughly equal to make the naturals PERFECTLY balanced.
I have no idea what you mean.
> For an example from physics, in the double-slit experiment from quantum
> mechanical theory the hypothesis is that the electron DOES NOT CARE
> where it ends up, so it has an equal probability of going in either
> direction which gives you an interference pattern.
>
> See http://physicsweb.org/articles/world/15/9/1
I have no idea why you think this example supports your original premise.
> So one way to understand is to consider the a prime p greater than 3
> does not care whether or not its residue is 1 or 2 modulo 3, so you get
> a random process.
You keep using terminology that implies a pathetic fallacy. Numbers don't
"care" at all. They have properties, like "primality" and "congruence to
1 mod 3". While the primes *as a set* are evenly divided (well, except
for 3 itself) into two sets, there is no reason to believe that the *sequence*
of such digits defined over the primes are show no patterns, any more than
the digits of 4/33.
> It may be that this is the basis of randomness in our natural world.
Babble.
On the contrary, your restrictions (imprecise as they are) apply only
to the composite numbers, not to the prime numbers. You haven't actually
said anything about "randomness" at all. Nothing about your derivation
shows that the sequence of residues is random. (How could it? After all,
nothing in your argument mentioned sequences at all!)
> If there are additional rules, then those can be logically considered
> or even empirically derived, and again I give the example of 1/(ln x)
> as a case in point.
>
> So there are examples with primes with non-random behavior, so you have
> the prime number theorem.
>
> See http://en.wikipedia.org/wiki/Prime_number_theorem
>
> And as this IS such a well-known area where there are HUGE lists of
> primes, it seems reasonable to suppose that if such rules existed they
> would have been found.
>
> Yet if you push the idea of randomness between primes with this residue
> thing, you come up with the theory that (p-2)/(p-1) should feature
> prominently, and not surprisingly to me, it does--
>
> See: http://mathworld.wolfram.com/TwinPrimesConstant.html
>
> For readers wondering why (p-2)/(p-1) should feature prominently, note
> that with p=3, you just have 1/2, and in general given a prime p that
> probability that p+2 does NOT have p as a factor is given by 1 -
> 1/(p-1) = (p-2)/(p-1).
>
> So there is direct evidence as well for the randomness position.
The only prime p for which p+2 has p as a prime factor is 2. I suspect
that you've said something other than what you meant, or frankly, that
you just don't know that you are talking about. The appearance of p-2
in the formulas on mathword are the result of algebraic simplication,
and nothing more.
> So I have a logical argument. A prediction about the significance of a
> key mathematical expression based on the logical position, and
> demonstration within the mathematical literature of that very
> expression.
Wow.
> So there is overwhelming evidence in support of the position of
> randomness here, which comes even from the camp of mathematicians
> themselves.
>
> The will to deny that evidence is the problem.
>
> And notice, all most mathematicians have to do is--nothing.
>
> They simply do not acknowledge the debate, while a few argue endlessly,
> always at the end maintaining that they are right, I am wrong, and that
> the evidence is in their favor.
>
> There is no intellectual discourse when one side refuses to follow
> basic rules.
>
> Mathematicians have enough power and enough trust from the world
> community that they feel impervious to the truth, even when
> dramatically it can be shown using their own research.
>
> They behave as if they have total power, and the world lets them, so as
> long as it does, they do, and part of the consequence is that you do
> not know what you may think you know about prime numbers.
"Thus, those who are skilled think that they are skilled because
they are competent to judge. Those who are unskilled think that
they are skilled because they are incompetent to judge. Therefore,
whoever you are, you think that you are skilled, and there is
no internal way of finding out if you are deluding yourself.
The possibility that we are, in fact, all duffers goes a long
way toward explaining the sorry state of so many things in the
world, in spite of the fact that we each think we could do
better if only we had some say in the matter."
-- Jef Raskin
Mark
> James Harris
mensa...@aol.com
No it doesn't. You get an interference pattern because it goes
through both slits. I thought you had a degree in physics?
>
> See http://physicsweb.org/articles/world/15/9/1
>
> So one way to understand is to consider the a prime p greater than 3
> does not care whether or not its residue is 1 or 2 modulo 3, so you get
> a random process.
>
> It may be that this is the basis of randomness in our natural world.
Yeah, sure.
Strange that despite the evidence, you're wrong. That's a bigger
mystery than the double-slit experiment.
>
> So I have a logical argument. A prediction about the significance of a
> key mathematical expression based on the logical position, and
> demonstration within the mathematical literature of that very
> expression.
>
> So there is overwhelming evidence in support of the position of
> randomness here, which comes even from the camp of mathematicians
> themselves.
>
> The will to deny that evidence is the problem.
>
> And notice, all most mathematicians have to do is--nothing.
>
> They simply do not acknowledge the debate, while a few argue endlessly,
> always at the end maintaining that they are right, I am wrong, and that
> the evidence is in their favor.
>
> There is no intellectual discourse when one side refuses to follow
> basic rules.
>
> Mathematicians have enough power and enough trust from the world
> community that they feel impervious to the truth, even when
> dramatically it can be shown using their own research.
>
> They behave as if they have total power, and the world lets them, so as
> long as it does, they do, and part of the consequence is that you do
> not know what you may think you know about prime numbers.
Do you even have a computer? Are you posting through some
Internet cafe? Did you even have one back in Atlanta, or was all
your Java research done on the job you got fired from? How come
you never write Java programs anymore or talk about deleting stuff
from your hard drive? Is the lack of a computer why your examples
lately are so trivially wrong?
If you had taken me up on my offer of an interview, I could have
arranged for you to have a new computer. The offer is still open,
but since I just paid my property taxes, you'll have some time to
ponder it.
>
>
> James Harris
William Hughes
jst...@msn.com wrote:
<a long article in which he completely and
conspicuously ignores the fact that a residue of
1 is more likely to be followed by a residue of 2>
How do you reconcile the observed fact that a residue
of 1 is more likely to be followed by a residue of 2 with
your claim that the sequence is random?
-William Hughes
Tim Peters
He doesn't, doesn't have the tools (knowledge) to make progress of any kind
with this, and is seemingly too deluded to even consider that he could
profit from studying.
Worse, he clamped his eyes shut and ignored all the evidence posted. If he
hadn't, he /might/ have noticed that the larger the primes people tried, the
teensier bit closer to an even distribution they saw. That was interesting
enough to me to dig deeper.
I've posted quite a bit about what's "really" going on here already, and
won't repeat that here, except to repeat that I suspect the distribution is
in fact /asymptotically/ even, and that this probably follows in a messy way
from Hardy & Wright's conjectured asymptotic formulas for various prime-gap
densities. There are even clear reasons for why that's not apparent at the
relatively tiny primes people tried (me up to 10^9, another poster up to
about 10^13), and for why 11 and 22 specifically are "under represented"
across these spans of "small" primes. 11 or 22 occur when and only when the
gap between adjacent primes is divisible by 6, and while a prime gap of 6 is
conjectured to be asymptotically twice (exactly twice, BTW) as common as
gaps of 2 or 4, across "small" primes gaps of 6 are significantly less
frequent than their conjectured asymptotic density, and despite that 6 is in
fact by far the most common gap across "small" primes(*). Gaps of 12, 18,
24, ..., 6000000, (other gaps divisible by 6) are /way/ less frequent in
this range than their conjectured asymptotic densities.
Working with much larger primes shows a different story, but is expensive to
do. Here's a trial that worked with 251-bit primes. I can't afford the
time to /prove/ primality for integers in this range, so instead this
settles for strong probable primes: integers that aren't divisible by any
prime < 30000, and in addition pass 20 random-base rounds of the
Rabin-Miller probabilistic primality test. There's a vanishingly small
chance of erroneously accepting a composite this way (worst cases for
Rabin-Miller are rare, and even for worst-case values only 1 in 4^20 = 2^40
can be expected to pass a 20-round test).
The first strong probable prime tried happened to be (picked by a
pseudo-random process):
2516552117631275271491127081148545403086308796861212790243626842984599674611
From there, the next larger 7999 probable primes were found, and 4000
non-overlapping pairs of adjacent primes were formed. These are the stats
for their residues mod 3:
(1, 1) 954
(1, 2) 1018
(2, 1) 1034
(2, 2) 994
chisq 3.632 with 3 degrees of freedom
That's a thoroughly unremarkable chi-square statistic: the probability of
seeing one <= 3.632 if the sequence were truly uniformly random is about
0.70. A small bias against 11 and 22 may or may not be apparent there.
Trying 300-bit probable primes, the first tried was (split across lines):
1997292095801987703489887151648570381008658007
776476818511181008606152931224561215249751379
and the stats:
(1, 1) 998
(1, 2) 1016
(2, 1) 999
(2, 2) 987
chisq 0.423 with 3 degrees of freedom
There the chi-square statistic is in danger of getting in trouble on the
/other/ end of the scale: the probability of seeing a value <= 0.423 is
about 0.065 -- if anything, this is /too/ evenly distributed(!).
I don't know at which range the stats first stop being insanely bad. I had
reason to suspect (and later confirmed) they remain insane at least for the
primes through 10^20, so leaped to ~300 bit primes. The run for 250-bit
primes above was part of a search downward to try to bound the range. I
haven't completed that search, and probably won't (the time I carved out for
this is gone).
(*) So which is /the/ most common prime gap? If Hardy & Littlewood were
right, there isn't one: in the limit, there are infinite sequences
of increasing gaps:
m_1 < m_2 < m_3
such that the number of primes <= x with gap m_1 is asymptotically
less than the same thing for m_2, which in turn is less than for m_3,
etc.
A specific such sequence, and in a precise sense "the best" such
sequence, is given by the primorials, the product of all primes <=
some given prime. 6 = 2*3 is a primorial, and the next is 2*3*5 = 30,
then 2*3*5*7 = 210, and so on. While, e.g., /asymptotically/ gaps
of 210 are conjectured to be more frequent than gaps of 6, there are
no gaps of 210 at all before 20831323. That's an example of why
the asymptotics have no chance of being accurate for "small"
primes.
Why are those "most common"? Intuitively, consider p + 2*3*5*7.
If p is prime and > 7, then p + 2*3*5*7 can't be divisible by any
of {2, 3, 5, 7} either. That makes p + 2*3*5*7 significantly more
likely to be a prime than, say, p+2 (of which, for prime p > 2, we
can only be sure isn't divisible by 2).
William Hughes
Tim Peters wrote:
> [William Hughes]
> > jst...@msn.com wrote:
> >
> > <a long article in which he completely and
> > conspicuously ignores the fact that a residue of
> > 1 is more likely to be followed by a residue of 2>
> >
> > How do you reconcile the observed fact that a residue
> > of 1 is more likely to be followed by a residue of 2 with
> > your claim that the sequence is random?
>
> He doesn't, doesn't have the tools (knowledge) to make progress of any kind
> with this, and is seemingly too deluded to even consider that he could
> profit from studying.
>
> Worse, he clamped his eyes shut and ignored all the evidence posted.
Indeed, and while he has a history of ignoring unpalatable facts, this
is an extreme case.
Very interesting. I note that you saw similar behaviour
(non-uniformities in the
distributions of pairs of consecutive residues) with other primes.
Do you have any comments or conjectures on these two questions:
a: Are the pair distributions asympotically uniform for any prime?
b: Why are the pair distributions generally non-uniform for "small"
ranges?
(Note that even if the pair distributions (p=3) are asymptotically
uniform,
any finite seqence of residues is still compressible (even ignoring the
obvious)
by compressing the sequence for the first 10^20 or so primes and
providing
the rest of the sequence verbatim. Thus the question of asymptotic
uniformity does not bear directly on the question of whether the
sequence
is random.)
-William Hughes
David Bernier
I think it would be interesting to see by how much the sequence:
5-3*floor(5/3), ... 1000000007-3*floor(1000000007/3)
can be compressed by Winzip, Gzip, bzip2, etc.
David Bernier
Tim Peters
distribution using 251- and 300-bit (probable) primes; but notes that
gross imbalance still exists at primes at least through 10^20]
[William Hughes]
> Very interesting. I note that you saw similar behaviour
> (non-uniformities in the distributions of pairs of consecutive
> residues) with other primes.
Yes, and imbalance is more obvious earlier, and persists through larger
primes, the larger the p you take.
A specific example given was p=541, where it's impossible to see two
consecutive equal values before at least 10^15 because there are no primes
with gap 2*541 (or gap divisible by that) smaller than 10^15 (and I don't
even know whether the range 10^15-10^16 contains such a prime -- here I'm
going by that the largest gap for /any/ prime < 10^15 is 906:
http://www.research.att.com/~njas/sequences/A053303
).
Similarly it's impossible to see 1 followed by 2 in the mod(p, 541)
sequence, or 2 followed by 3, or i followed by i+1, at least before looking
at primes > 10^12, because the largest gap for any prime < 10^12 is 540,
less than the smallest gap that "could work" (541+1 = 542). In fact, it
turns out that /most/ of the possible 540^2 non-zero residue pairs can never
be seen before reaching what most of us used to <wink> think of as being
"pretty big" primes.
Therefore it turned out to be dead easy to write a dead simple "predictor"
for the mod-541 case that did 100x better than "chance level" for primes
into the billions. Chance level thinks all 540^2 pairs are equally likely,
while a simple learner quickly discovers that most pairs are never seen.
> Do you have any comments or conjectures on these two questions:
Yes, but I've already written about these multiple times, and it's getting
tedious now ;-) Unfortunately, thanks to James creating an endless number
of new threads irresponsibly cross-posted all over creation, even I have
trouble finding them now. I'll regurgitate all the "high spots" once more:
> a: Are the pair distributions asympotically uniform for any prime?
I suspect so, but nobody can prove it. My analysis builds on Hardy &
Littlewood's 1920's conjectures (still unproved, but still more-or-less
"believed") about the asymptotic densities of prime gaps, so is at least as
dubious as those conjectures. The distribution here is really all about
prime gaps, and little about those /has/ been proved, not even for gap 2.
Let g_i = p_(i+1) - p_i, i.e., g_i is the gap between the i'th prime and its
successor. Then two adjacent terms in James's sequence are equal if and
only if
p_(i+1) = p_i (mod p) iff
p_(i+1) - p_i = 0 (mod p) iff
g_i = 0 (mod p) iff, and assuming i > 1
g_i = 0 (mod 2*p) since g_i also must be even when i > 1
So in the p=3 case, adjacent terms are equal iff g_i is divisible by 6. <1,
2> occurs (exercise for the reader) iff g_i = 4 (mod 6), and <2, 1> occurs
iff g_1 = 2 (mod 6).
The smallest g_i in those three classes are 2, 4 and 6. The H&L conjectures
say that the number of primes with gap 2 is asymptotically equal to the
number with gap 4, and that the number with gap 6 is asymptotically equal to
the sum of those two (gap 6 is twice as frequent as gap 2 (or gap 4)). So
if 2, 4, and 6 were the only /possible/ gaps, we'd almost be there. It
would just remain to show that the 6|gap case is asymptotically equally
split between <1, 1> and <2, 2>.
But there are an infinite number of possible prime gaps, so continuing the
analysis gets much messier. I haven't completed it to my satisfaction.
For p > 3, similar analysis applies at the start. The (p-1)^2 possible
residue pairs are partitioned into p classes, corresponding to the
associated gap's residue class mod p. For example, for the pair <12, 21> at
p=541:
p_i = 12 (mod 541)
and
p_(i+1) = 21 (mod 541)
implies
g_i = p_(i+1) - p_i = 21 - 12 = 9 (mod 541)
so pair <12, 21> belongs to the residue 9 class. It's easy to show that the
gap residue class 0 corresponds to the p-1 "both equal" pairs, and that the
remaining p-1 gap residue classes each correspond to p-2 "not equal" pairs.
Intriguingly, there's one more pair in the residue 0 class than in any of
the other residue classes. Therefore if it were the residue classes that
were equally likely, and within a residue class everything split evenly,
then <i, i> would have a smaller count than <i, j> for i != j. But just as
in the p=3 case, because /all/ the residue-0 gaps are divisible by p,
computing their Hardy-Littlewood constants removes a "natural factor" of
(p-1)/(p-2) from each, and simultaneously throws away all factors of p,
leaving what remains "looking a lot like" all the other Hardy-Littlewood
gap-constant computations for the "not equal" pairs. Of course the common
(p-1)/(p-2) can be factored out of the residue-0 sum.
All that leaves me conjecturing that the residue 0 class is (p-1)/(p-2)
times more likely than each of the other residue classes, which are equally
likely; i.e., that residue class 0 occurs with probability 1/(p-1), while
each of the p-1 non-zero residue classes occurs with probability
(p-2)/(p-1)^2. It's a good sign that those add to 1 :-) Also a good sign
that, if the probability for a residue class is split evenly among the pairs
in that class, then for a non-zero residue class (containing p-2 pairs) we
get:
(p-2)/(p-1)^2 / (p-2) = 1/(p-1)^2 per pair
and for the zero residue class (containing p-1 pairs) we also get:
1/(p-1) / (p-1) = 1/(p-1)^2 per pair
Is that a proof? LOL! I'm not James ;-) It's a sketch of what looks like a
promising approach that hits difficulties right after what I described. I
wrote a program to do all those calculations (computing and summing all the
Hardy-Littlewood constants, partitioned by residue class), and tried it over
all odd primes < 1000, for ever-increasing "maximum gap" cutoffs (since
these are infinite sums, some notion of limit needs to get involved).
/Numerically/ they appear to converge, and to the values conjectured above,
although that's proving harder to show rigorously than I guessed at first.
And at second :-(.
> b: Why are the pair distributions generally non-uniform for "small"
> ranges?
It's well-known that the simplest (Hardy & Littlewood) conjectures for the
number of primes <= x with prime gap g give poor approximations for g > 4
and x "small". They can vastly overestimate. For example, /asymptotically/
the number of primes <= x with prime gap 2 is conjectured to be equal to the
number with prime gap 2^10000000 (or any other power of 2). At small
(compared to 2^10000000) x, the latter is clearly an absurd estimate -- the
actual count is 0 then.
Then there are an infinite number of prime gaps feeding into each residue
class in the current problem, and the asymptotics "predict" non-trivial
counts for /all/ of them, while in reality "almost all" counts are exactly 0
no matter how large a finite x we look at.
Finally, and most importantly, H&L's conjectures leave us with this seeming
paradox: the number of primes <= x with gap g is asymptotically equal to
the number of primes p <= x such that p+g is also a prime /regardless/ of
whether some other prime(s) appear between p and p+g. H&L were actually
conjecturing about the latter and their formulas work well for that. At
least at small x, it's not at all unlikely that if, e.g., p and p+6 are both
prime, then p+2 or p+4 are also prime, leaving a gap of 2 or 4 instead of 6.
This leaves the asymptotics overestimating the ratio of the gap-6 count to
the gap-2 count in "small reality", or, IOW, "11 and 22 don't show up often
enough". Note that this can't happen for gaps of 2 or 4 (e.g., if p and p+4
are prime, it's impossible that p+2 is also prime (one of the three must be
divisible by 3), provided p > 3).
That's the /primary/ problem "in the small", and is the real reason for why
the asymptotic formulas for gaps 2 and 4 appear to work very well for small
x, while the formulas for g > 4 suck eggs for small x. Brent developed what
appear to be much better formulas for g > 4 and small x, with the same
asymptotic values, but very different behavior at small x:
"The Distribution of Small Gaps Between Successive Primes"
http://wwwmaths.anu.edu.au/~brent/pd/rpb021.pdf
That basically applies inclusion-exclusion to the H&L asymptotics in an
attempt to distinguish between "prime gap of g" and "merely p and p+g are
both prime" cases. His computational results are very good, but note that
it takes an enormous amount of work (exponential in g/2) to derive one of
his formulas for gap g. I haven't checked this, but I fully expect that if
you plug in Brent's formulas for small-prime gaps, instead of the much
simpler H&L densities, they'd do a good job of predicting the distributions
we've actually seen across primes < 10^15.
> (Note that even if the pair distributions (p=3) are asymptotically
> uniform, any finite seqence of residues is still compressible (even
> ignoring the obvious) by compressing the sequence for the first
> 10^20 or so primes and providing the rest of the sequence verbatim.
> Thus the question of asymptotic uniformity does not bear directly on
> the question of whether the sequence is random.)
Wholly agreed. This whole exercise on my end was predicated on a "let's
pretend" willingness to ignore the obvious, and just see whether the
sequence as given could pass standard statistical tests for randomness. I
happen to have a lot of old experience writing production pseudo-random
number generators, and when James posted his sequence my brain /immediately/
said "hey, that looks fishy! it alternates more often than 'it should'".
That's why I did a pair-distribution test to begin with. As I said near the
start, that's not even an especially good test, but experience taught me to
use the simplest test that gets the job done ;-) "Good enough" uniformity
of singletons and pairs (and triples and quads and ...) are necessary to
pass standard tests, but not sufficient.
In any case, nothing can change that this sequence /does/ fail that very
simple test spectacularly at least across primes through 10^20, and even if
achievable, "asymptotically but never usefully random-acting" is a novel
concept of dubious utility to me :-) It remains interesting to me, and for
its own sake, to know whether k-tuples of adjacent primes modulo a fixed
prime p are asymptotically equidistributed (and that's already been proved
for k=1, but since k>1 ends up being about the distribution of prime gaps
there's no immediate prospect for proof).
David Bernier
In the primes mod 3 sequence, for primes>3 and < 10^7, we get a finite
subsequence of 664,577 numbers in {1,2}.
I used a simplistic encoding 1 -> a byte '1', 2 -> a byte '2'.
A 664,577-byte file didn't even compress by a factor of 8 using the
Winzip default. I think I got 84% reduction with the PPmD option;
but that leaves 16%, or 0.16 > 1/8.
The obvious thing is to code in binary, and then compress. But
I've put that on hold. Another possibility is to develop
a custom compressor based on all of Tim Peters pretty considerable
computations and comments. It's not clear to me how much
pseudo-entropy is lost for subsequences of length ~= pi(10^9)
using the correlations he found. I think a realistic
probabilistic model for these correlations could be based on
Markov chains, but I don't know how to compute entropies of Markov
chains or processes.
I thought James might change his mind after I
gave a tennis tournament analogy, but nope.
David Bernier
David C. Ullrich
Look. First, you have not given a proof that 1 and 2 are equally
likely for p mod 3. But more important, nobody's _disputing_ that
fact. It does not follow that the sequence is random.
For example, in a random sequence all four possible consecutive
pairs 11, 12, 21, 22 would be equally likely, and that's simply
not so.
>> It's known - a difficult theorem, actually - that p mod 3 is just
>> as likely to be '1' as it is to be '2'.
>>
>
>Yet I just stepped through a simple logical argument relying on the
>triviality that the naturals follow a highly rigid order, which to me
>gets back to an easy way to fool MOST people in an area widely believed
>to be difficult, which is to claim that it's difficult!!!
>
>Naturals are either primes, products of primes or 1.
>
>Prime preferences push outward to their products, so that a prime
>preference becomes a natural number preference.
Fascinating. Let's see. Most primes are odd. In fact there's only
one even prime; asymptotically 100% of the primes are odd.
Primes show a _huge_ preference for oddness.
Now we apply the principle that "prime preferences push outward
to their products" and we conclude that the natural numbers show
a huge preference for oddness. Does this sound right to you?
Looks to me like half the natural numbers are even.
************************
David C. Ullrich
Ken S. Tucker
David C. Ullrich wrote:
...
> Fascinating. Let's see. Most primes are odd. In fact there's only
> one even prime; asymptotically 100% of the primes are odd.
> Primes show a _huge_ preference for oddness.
>
> Now we apply the principle that "prime preferences push outward
> to their products" and we conclude that the natural numbers show
> a huge preference for oddness. Does this sound right to you?
> Looks to me like half the natural numbers are even.
Is "10.4" even? It is a prime number.
Ken
mensa...@aol.com
But isn't that just an illusion? On the Collatz Tree, every odd
number extends to infinity by multiplication by 2. So aren't
there infinitely more even numbers than odd?
Tim Peters
>> I think it would be interesting to see by how much the sequence:
>> 5-3*floor(5/3), ... 1000000007-3*floor(1000000007/3)
>> can be compressed by Winzip, Gzip, bzip2, etc.
I was going to ask whether it was interesting enough to try it yourself, but
you beat me to it :-)
[also David]
> In the primes mod 3 sequence, for primes>3 and < 10^7, we get a finite
> subsequence of 664,577 numbers in {1,2}.
>
> I used a simplistic encoding 1 -> a byte '1', 2 -> a byte '2'.
> A 664,577-byte file didn't even compress by a factor of 8 using the
> Winzip default.
Which is pretty pathetic, since there are only two possible byte values, so,
e.g., a dead-simple Huffman encoder would assign a 1-bit "code" to each, and
get a factor of 8 compression even if the distribution of bytes were truly
random. But that's not the kind of thing WinZip is aiming at -- it's only
"pretty pathetic" compared to theoretical alternatives.
> I think I got 84% reduction with the PPmD option; but that leaves 16%,
> or 0.16 > 1/8.
>
> The obvious thing is to code in binary, and then compress.
I expect that will compress much worse using most common compression
algorithms. Those aim at multi-byte repetitions, not at bit-level patterns.
> But I've put that on hold. Another possibility is to develop
> a custom compressor based on all of Tim Peters pretty considerable
> computations and comments.
An "arithmetic encoder" might work well for the binary form, but that's
encumbered by patents so you might have to write your own implementation. I
can tell you for a fact it's not interesting enough to me to do it ;-)
> It's not clear to me how much pseudo-entropy is lost for subsequences
> of length ~= pi(10^9) using the correlations he found. I think a
> realistic probabilistic model for these correlations could be based on
> Markov chains, but I don't know how to compute entropies of Markov chains
> or processes.
>
> I thought James might change his mind after I gave a tennis tournament
> analogy, but nope.
Don't feel bad: James almost never admits to changing his mind based on
what anyone says. Much more likely is that he'll announce he's discovered
"the reason" for why something didn't work out, the reason given will be a
new one (not one of the /actual/ reasons handed to him 2 to 200 times). You
can sometimes find distorted echoes of the actual reasons in such
explanations. For example, maybe he'll "explain" this one by imagining a
two-slit experiment performed on a tennis court. Then you'll know that one
of your words got through, although none of your meaning ;-)
Tim Peters
>> But if there is no preference then no rules for a preference can exist,
>> meaning that the only rule is an equal probability for each to occur.
[David C. Ullrich]
> Look. First, you have not given a proof that 1 and 2 are equally
> likely for p mod 3. But more important, nobody's _disputing_ that
> fact.
Except that we probably should dispute it :-(. Sloppy statements about
probability are a useful shorthand among people who know what they're
talking about, but James hasn't earned that privilege, and when we (I'm as
guilty as anyone, and much more so than some) let one of his sloppy
statements slide, we're implicitly letting his misunderstandings slide too.
In this case, what nobody /actually/ disputes is that, as x approaches
infinity, the ratio of:
the number of primes <= x congruent to 1 (mod 3)
to
the number of primes <= x congruent to 2 (mod 3)
has a limit, and that the limit is 1. But that's not what James has in mind
at all, and the rest of us know perfectly well that what we actually agreed
to above doesn't imply /anything/ about the ratio of the counts at any
finite x. I'm willing to agree that will also be "close to" 1, and you are
too, because beyond the above we also know at least the general form of the
error term for the asymptotics here.
But James doesn't have a clue about that either, so what would amount to
harmless (actually helpful) agreement to a sloppy statement when made by
many others sadly amounts to encouraging James to proceed building on a
foundation of thin soup.
Then again, most aren't really trying to "help James" anyway, and it's more
fun to ignore the hopeless parts and find an interesting place to explore.
> It does not follow that the sequence is random.
> For example, in a random sequence all four possible consecutive
> pairs 11, 12, 21, 22 would be equally likely, and that's simply
> not so.
Although I believe it probably /is/ so asymptotically here, but can't be
seen before using very much larger primes (e.g., in one run using 300-bit
primes, the chi-square statistic was so small it nudged into the "hmm, this
is /too/ evenly distributed to be random" end of the scale -- posted a lot
about that yesterday -- it has to do with the distribution of prime /gaps/,
and that the conjectured asymptotics for gaps > 4 are bad estimates at least
across primes < 10^20).
Of course, if true, "random" doesn't follow from that either.
> ...
> Now we apply the principle that "prime preferences push outward
> to their products" and we conclude that the natural numbers show
> a huge preference for oddness. Does this sound right to you?
> Looks to me like half the natural numbers are even.
Shows what you know ;-)
> ...
William Hughes
Thanks
-William
jst...@msn.com
If p mod 3 gives a random sequence as you march up primes p greater
than 3, then it gives the random sequence for two cases. That is, any
random sequence of only two elements, is the same as any other, so
anything that can be seen in any such random sequence can be seen in
it.
It is possible in a random sequence of flips of 1's and 0's to think
you see 1 showing up after you have 0, more than random would allow,
over an interval of arbitrary length.
One of the more interesting thinking fallacies that many people are
prey to is the belief that apparent patterns in random sequences
disprove randomness, and a similar argument is used by Creationists
fighting against evolutionary theory.
People are susceptible to the pattern argument, but that does not make
it true.
There are simple reasons for why people are easily drawn in by it as in
the real world, assuming patterns could keep you from getting eaten by
some big cat or other predator, while being wrong about them, just gave
a false alarm.
Better to jump and survive than stick around pondering if it was a real
pattern or some random event, and maybe get eaten. Those odds are
easy!
So it's built into the human brain to see patterns even when faced with
random.
Why don't you start at some prime way up the ladder, like the millionth
prime, and see if that pattern still seems to hold.
Report back what you or someone else finds.
James Harris
William Hughes
jst...@msn.com wrote:
> William Hughes wrote:
> > jst...@msn.com wrote:
> >
> > <a long article in which he completely and
> > conspicuously ignores the fact that a residue of
> > 1 is more likely to be followed by a residue of 2>
> >
> > How do you reconcile the observed fact that a residue
> > of 1 is more likely to be followed by a residue of 2 with
> > your claim that the sequence is random?
> >
> > -William Hughes
>
> If p mod 3 gives a random sequence as you march up primes p greater
> than 3, then it gives the random sequence for two cases. That is, any
> random sequence of only two elements, is the same as any other, so
> anything that can be seen in any such random sequence can be seen in
> it.
>
> It is possible in a random sequence of flips of 1's and 0's to think
> you see 1 showing up after you have 0, more than random would allow,
> over an interval of arbitrary length.
This is silly. Your sequence is deteministic. For such a sequence
random and incompressible are the same thing.
-William Hughes
marcus_b
No, this argument doesn't work. Let p be a prime bigger than 3.
The multiples of p are congruent to 0, 1, or 2 mod 3, and each of
these has a relative frequency of 1/3. It doesn't matter whether p
itself is congruent to 1 or 2 mod 3. That is, the fact that the
multiples of p have equal probabilities of being congruent to 1 or
2 doesn't tell you anything about p itself. Your statement that
"(if) the residues modulo 3 show a preference for 1 then their
products would also show that preference" is obviously nonsense.
> But if there is no preference then no rules for a preference can exist,
> meaning that the only rule is an equal probability for each to occur.
>
> > It's known - a difficult theorem, actually - that p mod 3 is just
> > as likely to be '1' as it is to be '2'.
> >
>
> Yet I just stepped through a simple logical argument relying on the
> triviality that the naturals follow a highly rigid order, which to me
> gets back to an easy way to fool MOST people in an area widely believed
> to be difficult, which is to claim that it's difficult!!!
>
> Naturals are either primes, products of primes or 1.
>
> Prime preferences push outward to their products, so that a prime
> preference becomes a natural number preference.
>
No - the multiples of primes which are congruent to 1 mod 3 are
equally likely to be 0, 1, or 2 mod 3. Take p = 7 as an example.
Of course 7 is congruent to 1 mod 3. The first 30 multiples are
7, 14, 21, 28, 35, 42, 49, 56, 63, 70, 77, 84, 91,
98, 105, 112, 119, 126, 133, 140, 147, 154, 161, 168, 175,
182, 189, 196, 203, 210.
Of these, exactly 10 (1/3 of them) are congruent to 1 mod 3. Prime
preferences do NOT "push outward to their products". That is obvious
hogwash.
> But natural numbers are simple to most people, as you have
>
> 1, 2, 3, 4, 5 and so on out to infinity
>
> so I connect a supposedly difficult area to trivialities.
>
No, your argument sounds good superficially but it is totally bogus.
This theorem is lots harder than you think.
Since your argument for singletons is clearly bogus, you have
no argument at all for pairs, triplets, etc.
Marcus.
[snip]
Tim Peters
> ...
> Why don't you start at some prime way up the ladder, like the millionth
> prime, and see if that pattern still seems to hold.
>
> Report back what you or someone else finds.
I covered that range in my very first reply to you, and subsequent replies
went thru (all of) the first 51 million primes, where the chi-square
statistic got so ludicrously large there was less than 1 chance in 10^60000
that a truly random sequence could have produced a value so large. Later
replies (from me & others) did spottier checking at ever-larger primes with
no joy. And then ...
Catch up (from the above, you still need to digest the very first reply),
and you'll eventually find residue-pair results more in tune with your
prejudices.
b92...@yahoo.com
jst...@msn.com wrote:
> I find it fascinating watching math people lie on newsgroups about
> basic things.
>
> Some of you may think you understand probability and statistics, but
> consider a question like, if someone flips a coin 10 times and gets 10
> heads, and THEN flips the coin 10 times and get 10 tails, is the coin
> flawed?
>
> If you think the answer is yes, you're wrong.
>
> The correct answer is, not enough information given.
>
> If you think there is no way the coin can do that given the odds, then
> you are wrong.
>
> But no trained scientist (properly trained) would get the question
> wrong.
>
> So then, can a chi square test or any other statistical test prove that
> p mod 3 with p a prime greater than 3 does NOT gives a random sequence?
Please explain how "p mod 3" ; just one number; can be considered a