Data analysis #001
Gary Carson
Data Mining and Analysis for Poker Players
Chapter 1 Data
1.1 What the course is all about
Data analysis. Data mining. It's all about data. You need to collect it and
you need to figure out what it means. Data will set you free if you know how to
collect data in a meaningful way, and know how to read it and listen to it.
Once you get it you won t have to rely on pronouncements from self anointed
experts to know the truth.
Personal computers and the internet has set us free from the shackles of
believing pronouncements from others by providing us with lots and lots of data
that we can analyze and powerful tools to do the analysis, so we can uncover
truth and actual facts all by ourselves.
The goal of this course is to provide students with a basic understanding of a
variety of research tools, including mathematical modeling, statistical
analysis, and qualitative analysis which can be used in the improvement of
understanding various poker phenomena.
Tools to address the cards, the betting, the players, and bankroll and risk
management will be studied. All of these tools are intended to be used in
collecting and understanding observations the analysis of data.
Poker is a process with inputs of cards, money and people and outputs of money.
The process itself is reflected by player decisions. Data is available on the
inputs, the outputs, and the process itself. We collect that data in different
ways, sometimes by observation, sometimes by probability calculations, by
knowledge of human behavior, by the optimization of mathematical models, etc.
All this data can be combined to tell us stories about the game of poker.
Data is some kind of observation or measurement that represents some element of
information. It s really kind of a vague concept. A number, a name, a symbol,
a value are all forms of data.
We're going to talk about data that a poker player might collect or observe and
various methods you might use to extract, organize, or use the information
represented by the data.
The analysis of data is the extraction of information from collections of data.
In the analysis of poker, the idea is to use that information to improve our
monetary results. Often that involves improving our playing decisions during a
hand, but it sometimes involves nothing more than improving our ability to
identify weak opponents to play against.
The intended use of the information we can extract from data always involves
some kind of improvement of our decision making.
Sometimes the decision making is just about whether or not to play, or what
table to play at, sometimes it's more refined and focused on decisions during
the play of a hand such as whether to fold or raise.
Data analysis is a form of information reduction. We don t add to our
collection of information when we analyze data, we reorganize it into a
meaningful form, and we usually do that with reduction techniques. Every
observation in a large collection of data is a piece of information. But
unorganized and unreduced data doesn t often provide the information in a
meaningful form. We might use graphical tools to convert a mass of numbers into
a picture, or we might use computational tools to summarize hundreds of
numerical values with just three or four values.
Of course, it's also possible to analyze data, and the information contained in
the data, just to satisfy natural curiosity. We aren't going to answer "What
are the odds of that?" kinds of questions here, although some of the tools we'll
discuss can be used for such curiously satisfaction research. We ll also
develop some tools that can be used to illustrate why seemingly unusual events
often aren t unusual at all.
We ll be touching on topics such as descriptive statistics, estimation,
graphical analysis, probability models, stochastic methods, optimization, game
theory, correlation and regression. None of those topics will be covered in
complete detail, but hopefully we'll give you enough detail for you to be able
to apply the tools to analyze common poker decisions. Of course, we're not
going to be complete, we're going to focus on the use of statistics and
probability models in the analysis of data related to poker.
We'll be using a couple of software tools. PokerTracker will be used to create
a database of results from online site hand histories. The data will also be
extracted into Excel for further analysis. The plan is to mostly focus on data
from 6 handed 1/2 blind NL Hold em games on UltimateBet. We ll also be using
computational and simulation tools Poker Stove, TwoDimes.net, and Texas Turbo
Hold em to generate some of our own data.
1.2 Types of data
1.2.1. Categorizing data
There are many ways to characterize data. They can be numeric, or non-numeric.
Numeric data can be discrete or continuous. Non-numeric data can be nominal or
ordinal (we ll define that in a moment). In social science research data is
often classified as survey data or official data, a classification based on
source. Survey data might be self-reported or it could be observational. Self
reported data is just the response to a question you ask. Or, alternatively you
can just observe some behavior of interest and make note of it. When using
things like self reported mail or phone surveys, or internet surveys, you need
to take care that non-responders aren t somehow of a distinct nature from those
who voluntarily respond.
We can classify data based on it s mathematical properties, it s method of
collection, it s source, on the measurement techniques used in it s collection,
how it s going to be analyzed, or many other categories.
Common categories of data include various scales and binary responses. Binary
responses are just yes/no type responses. Data that only has two possible
values. It might be male/female or aggressive versus non-aggressive. But
sometimes seemingly binary measurements such as aggressive/non-aggressive are
better measured using what s called scales.
A scale is on ordered set of responses or values, ordered so that one response
is bigger or better in some way than the next one. An aggression scale might be
very aggressive, slightly aggressive, non-aggressive, and passive. That would
be a decreasing scale of aggressiveness that probably captures the essence of
the concept in a better way than the simple binary categories of
aggressive/non-aggressive.
This first chapter focuses on different ways to classify data and some
mathematical properties that data may or may not have. In the second chapter we
talk just a little about data that s been organized into a formalized
distribution of data. We ll get into actually computing some statistical
measurements in Chapter 3, and move on from there into more complex, and more
robust research methods. etc, etc
Comments? Questions?
Gary Carson
I'll be posting an updated outline and other links and updates on my website.
I'll try to do the next "lecture" on next Tuesday.
http://www.garycarson.com
_______________________________________________________________
Posted using RecPoker.com v2.2 - http://www.recpoker.com
TheMiner
hey I like that 'data mining' slogan. I will mine into your head and take you
out is my slogan...hence..TheMINER
:)
eleaticus
"Gary Carson" <garyc...@alumni.northwestern.edu> wrote in message
news:1140558767$738...@recpoker.com...
>
> Data Mining and Analysis for Poker Players
> A scale is on ordered set of responses or values, ordered so that one
response
> is bigger or better in some way than the next one.
A traditional and meaningful scale is the 'nominal' scale which does NOT
order anything except maybe the psychological response to it. Ex: the
subject names are placed in a hat and drawn at random, first out into group
C, second to group B, third to group A, then 4th-6th in those groups in
reverse order maybe. Etc. "A, B, C" appears to the eye of anyone who knows
the alphabet to be an ordering, but obviously isn't. The famous example is
'football' numbers, the subject of a long, famous and (on one side) idiotic
debate, the football players being given numbers based on their kind of
position (lineman, backs, offensive, defensive) but the moron side thought
that being numbers it was ok to use mathematical statistics such as t-tests
on them, measures that are only valid for 'ratio' and 'interval' scales, and
not even valid for 'ordinal' scales. The test being used to decide if
upperclassmen were being favored with higher numbers (or some such).
Yes, the word 'scale' does feel like it necessitates an ordering, but
failure to include the nominal leaves the domain of the word incomplete.
> This first chapter focuses on different ways to classify data and some
> mathematical properties that data may or may not have.
Ratio, interval, ordinal, nominal, in decreasing order of useful math
properties.
eleaticus
ee-lee-AT-i-cus
Gary Carson
Yes, that's a good point, some people do use the word scale to include just
arbitrary labels, but I don't. I'm going to reserve the use of the word scale
to include only some kind of ordering, even if only a rough ordering.
Here's an example of how I'll be using the term "scale".
I think that's the more common usage, but whether it is or not, that'll be the
usage I'll be using.
On another topic, your misconception that particular statistical procedures are
related to "level of measurement" is a common fallacy.
In the early 50's, the idea of level of measurement was developed as a guide to
which computational procedures were appropriate -- that's because there were
computational shortcuts depending on certain measurement properties. But, that
wasn't really statistical, it was computational. Since few computatoins are
done by hand anymore it doesn't really matter much any more.
Statistical procedures assume distributional characteristics, not measurement
characteristics. There was a very good, quite readable, article in American
Statistitician in the early '90s (or maybe late 80's) on the subject. I think
I'll talk about levels of measurements in the 3rd lecture or something like
that.
But, anyway, thanks for suggesting that I make my usage of the term scale more
clear.
On Feb 21 2006 10:21 PM, eleaticus wrote:
> "Gary Carson" wrote in message
Gary Carson
http://www.garycarson.com
_______________________________________________________________
Your Online Poker Community - http://www.recpoker.com
Gary Carson
On Feb 21 2006 11:28 PM, Dwight Abbott wrote:
>
> What no homework assignment?
You could read chapter 1 (it's on the website for download). I'm going to
be going slowly, chapter 1 will take 3-4 lectures.
Looks very interesting Gary. I'm all ears.
>
> Dwight
>
> On Feb 21 2006 1:52 PM, Gary Carson wrote:
>
> >
> > Data Mining and Analysis for Poker Players
> > A scale is on ordered set of responses or values, ordered so that one
> > response
> > is bigger or better in some way than the next one. An aggression scale
> > might
> > be
> > very aggressive, slightly aggressive, non-aggressive, and passive. That
> > would
> > be a decreasing scale of aggressiveness that probably captures the essence
> > of
> > the concept in a better way than the simple binary categories of
> > aggressive/non-aggressive.
> > This first chapter focuses on different ways to classify data and some
> > mathematical properties that data may or may not have. In the second
> > chapter
> > we
> > talk just a little about data that s been organized into a formalized
> > distribution of data. We ll get into actually computing some statistical
> > measurements in Chapter 3, and move on from there into more complex, and
> > more
> > robust research methods. etc, etc
> >
> > Comments? Questions?
> >
> > Gary Carson
> > I'll be posting an updated outline and other links and updates on my
> > website.
> > I'll try to do the next "lecture" on next Tuesday.
> > http://www.garycarson.com/
> >
> >
>
>
Gary Carson
http://www.garycarson.com
_______________________________________________________________
* New Release: RecPoker.com v2.2 - http://www.recpoker.com
Necron99
Can you please not use a font that presents as bold in web based readers.
Makes it vey hard to read.
** insert witty/funny original comment **
Gary Carson
On Feb 21 2006 11:57 PM, Necron99 wrote:
> Looking forward to it.
> Can you please not use a font that presents as bold in web based readers.
> Makes it vey hard to read.
I'll try.
You can download it in word format from the website, I just did a cut and past
from that document -- the headers are bold, not the text. Is this translated
okay below?
One mathematical property that we often implicitly assume, but which sometimes
doesn t hold when dealing with poker related data (or in gambling in general) is
transitivity. That s the property that if A > B and B > C then we know that A >
C. Whenever we re dealing with processes based on multi-player match-ups,
whether it s a poker game or a basketball tournament, this property of
transitivity does not hold.
Gary Carson
http://www.garycarson.com
_______________________________________________________________
Posted using RecPoker.com v2.2 - http://www.recpoker.com
Necron99
Thanks
_______________________________________________________________
Watch Lists, Block Lists, Favorites - http://www.recpoker.com
Peg Smith
>Comments? Questions?
I'm impressed that you're doing this much work for the group. I
haven't made much of an effort to learn much here for a long time, but
I'll read every word of this. This first installment was very well
done, clear and understandable, which it needs to be since it isn't a
topic I'm normally interested in. If it had been too complex or
unclear I wouldn't hang in there. I respect your knowledge and I'm
looking forward to the rest.
Peg
Peg Smith
>I'm going to
>be going slowly, chapter 1 will take 3-4 lectures.
We who are old, tired and lazy appreciate that.
Peg
Gary Carson
Don't worry. Your eyes will glaze over soon.
On Feb 22 2006 1:36 AM, Peg Smith wrote:
Gary Carson
http://www.garycarson.com
_______________________________________________________________
Block Lists, Favorites, and more - http://www.recpoker.com
James L. Hankins
"Peg Smith" <PegSm...@aol.comnojunk> wrote in message
news:0jfnv1hm7v01fgb7d...@4ax.com...
Yep. This should be good.
Ron Dworkin
tictoe
> Data Mining and Analysis for Poker Players
> Chapter 1 Data
The only CS course I did badly in was Simulations. I thought it was
easy. Write a program to simulate a cement factory with trucks and
drivers. Not only was my program incorrect, I had no idea it was
wrong until after I got it back from the instructor. The answers it
printed out looked reasonable to me. I didn't finish the class. I'm not
much into humiliation. :)
I planned on taking a full year of statistics. The first day of class I
get an instructor with a set of fuzzy dice. I didn't think his lecture
was well organized and I need a pretty structured environment for
math classes... I ended up dropping it and taking Probabilty
instead. Tough class (Calculus hurts my brane), but I did make it
through. My brain much prefers things like graph theory and
programming data structures type classes.
At the moment I am refiguring out C programming (for fun, I needed
something besides poker to keep my head from rotting and I finally
have a practical project to do yay!). I'm writing a session data
gathering program for hand histories (yes, I know there are programs
like Poker Tracker, but I have the time to fiddle around myself, and
it's fun :) )
So this sort of topic/class should be pretty helpful to me. Thank you
Gary very much for putting it out there for me to read and follow along.
tictoe