Dummy Variable Regression Model Pdf

[Pernilla Gendron]

Inregression analysis, a dummy variable (also known as indicator variable or just dummy) is one that takes a binary value (0 or 1) to indicate the absence or presence of some categorical effect that may be expected to shift the outcome.[1] For example, if we were studying the relationship between biological sex and income, we could use a dummy variable to represent the sex of each individual in the study. The variable could take on a value of 1 for males and 0 for females (or vice versa). In machine learning this is known as one-hot encoding.

Dummy variables are commonly used in regression analysis to represent categorical variables that have more than two levels, such as education level or occupation. In this case, multiple dummy variables would be created to represent each level of the variable, and only one dummy variable would take on a value of 1 for each observation. Dummy variables are useful because they allow us to include categorical variables in our analysis, which would otherwise be difficult to include due to their non-numeric nature. They can also help us to control for confounding factors and improve the validity of our results.

As with any addition of variables to a model, the addition of dummy variables will increase the within-sample model fit (coefficient of determination), but at a cost of fewer degrees of freedom and loss of generality of the model (out of sample model fit). Too many dummy variables result in a model that does not provide any general conclusions.

Dummy variables are useful in various cases. For example, in econometric time series analysis, dummy variables may be used to indicate the occurrence of wars, or major strikes. It could thus be thought of as a Boolean, i.e., a truth value represented as the numerical value 0 or 1 (as is sometimes done in computer programming).

Dummy variables may be extended to more complex cases. For example, seasonal effects may be captured by creating dummy variables for each of the seasons: D1=1 if the observation is for summer, and equals zero otherwise; D2=1 if and only if autumn, otherwise equals zero; D3=1 if and only if winter, otherwise equals zero; and D4=1 if and only if spring, otherwise equals zero. In the panel data fixed effects estimator dummies are created for each of the units in cross-sectional data (e.g. firms or countries) or periods in a pooled time-series. However in such regressions either the constant term has to be removed, or one of the dummies removed making this the base category against which the others are assessed, for the following reason:

If dummy variables for all categories were included, their sum would equal 1 for all observations, which is identical to and hence perfectly correlated with the vector-of-ones variable whose coefficient is the constant term; if the vector-of-ones variable were also present, this would result in perfect multicollinearity,[2] so that the matrix inversion in the estimation algorithm would be impossible. This is referred to as the dummy variable trap.

We want to perform linear regression of the police confidence score against sex, which is a binary categorical variable with two possible values (which we can see are 1= Male and 2= Female if we check the Values cell in the sex row in Variable View ). However, before we begin our linear regression, we need to recode the values of Male and Female. Why must we do this?

The codes 1 and 2 are assigned to each gender simply to represent which distinct place each category occupies in the variable sex . However, linear regression assumes that the numerical amounts in all independent, or explanatory, variables are meaningful data points. So, if we were to enter the variable sex into a linear regression model, the coded values of the two gender categories would be interpreted as the numerical values of each category. This would provide us with results that would not make sense, because for example, the sex Female does not have a value of 2.

A dummy variable is a variable created to assign numerical value to levels of categorical variables. Each dummy variable represents one category of the explanatory variable and is coded with 1 if the case falls in that category and with 0 if not. For example, in the dummy variable for Female, all cases in which the respondent is female are coded as 1 and all other cases, in which the respondent is Male, are coded as 0. This allows us to enter in the sex values as numerical. (Remember, these numbers are just indicators.)

Our sample of data has shown us that, on average, female respondents reported a police confidence score that is .436 points lower than male respondents. We want to know if this is a statistically significant effect in the population from which the sample was taken. To do this, we carry out a hypothesis test to determine whether or not b (the coefficient for females) is different from zero in the population. If the coefficient could be zero, then there is no statistically significant difference between males and females.

SPSS calculates a t statistic and a corresponding p-value for each of the coefficients in the model. These can be seen in the Coefficients output table. A t statistic is a measure of how likely it is that the coefficient is not equal to zero. It is calculated by dividing the coefficient by the standard error . If the standard error is small relative to the coefficient (making the t statistic relatively large), the coefficient is likely to differ from zero in the population.

The p-value is in the column labelled Sig . As in all hypothesis tests, if the p-value is less than 0.05, then the variable is significant at the 5% level. That is, we would have evidence to reject the null and conclude that b is different from zero.

In this example, t = -10.417 with a corresponding p-value of 0.000. This means that the chances of the difference between males and females that we have calculated is actually happening due to chance is very small indeed. Therefore, we have evidence to conclude that sex1 is a significant predictor of policeconf1 in the population.

A dummy variable is a binary variable that takes a value of 0 or 1. One adds such variables to a regression model to represent factors which are of a binary nature i.e. they are either observed or not observed.

The last four use-cases, namely the use of dummies to deseasonalize data, to represent fixed effects and treatment effects, and for modeling regression discontinuities all deserve their own separate chapters.

In the above data set, the aspiration variable is of type Standard or Turbo. Our regression goal is to estimate the effect of aspiration on vehicle price. To that end, we will introduce a dummy variable to encode aspiration as follows:

Notice that we have added only one dummy variable aspiration_std and not both, aspiration_std and aspiration_turbo. We did this to avoid perfect collinearity as every vehicle engine in the data set is either of type turbo or of type standard. There is no third type. In this case, regression intercept captures the effect of aspiration_turbo. Specifically, the estimated value of the regression intercept in the trained model is the estimated mean price of all turbo type automobiles.

Alternately, we could have added both aspiration_std and aspiration_turbo and left out the regression intercept. In this later case, because the model would not have the regression intercept, we would not be able to use the R-squared value to judge its goodness-of-fit.

The coefficients of the four style-specific dummies for hardtop, hatchback, sedan and wagon represent the extent to which the mean of the corresponding style deviates from the estimated mean price of convertibles.

In summary, our model has shown that on average, convertibles are the most expensive vehicle followed by sedans, wagons and hatchbacks in that order, and nothing useful can be said about the hardtop style in its ability to explain the variance in price.

We see that this model will estimate a constant expected change in price for each unit change in the number of cylinders. The model will estimate the difference in the mean price of 2 cylinder vehicles and 3 cylinder vehicles to be exactly the same as that between 3 and 4 cylinder vehicles and so on. In the real world we would not expect to see such a uniform variation in vehicle prices.

We have left out the dummy for num_of_cylinders_2. The intercept β_0 will capture the coefficient for num_of_cylinders_2. The coefficients of all dummy variables will contain the estimated deviation in the mean price for the respective category of vehicles from the estimated mean price of 2-cylinder vehicles. The 95% CIs can be calculated as illustrated above.

The first thing that catches the eye in the summary is the large adjusted R-squared of 0.618. The num_of_cylinders appears to have the capacity to by itself explain a whopping 61.8% of the variance in automobile prices.

The estimated means of 5, 6, 8 and 12-cylinder automobiles are all highly significant. 8-cylinder automobiles seem to be on-average the most expensive ones of the lot with their estimated mean price coming in at a colossal $25,880 more than their 2-cylinder brethren.

We see that the price does not change by a constant amount with each unit change in the number of cylinders. This vindicates the insight we had earlier that we ought not to represent num_of_cylinders as a simple integer-valued variable.

Thus, when we have an intercept in the regression model and we want to avoid perfect multicollinearity, we create only one dummy to encode a categorical variable that has two categories.

Newbie question here: I'm doing dummy variable regression using JMP for a school project. When I cross-referenced to Excel's version, I realized the coefficients are completely different even though the t-stats are the same.

3a8082e126