Desging and naming convention: contrast(s), restriction, constraint, constraints, constrained, restricted

[josef...@gmail.com]

I'm trying to collect some tools for contrast(s), restriction, constraint, constraints, transformations and try to figure out how we can make and name this consistently across our usage of it.

For now it is mostly linear/affine constraints of the form

R dot params = q

Our original usage in Wald tests, t_test and f_test just used `r_matrix` and `q_matrix`

which got shortened to just a r_matrix that can be an R array, and (R, q) tuple or list of strings.

The wald test return a `stats.contrast.ContrastResults` which we inherited from the pre-statsmodels version.

For estimation subject to linear restrictions:

GEE uses a 'constraint' (singular) keyword

GLM and Poisson fit_constrained uses `constraints` with the same meaning as r_matrix in Wald tests.

Penalized estimation allows for stochastic linear constraints, i.e. penalized `R params - q` with currently an API that is up for changing

(TheilGLS, Ridge-like with stochastic restrictions, uses r_matrix, q_matrix like the initial t_test, Penalization classes use `restrictions`.)

In sandbox.stats we have an old helper module `contrast_tools` to build contrast/constraints matrices 

names:

`restrictions` is what I automatically use if I don't pay attention, might come from econometric background

`contrast` is traditional statistics, but most of the time narrowly defined as columns sum to zero (R.sum(1) == 0), and is mainly used in the context of categorical variables.

`constraints`, `constrained` is what I converged and decided on when I wrote fit_constrained, because it sounds pretty neutral

'transformation' This is similar to constraints, but where we are interested in the magnitude of the constraint, e.g. gamma = R params - q. And we want to know the results about gamma and not about params. 

(It's essentially the same thing just looking in a different way, e.g. one of my early work-around for missing get_prediction was to use results.t_test for getting confidence intervals for linear predicted mean.)

My preference is to converge to the `constraints`, `constrained` as consistent naming convention.

e.g. 

- add constraints_tools  (or constraint_tools ?  tools for constraints)

- rename r_matrix to constraints in t_test, wald_test   (is positional arg but not backwards compatible if some users use it as keyword)

- rename GEE constraint to plural constraints

- clean up API and naming in penalized

also

GEE docstring uses `constraint` as a tuple (L, R)  with constraint L *  param = R

so it we don't agree on one-letter names, compared to wald test (R, q)

(aside: I don't like one-letter names, but they are needed in docstrings)

In standalone tests like t- or z-tests I call the constant part `value`, e.g.

mean1 - mean0 = value 

(restriction, value) = (R, q) ?

another one: multivariate endog

`_multivariate_ols_test` for MANOVA uses 

L * params * M = C

where params is 2-D and L and M are column and row constraints

and there are no constraints yet for individual params.

... ???

Where I would like to go:

consistent and comprehensive support for linear constraints

- automatic constraints

The raw version for specifying constraints by users is to provide the R and q arrays directly.

t_test, wald_test use patsy to allow strings to specify constraints based on variable or param names.

anova_lm, MANOVA and wald_test_terms construct zero-restrictions for terms automatically

t_test_pairwise (added in 0.9) automatically creates all-pairs comparison constraints for categorical/factors.

the plan is to add more automatic constraint building both for hypothesis testing and for constrained estimation

e.g. other "contrasts" like sum to 1 or all equal to mean, sequential diff ,

GAM PR includes special constraints for splines.

- estimation subject to constraints

extend to other classes, currently supported in GEE, GLM and Poisson, NegativeBinomial

There is also an old PR for RestrictedLS.

- estimation with penalizing constraints

The basic idea is equivalent to a partial Bayesian prior for constraints, e.g.

R params - q ~ N(0, Sigma)

except we are not Bayesians and allow, for example, for SCAD penalized linear restrictions.

Partial support for this is now in master, GAM PR has more for L2 penalization.

- nonlinear restriction

TODO

There is currently some support for special cases of linear restriction in post-estimation, e.g. discrete Margins and GLM prediction, and some prototype code, but generic support for nonlinear constraints is still missing.

We also don't have any generic nonlinear models, we have special cases like genmod, discrete models based on link function (single index models), and some special cases of statespace models that are essentially non-linear (like Markov-switching).

Josef