Description
The regression model cannot solve (the design matrix cannot be inverted) in the presence of multicollinearity. Multicollinearity occurs when two or more variables are redundant. Each variable should be independent of any other variable. An effective model will have explanatory variables that each gets at a different facet of the dependent variable you are trying to predict or understand.
Solution
- Remove any redundant fields from the set of explanatory variables.
- Identify and remove any explanatory variables that have the same value for all features (for example, a field containing all zeros).
- Create a scatterplot matrix for your explanatory variables (or a sample of them), and assess whether there are near perfect correlations. If so, consider removing one of the corresponding variables from your model.