Three common problems on supervised learning
1. Q:What is the difference between Linear Regression and Ridge Regression?
A: They are almost identical. Linear Regression uses Ordinary least squares (OLS) to get an unbiased and high variance solution. Things like multi-collinearity can cause Linear Regression to fail. Ridge Regression is solved pretty much the same way, but it adds a regularization constant. The constant is a source of bias and can decrease variance. A larger constant will inject more bias. We typically recommend using Ridge Regression with a small constant.
Ridge Regression is an adaptation of the popular and widely used linear regression algorithm. It enhances regular linear regression by slightly changing its cost function, which results in less overfit models. In this article, you will learn everything you need to know about Ridge Regression, and how you can start using it in your own machine learning projects.
2. Q:When performing regression, what is the difference between using Linear Regression and using a GLM?
A: GLMs are set up to handle a broader class of problems and do not make all of the same assumptions as Linear Regression. Without these assumptions they can’t be solved the same way. A GLM uses numerical methods to find Maximum Likelihood Estimators. Although different methods are used to get the solution, the model can still be interpreted the same way.
Ordinary linear regression is applied if the outcome is a continuous variable and is normally distributed. However, there are cases where these two assumptions do not hold true. In those situations a suite of Generalized Linear Models is applied. A GLM has three elements: random, systematic and link function which need to be specified in each model implementation.
3. Q:If Neural Networks are a universal learner and can be used to model just about anything, then why do we need other models? Why don’t we just focus on Neural Networks and forget the other models?
A: Neural Networks can be used to model just about anything… But that does not mean they are the best choice for every situation. They might require more data than is available. They might take too long to train and validate. They might take too long to score. You might need a model that can be interpreted.
Each type of model has strengths and weaknesses so there’s no free lunch in model selection.
Extended information about this question:
