Jason's Blog

This example shows the performances of extremely randomized trees, k nearest neighbors, linear regression, and ridge regression in model estimation and prediction for face completion, i.e. predicting the lower half of a face given the upper half. The dataset is "fetch_olivetti_faces" coming from sklearn library. From scikit-learn library, we should import: extremely randomized trees: ensemble.ExtraTreeRegressor k nearest neighbors: neighbors.KNeighborsRegressor linear regression: linear_model.LinearRegression ridge regression: linear_model.RidgeCV The example code is as follows: # face completion print(__doc__) import numpy as np import matplotlib.pyplot as plt from sklearn.datasets import fetch_olivetti_faces from sklearn.utils.validation import check_random_state from sklearn.ensemble import ExtraTreesRegressor from sklearn.neighbors import KNeighborsRegressor from sklearn.linear_model import LinearRegression from sklearn.linear_model import RidgeCV ...

Ridge regression can be used to deal with the multicollinearity. In this example, I want to study on the factors having influences on the beef consumption, using the time series data including beef consumption, price of beef, pork, chicken, and fish from 1975 to 2015. The dependent variable is beef consumption, and independent variables are the real price of beef, pork, chicken, and fish, and CPI. We can use ridge option in PROC REG for ridge regression, by setting the value of ridge parameter. The results will be stored in the data file "outest=". proc reg data =beef outvif outest =b ridge = 0 to 0.05 by .005 ; model beef = year pricebeef_real pricepork_real pricebroilers_real pricefish_real cpi / vif lackfit dwprob spec ; run ; proc print data =b; run ; The first plot is the estimation results of linear regression. We can find that there exists severe multicollinearity, according to the values of VIF. The second plot shows that the VIF decl...