Linear Regression using matrix method.

Defining matrix of Predictor Variable, X,

X <- matrix(c(3,4,5),nrow = 3)
X
##      [,1]
## [1,]    3
## [2,]    4
## [3,]    5

Defining matrix of Outcome Variable, Y

Y <- matrix(c(5,9,13),nrow = 3)
Y
##      [,1]
## [1,]    5
## [2,]    9
## [3,]   13

Defining X matrix with an extra column of 1’s

one_col <- rep(1,3)
X_1 <- matrix(rbind(one_col,c(3,4,5)),nrow = 3,byrow = TRUE)
X_1
##      [,1] [,2]
## [1,]    1    3
## [2,]    1    4
## [3,]    1    5

Calculating Beta matrix with the equation - B=((X’X)^-1)(X’Y)

B<-solve(t(X_1)%*%(X_1))%*%t(X_1)%*%Y
B
##      [,1]
## [1,]   -7
## [2,]    4

Coefficients for linear regression, Slope-B1, Intercept-B0

B0 = B[1]
B1 = B[2]
B0
## [1] -7
B1
## [1] 4

Finding residuals of the linear regression model, Res = Y-(B1*X+B0) which equals zero.

Y_Pred <-(B1*X+B0)
Res = Y-Y_Pred
Res
##               [,1]
## [1,] -1.776357e-14
## [2,] -1.776357e-14
## [3,] -1.776357e-14

Linear Regression using lm() function

l_reg <- lm(formula = Y ~ X)
summary(l_reg)
## Warning in summary.lm(l_reg): essentially perfect fit: summary may be unreliable
## 
## Call:
## lm(formula = Y ~ X)
## 
## Residuals:
## 1 2 3 
## 0 0 0 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       -7          0    -Inf   <2e-16 ***
## X                  4          0     Inf   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0 on 1 degrees of freedom
## Multiple R-squared:      1,  Adjusted R-squared:      1 
## F-statistic:   Inf on 1 and 1 DF,  p-value: < 2.2e-16

From the summary, R square value is 1 and residual error 0 which implies that model is a perfect fit but highly dependent on perdictor variable and is unrealistic.

Plotting X,Y, predicted values and linear model.