Nonlinearities

Nonlinearities

Nonlinearities

Nonlinearities

Logarithmic Models

The Natural Logarithm

The Natural Logarithm: Review I

• Exponents are defined as bn=b×b×⋯×b⏟n times
  • where base b is multiplied by itself n times

• Example: 23=2×2×2⏟n=3=8

• Logarithms are the inverse, defined as the exponents in the expressions above If bn=y, then logb(y)=n

  • n is the number you must raise b to in order to get y

• Example: log2(8)=3

The Natural Logarithm: Review II

• Logarithms can have any base, but common to use the natural logarithm (ln) with base e=2.71828... If en=y, then ln(y)=n

The Natural Logarithm: Properties

• Natural logs have a lot of useful properties:

1. ln(1x)=−ln(x)

2. ln(ab)=ln(a)+ln(b)

3. ln(xa)=ln(x)−ln(a)

4. ln(xa)=aln(x)

5. dlnxdx=1x

The Natural Logarithm: Example

• Most useful property: for small change in x, Δx:

ln(x+Δx)−ln(x)⏟Difference in logs≈Δxx⏟Relative change

• This allows us to very easily interpret coefficients as percent changes or elasticities

Elasticity

• An elasticity between two variables, EY,X describes the responsiveness of one variable to a change in another

• Measured in percentages: a 1% change in X will cause a E% change in Y

EY,X=%ΔY%ΔX=(ΔYY)(ΔXX)

• Numerator is relative change in Y, Denominator is relative change in X

Math FYI: Cobb Douglas Functions and Logs

• One of the (many) reasons why economists love Cobb-Douglas functions: Y=ALαKβ

• Taking logs, relationship becomes linear:

ln(Y)=ln(A)+αln(L)+βln(K)

• With data on (Y,L,K) and linear regression, can estimate α and β
  • α: elasticity of Y with respect to L
    • A 1% change in L will lead to an α% change in Y
  • β: elasticity of Y with respect to K
    • A 1% change in K will lead to a β% change in Y

Logarithms in R I

• The log() function can easily take the logarithm

gapminder<-gapminder %>%
  mutate(loggdp=log(gdpPercap)) # log GDP per capita
gapminder %>% head() # look at it

Logarithms in R II

• Note, log() by default is the natural logarithm ln(), i.e. base e
  • Can change base with e.g. log(x, base = 5)
  • Some common built-in logs: log10, log2

log10(100)

## [1] 2

log2(16)

## [1] 4

log(19683, base=3)

## [1] 9

Types of Logarithmic Models

• Three types of log regression models, depending on which variables we log

1. Linear-log model: Yi=β0+β1 ln(Xi)

2. Log-linear model: ln(Yi) =β0+β1Xi

3. Log-log model: ln(Yi) =β0+β1 ln(Xi)

Linear-Log Model

• Linear-log model has an independent variable (X) that is logged

Y=β0+β1ln(X)β1=ΔY(ΔXX)

• Marginal effect of X→Y: a 1% change in X→ a β1100 unit change in Y

Linear-Log Model in R

lin_log_reg<-lm(lifeExp~loggdp, data = gapminder)
library(broom)
tidy(lin_log_reg)

Linear-Log Model in R

lin_log_reg<-lm(lifeExp~loggdp, data = gapminder)
library(broom)
tidy(lin_log_reg)

Linear-Log Model in R

lin_log_reg<-lm(lifeExp~loggdp, data = gapminder)
library(broom)
tidy(lin_log_reg)

Linear-Log Model in R

lin_log_reg<-lm(lifeExp~loggdp, data = gapminder)
library(broom)
tidy(lin_log_reg)

Linear-Log Model Graph I

ggplot(data = gapminder)+
  aes(x = gdpPercap,
      y = lifeExp)+
  geom_point(color="blue", alpha=0.5)+
  geom_smooth(method="lm",
              formula=y~log(x),
              color="orange")+
  scale_x_continuous(labels=scales::dollar,
                     breaks=seq(0,120000,20000))+
  scale_y_continuous(breaks=seq(0,90,10),
                     limits=c(0,90))+
  labs(x = "GDP per Capita",
       y = "Life Expectancy (Years)")+
  theme_classic(base_family = "Fira Sans Condensed",
           base_size=20)

Linear-Log Model Graph II

ggplot(data = gapminder)+
  aes(x = loggdp,
      y = lifeExp)+ 
  geom_point(color="blue", alpha=0.5)+
  geom_smooth(method="lm", color="orange")+
  scale_y_continuous(breaks=seq(0,90,10),
                     limits=c(0,90))+
  labs(x = "Log GDP per Capita",
       y = "Life Expectancy (Years)")+
  theme_classic(base_family = "Fira Sans Condensed",
           base_size=20)

Log-Linear Model

• Log-linear model has the dependent variable (Y) logged

ln(Y)=β0+β1Xβ1=(ΔYY)ΔX

• Marginal effect of X→Y: a 1 unit change in X→ a β1×100 % change in Y

Log-Linear Model in R (Preliminaries)

• We will again have very large/small coefficients if we deal with GDP directly, again let's transform gdpPercap into $1,000s, call it gdp_t

• Then log LifeExp

gapminder <- gapminder %>%
  mutate(gdp_t=gdpPercap/1000, # first make GDP/capita in $1000s
         loglife=log(lifeExp)) # take the log of LifeExp
gapminder %>% head() # look at it

Log-Linear Model in R

log_lin_reg<-lm(loglife~gdp_t, data = gapminder)
tidy(log_lin_reg)

Log-Linear Model in R

log_lin_reg<-lm(loglife~gdp_t, data = gapminder)
tidy(log_lin_reg)

Log-Linear Model in R

log_lin_reg<-lm(loglife~gdp_t, data = gapminder)
tidy(log_lin_reg)

Log-Linear Model in R

log_lin_reg<-lm(loglife~gdp_t, data = gapminder)
tidy(log_lin_reg)

Linear-Log Model Graph I

ggplot(data = gapminder)+
  aes(x = gdp_t,
      y = loglife)+
  geom_point(color="blue", alpha=0.5)+
  geom_smooth(method="lm", color="orange")+
  scale_x_continuous(labels=scales::dollar,
                     breaks=seq(0,120,20))+
  labs(x = "GDP per Capita ($ Thousands)",
       y = "Log Life Expectancy")+
  theme_classic(base_family = "Fira Sans Condensed",
           base_size=20)

Log-Log Model

• Log-log model has both variables (X and Y) logged

ln(Y)=β0+β1ln(X)β1=(ΔYY)(ΔXX)

• Marginal effect of X→Y: a 1% change in X→ a β1 % change in Y

• β1 is the elasticity of Y with respect to X!

Log-Log Model in R

log_log_reg<-lm(loglife~loggdp, data = gapminder)
tidy(log_log_reg)

Log-Log Model in R

log_log_reg<-lm(loglife~loggdp, data = gapminder)
tidy(log_log_reg)

Log-Log Model in R

log_log_reg<-lm(loglife~loggdp, data = gapminder)
tidy(log_log_reg)

Log-Log Model in R

log_log_reg<-lm(loglife~loggdp, data = gapminder)
tidy(log_log_reg)

Log-Log Model Graph I

ggplot(data = gapminder)+
  aes(x = loggdp,
      y = loglife)+
  geom_point(color="blue", alpha=0.5)+
  geom_smooth(method="lm", color="orange")+
  labs(x = "Log GDP per Capita",
       y = "Log Life Expectancy")+
  theme_classic(base_family = "Fira Sans Condensed",
           base_size=20)

Comparing Models I

• Hint: the variable that gets logged changes in percent terms, the variable not logged changes in unit terms

Comparing Models II

library(huxtable)
huxreg("Life Exp." = lin_log_reg,
       "Log Life Exp." = log_lin_reg,
       "Log Life Exp." = log_log_reg,
       coefs = c("Constant" = "(Intercept)",
                 "GDP ($1000s)" = "gdp_t",
                 "Log GDP" = "loggdp"),
       statistics = c("N" = "nobs",
                      "R-Squared" = "r.squared",
                      "SER" = "sigma"),
       number_format = 2)

Comparing Models III

Linear-Log	Log-Linear	Log-Log
^Yi=^β0+^β1ln(Xi)	ln(^Yi)=^β0+^β1Xi	ln(^Yi)=^β0+^β1ln(Xi)
R2=0.65	R2=0.30	R2=0.61

When to Log?

• In practice, the following types of variables are logged:

  • Variables that must always be positive (prices, sales, market values)
  • Very large numbers (population, GDP)
  • Variables we want to talk about as percentage changes or growth rates (money supply, population, GDP)
  • Variables that have diminishing returns (output, utility)
  • Variables that have nonlinear scatterplots

• Avoid logs for:

  • Variables that are less than one, decimals, 0, or negative
  • Categorical variables (season, gender, political party)
  • Time variables (year, week, day)

Comparing Coefficients of Different Units I

^Yi=β0+β1X1+β2X2

• We often want to compare coefficients to see which variable X1 or X2 has a bigger effect on Y

• What if X1 and X2 are different units?

Comparing Coefficients of Different Units II

• An easy way is to standardize the variables (i.e. take the Z-score)

Xstd=X−¯Xsd(X)

Comparing Coefficients of Different Units: Example

^Salaryi=−2,869,439.40+12,417,629.72Batting averagei+129,627.36Home runsi^Salaryistd=0.00+0.14Batting averagestdi+0.48Home runsstdi

• Marginal effect on Y (in standard deviations of Y) from 1 standard deviation change in X
  • ^β1: a 1 standard deviation increase in Batting Average increases Salary by 0.14 standard deviations
  • $0.14 \times 2,764,512=\$387,032$
  • ^β2: a 1 standard deviation increase in Home Runs increases Salary by 0.48 standard deviations
  • $0.48 \times 2,764,512=\$1,326,966$

Standardizing in R

• Use the scale() command inside mutate() function to standardize a variable

gapminder<-gapminder %>%
  mutate(std_life = scale(lifeExp),
         std_gdp = scale(gdpPercap)) 
std_reg<-lm(std_life~std_gdp, data = gapminder)
tidy(std_reg)

Joint Hypothesis Testing I

• Maybe region doesn't affect wages at all?

• H0:β2=0,β3=0,β4=0

• This is a joint hypothesis to test

Joint Hypothesis Testing II

• A joint hypothesis tests against the null hypothesis of a value for multiple parameters: H0:β1=β2=0

  the hypotheses that multiple regressors are equal to zero (have no causal effect on the outcome)

• Our alternative hypothesis is that: H1: either β1≠0 or β2≠0 or both

  or simply, that H0 is not true

Types of Joint Hypothesis Tests

• Three main cases of joint hypothesis tests:

1. H0: β1=β2=0

   • Testing against the claim that multiple variables don't matter
   • Useful under high multicollinearity between variables
   • Ha: at least one parameter ≠ 0

2. H0: β1=β2

   • Testing whether two variables matter the same
   • Variables must be the same units
   • Ha:β1(≠,<, or >)β2

3. H0: ALL β's =0

   • The "Overall F-test"
   • Testing against claim that regression model explains NO variation in Y

Joint Hypothesis Tests: F-statistic

• The F-statistic is the test-statistic used to test joint hypotheses about regression coefficients with an F-test

• This involves comparing two models:

  1. Unrestricted model: regression with all coefficients
  2. Restricted model: regression under null hypothesis (coefficients equal hypothesized values)

• F is an analysis of variance (ANOVA)

  • essentially tests whether R2 increases statistically significantly as we go from the restricted model$\rightarrow$unrestricted model

• F has its own distribution, with two sets of degrees of freedom

Joint Hypothesis F-test: Example I

• H0:β2=β3=β4=0

• Ha: H0 is not true (at least one βi≠0)

Joint Hypothesis F-test: Example II

• Unrestricted model:

^Wagei=^β0+^β1Malei+^β2Northeasti+^β3Midwesti+^β4Southi

• Restricted model:

^Wagei=^β0+^β1Malei

• F: does going from restricted to unrestricted statistically significantly improve R2?

Calculating the F-statistic

Calculating the F-statistic

Calculating the F-statistic