Multinomial Logistic Regression
This section builds upon the concepts discussed in the Logistic Regression section. In this section we will focus on unordered choices.
Random Utility Basis
For the i t h i^{th} i t h J J J j j j
U i j = w ′ i β j + z ′ i j γ j + ε i j U_{ij} = \bold{w'}_i \boldsymbol{\beta}_j + \bold{z'}_{ij}\boldsymbol{\gamma}_j + \varepsilon_{ij} U ij = w ′ i β j + z ′ ij γ j + ε ij Variables are defined as follows:
U i j : = U_{ij}:= U ij := i i i j j j w i ′ : = w'_i:= w i ′ := i ′ s i's i ′ s z i j ′ : = z'_{ij}:= z ij ′ := j j j
If the consumer chooses j j j
P [ U i j > U i k ] , ∀ k ≠ j . P[U_{ij}>U_{ik}], \hspace{10px} \forall k\neq j. P [ U ij > U ik ] , ∀ k = j . Assume there are 4 choices { A , B , C , D } \{A,B,C,D\} { A , B , C , D } i t h i^{th} i t h
U i A = w ′ i β A + z ′ i A γ A ⏟ V i A + ε i A = V i A + ε i A U i B = w ′ i β B + z ′ i B γ B + ε i B = V i B + ε i B U i C = w ′ i β C + z ′ i C γ C + ε i C = V i C + ε i C U i D = w ′ i β D + z ′ i D γ D + ε i D = V i D + ε i D \begin{align*}
U_{iA} &= \underbrace{\bold{w'}_i \boldsymbol{\beta}_A + \bold{z'}_{iA}\boldsymbol{\gamma}_A}_{\bold{V}_{iA}} + \varepsilon_{iA}=\bold{V}_{iA}+ \varepsilon_{iA}\\
U_{iB} &= \bold{w'}_i \boldsymbol{\beta}_B + \bold{z'}_{iB}\boldsymbol{\gamma}_B + \varepsilon_{iB}=\bold{V}_{iB}+ \varepsilon_{iB}\\
U_{iC} &= \bold{w'}_i \boldsymbol{\beta}_C + \bold{z'}_{iC}\boldsymbol{\gamma}_C + \varepsilon_{iC}=\bold{V}_{iC}+ \varepsilon_{iC}\\
U_{iD} &= \bold{w'}_i \boldsymbol{\beta}_D + \bold{z'}_{iD}\boldsymbol{\gamma}_D + \varepsilon_{iD}=\bold{V}_{iD}+ \varepsilon_{iD}
\end{align*} U i A U i B U i C U i D = V i A w ′ i β A + z ′ i A γ A + ε i A = V i A + ε i A = w ′ i β B + z ′ i B γ B + ε i B = V i B + ε i B = w ′ i β C + z ′ i C γ C + ε i C = V i C + ε i C = w ′ i β D + z ′ i D γ D + ε i D = V i D + ε i D If i t h i^{th} i t h C C C
U i C > U i A and U i C > U i B and U i C > U i D . U_{iC}>U_{iA} \text{ and } U_{iC}>U_{iB} \text{ and } U_{iC}>U_{iD}. U i C > U i A and U i C > U i B and U i C > U i D . This implies,
P [ y i = C ] = P i C = P [ U i C > U i j , ∀ j ≠ C ] = P [ V i C + ε i C > V i j + ε i j , ∀ j ≠ C ] = P [ ε i j < V i C + ε i C − V i j , ∀ j ≠ C ] \begin{align*}
P[y_i=C]=P_{iC}&=P[U_{iC}>U_{ij},\hspace{7px}\forall j \neq C]\\
&=P[\bold{V}_{iC}+ \varepsilon_{iC}>\bold{V}_{ij}+ \varepsilon_{ij},\hspace{7px}\forall j \neq C]\\
&=P[\varepsilon_{ij}<\bold{V}_{iC}+ \varepsilon_{iC} -\bold{V}_{ij},\hspace{7px}\forall j \neq C]
\end{align*} P [ y i = C ] = P i C = P [ U i C > U ij , ∀ j = C ] = P [ V i C + ε i C > V ij + ε ij , ∀ j = C ] = P [ ε ij < V i C + ε i C − V ij , ∀ j = C ] Assume that ε i C \varepsilon_{iC} ε i C
P i C ∣ ε i C = P [ ε i j < V i C + ε i C − V i j ∣ ε i C , ∀ j ≠ C ] \begin{align*}
P_{iC}|\varepsilon_{iC}&=P[\varepsilon_{ij}<\bold{V}_{iC}+ \varepsilon_{iC} -\bold{V}_{ij}|\varepsilon_{iC},\hspace{7px}\forall j \neq C]
\end{align*} P i C ∣ ε i C = P [ ε ij < V i C + ε i C − V ij ∣ ε i C , ∀ j = C ] Since ε i ′ s \varepsilon_{i}'s ε i ′ s
P i C ∣ ε i C = P [ ε i A < V i C + ε i C − V i A ∣ ε i C ] ⋅ P [ ε i B < V i C + ε i C − V i B ∣ ε i C ] ⋅ P [ ε i D < V i C + ε i C − V i D ∣ ε i C ] . \begin{align*}
P_{iC}|\varepsilon_{iC}&=P[\varepsilon_{iA}<\bold{V}_{iC}+ \varepsilon_{iC} -\bold{V}_{iA}|\varepsilon_{iC}]\cdot P[\varepsilon_{iB}<\bold{V}_{iC}+ \varepsilon_{iC} -\bold{V}_{iB}|\varepsilon_{iC}]\cdot P[\varepsilon_{iD}<\bold{V}_{iC}+ \varepsilon_{iC} -\bold{V}_{iD}|\varepsilon_{iC}].
\end{align*} P i C ∣ ε i C = P [ ε i A < V i C + ε i C − V i A ∣ ε i C ] ⋅ P [ ε i B < V i C + ε i C − V i B ∣ ε i C ] ⋅ P [ ε i D < V i C + ε i C − V i D ∣ ε i C ] . We have assumed that each ε i j \varepsilon_{ij} ε ij
f ( ε i j ) = e − ε i j ⋅ e − e − ε i j , F ( ε i j ) = e − e − ε i j . \begin{align*}
f(\varepsilon_{ij})&=e^{-\varepsilon_{ij}}\cdot e^{-e^{-\varepsilon_{ij}}},\\
F(\varepsilon_{ij})&=e^{-e^{-\varepsilon_{ij}}}.
\end{align*} f ( ε ij ) F ( ε ij ) = e − ε ij ⋅ e − e − ε ij , = e − e − ε ij . Therefore
P i C ∣ ε i C = e − e − ( V i C + ε i C − V i A ) ⋅ e − e − ( V i C + ε i C − V i B ) ⋅ e − e − ( V i C + ε i C − V i D ) = ∏ j ≠ C e − e − ( V i C + ε i C − V i j ) . \begin{align*}
P_{iC}|\varepsilon_{iC}&=e^{-e^{-(\bold{V}_{iC}+ \varepsilon_{iC} -\bold{V}_{iA})}}\cdot e^{-e^{-(\bold{V}_{iC}+ \varepsilon_{iC} -\bold{V}_{iB})}} \cdot e^{-e^{-(\bold{V}_{iC}+ \varepsilon_{iC} -\bold{V}_{iD})}}\\
&=\prod_{j\neq C}e^{-e^{-(\bold{V}_{iC}+ \varepsilon_{iC} -\bold{V}_{ij})}}.
\end{align*} P i C ∣ ε i C = e − e − ( V i C + ε i C − V i A ) ⋅ e − e − ( V i C + ε i C − V i B ) ⋅ e − e − ( V i C + ε i C − V i D ) = j = C ∏ e − e − ( V i C + ε i C − V ij ) . Using the law of total probability
P i C = ∫ P i C ∣ ε i C ⋅ f ( ε i C ) d ε i C = ∫ ( ∏ j ≠ C e − e − ( V i C + ε i C − V i j ) ) ⋅ f ( ε i C ) d ε i C = ∫ ( ∏ j ≠ C e − e − ( V i C + ε i C − V i j ) ) ⋅ e − ε i C ⋅ e − e − ε i C d ε i C = e V i C ∑ j e V i j = exp ( V i C ) ∑ j exp ( V i j ) = exp ( w ′ i β C + z ′ i C γ C ) exp ( w ′ i β A + z ′ i A γ A ) + exp ( w ′ i β B + z ′ i B γ B ) + exp ( w ′ i β C + z ′ i C γ C ) + exp ( w ′ i β D + z ′ i D γ D ) \begin{align*}
P_{iC}&=\int P_{iC}|\varepsilon_{iC}\cdot f(\varepsilon_{iC})d\varepsilon_{iC}\\
&=\int\Big(\prod_{j\neq C}e^{-e^{-(\bold{V}_{iC}+ \varepsilon_{iC} -\bold{V}_{ij})}}\Big)\cdot f(\varepsilon_{iC})d\varepsilon_{iC}\\
&=\int\Big(\prod_{j\neq C}e^{-e^{-(\bold{V}_{iC}+ \varepsilon_{iC} -\bold{V}_{ij})}}\Big)\cdot e^{-\varepsilon_{iC}}\cdot e^{-e^{-\varepsilon_{iC}}}d\varepsilon_{iC}\\
&=\frac{e^{\bold{V}_{iC}}}{\sum_je^{\bold{V}_{ij}}}=\frac{\text{exp}(\bold{V}_{iC})}{\sum_j \text{exp}(\bold{V}_{ij})}\\
&=\frac{\text{exp}(\bold{w'}_i \boldsymbol{\beta}_C + \bold{z'}_{iC}\boldsymbol{\gamma}_C)}{\text{exp}(\bold{w'}_i \boldsymbol{\beta}_A + \bold{z'}_{iA}\boldsymbol{\gamma}_A) + \text{exp}(\bold{w'}_i \boldsymbol{\beta}_B + \bold{z'}_{iB}\boldsymbol{\gamma}_B) + \text{exp}(\bold{w'}_i \boldsymbol{\beta}_C + \bold{z'}_{iC}\boldsymbol{\gamma}_C) + \text{exp}(\bold{w'}_i \boldsymbol{\beta}_D + \bold{z'}_{iD}\boldsymbol{\gamma}_D)}
\end{align*} P i C = ∫ P i C ∣ ε i C ⋅ f ( ε i C ) d ε i C = ∫ ( j = C ∏ e − e − ( V i C + ε i C − V ij ) ) ⋅ f ( ε i C ) d ε i C = ∫ ( j = C ∏ e − e − ( V i C + ε i C − V ij ) ) ⋅ e − ε i C ⋅ e − e − ε i C d ε i C = ∑ j e V ij e V i C = ∑ j exp ( V ij ) exp ( V i C ) = exp ( w ′ i β A + z ′ i A γ A ) + exp ( w ′ i β B + z ′ i B γ B ) + exp ( w ′ i β C + z ′ i C γ C ) + exp ( w ′ i β D + z ′ i D γ D ) exp ( w ′ i β C + z ′ i C γ C )