Where is my mistake in this definition of Bayes Factor?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty{ margin-bottom:0;
}
up vote
4
down vote
favorite
From "The Bayesian Choice" by Christian P. Robert.
The definition of the Bayes factor is given to be the ratio of the posterior probabilities of the null and the alternative hypothesis over the ratio of the prior probabilities of the null and alternative.
ie $$B_{01}^{pi}= frac{ frac{P(theta in Theta_{0}|x)}{P(theta in Theta_{1}|x)}}{frac{p(theta in Theta_{0})}{ p(theta in Theta_{1})}}$$
Which the author shortly simplifies to be
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
But when I try to write out all the terms and use that
$$pi(theta|x)=frac{f(x|theta) pi(theta)}{int_{Theta}f(x|theta) pi(theta)dtheta}$$
I get
$$B_{01}^{pi}=frac{f(x|theta_{0})int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}{f(x|theta_{1})int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}$$
instead.
Anyone have any idea of where I went wrong? Probably some simple mistake I made or error.
hypothesis-testing bayesian mathematical-statistics odds-ratio marginal
add a comment |
up vote
4
down vote
favorite
From "The Bayesian Choice" by Christian P. Robert.
The definition of the Bayes factor is given to be the ratio of the posterior probabilities of the null and the alternative hypothesis over the ratio of the prior probabilities of the null and alternative.
ie $$B_{01}^{pi}= frac{ frac{P(theta in Theta_{0}|x)}{P(theta in Theta_{1}|x)}}{frac{p(theta in Theta_{0})}{ p(theta in Theta_{1})}}$$
Which the author shortly simplifies to be
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
But when I try to write out all the terms and use that
$$pi(theta|x)=frac{f(x|theta) pi(theta)}{int_{Theta}f(x|theta) pi(theta)dtheta}$$
I get
$$B_{01}^{pi}=frac{f(x|theta_{0})int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}{f(x|theta_{1})int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}$$
instead.
Anyone have any idea of where I went wrong? Probably some simple mistake I made or error.
hypothesis-testing bayesian mathematical-statistics odds-ratio marginal
add a comment |
up vote
4
down vote
favorite
up vote
4
down vote
favorite
From "The Bayesian Choice" by Christian P. Robert.
The definition of the Bayes factor is given to be the ratio of the posterior probabilities of the null and the alternative hypothesis over the ratio of the prior probabilities of the null and alternative.
ie $$B_{01}^{pi}= frac{ frac{P(theta in Theta_{0}|x)}{P(theta in Theta_{1}|x)}}{frac{p(theta in Theta_{0})}{ p(theta in Theta_{1})}}$$
Which the author shortly simplifies to be
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
But when I try to write out all the terms and use that
$$pi(theta|x)=frac{f(x|theta) pi(theta)}{int_{Theta}f(x|theta) pi(theta)dtheta}$$
I get
$$B_{01}^{pi}=frac{f(x|theta_{0})int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}{f(x|theta_{1})int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}$$
instead.
Anyone have any idea of where I went wrong? Probably some simple mistake I made or error.
hypothesis-testing bayesian mathematical-statistics odds-ratio marginal
From "The Bayesian Choice" by Christian P. Robert.
The definition of the Bayes factor is given to be the ratio of the posterior probabilities of the null and the alternative hypothesis over the ratio of the prior probabilities of the null and alternative.
ie $$B_{01}^{pi}= frac{ frac{P(theta in Theta_{0}|x)}{P(theta in Theta_{1}|x)}}{frac{p(theta in Theta_{0})}{ p(theta in Theta_{1})}}$$
Which the author shortly simplifies to be
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
But when I try to write out all the terms and use that
$$pi(theta|x)=frac{f(x|theta) pi(theta)}{int_{Theta}f(x|theta) pi(theta)dtheta}$$
I get
$$B_{01}^{pi}=frac{f(x|theta_{0})int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}{f(x|theta_{1})int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}$$
instead.
Anyone have any idea of where I went wrong? Probably some simple mistake I made or error.
hypothesis-testing bayesian mathematical-statistics odds-ratio marginal
hypothesis-testing bayesian mathematical-statistics odds-ratio marginal
edited yesterday
Xi'an
52.2k688339
52.2k688339
asked yesterday
Quality
1929
1929
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
4
down vote
accepted
My deepest apologies, there is a typo in the final expression! Indeed, here is the complete text from my book (p.231):
Definition 5.5. The Bayes factor is the ratio of the posterior probabilities of the null and the alternative hypotheses over the
ratio of the prior probabilities of the null and the alternative
hypotheses, i.e., $$ B^pi_{01}(x) = {P(theta in Theta_ 0mid x)
over P(theta in Theta_1mid x)} bigg/ {pi(theta in Theta_ 0)
over pi(theta in Theta_ 1)}. $$ This ratio evaluates the
modification of the odds of $Theta_0$ against $Theta_1$ due to the
obs and can naturally be compared to $1$, although an exact
comparison scale can only be based upon a loss function. In the
particular case where $Theta_0={theta_0}$ and
$Theta_1={theta_1}$, the Bayes factor simplifies to the usual
likelihood ratio $$ B^pi_{01} (x) = {f(x|theta_0)over
f(x|theta_1)}. $$ In general, the Bayes factor depends on prior
information, but is still proposed as an ``objective'' Bayesian
answer, since it partly eliminates the influence of the prior modeling
and emphasizes the role of the observations. Actually, it can be
perceived as a Bayesian likelihood ratio since, if $pi_0$ is the
prior distribution under $H_0$ and $pi_1$ the prior distribution
under $H_1$, $B^pi_{01}(x)$ can be written as begin{equation}
B^pi_{01} (x) = {int_{Theta_0} f(x|theta_0)pi_0(theta)
,text{d}theta over int_{Theta_1} f(x|theta_1)pi_1(theta)
,text{d}theta}
=frac{m_0(x)}{m_1(x)},, end{equation} thus replacing the likelihoods with the marginals under both hypotheses.
The prior is thus defined as a mixture:
$$pi(theta)=pi(theta in Theta_ 0)timespi_0(theta)timesmathbb{I}_{Theta_0}(theta)+pi(theta in Theta_ 1)timespi_1(theta)timesmathbb{I}_{Theta_1}(theta)$$where
$$pi(theta in Theta_ 0)=rho_0qquadtext{and}qquadpi(theta in Theta_ 1)=1-rho_0stackrel{text{def}}{=}rho_1$$
are the prior weights of both hypotheses and
$$int_{Theta_0} pi_0(theta_0)text{d}theta_0=int_{Theta_1} pi_1(theta_1)text{d}theta_1=1$$Therefore
begin{align*}P(theta in Theta_ 0|x)=int_{Theta_0} pi(theta_0|x)text{d}theta_0&=int_{Theta_0} pi(theta_0|x)text{d}theta_0\
&= frac{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} pi(theta_1)f(x|theta_1)text{d}theta_1}\
&=frac{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}
end{align*}
and
$$P(theta in Theta_1mid x)=frac{int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}$$Hence,
$$frac{P(theta in Theta_ 0|x)}{P(theta in Theta_1mid x)}=frac{rho_0int_{Theta_0} pi_0(theta_0)f(x|theta_0)text{d}theta_0}{rho_1int_{Theta_1} pi_1(theta_1)f(x|theta_1)text{d}theta_1}=frac{rho_0}{1-rho_0}B^pi_{01}(x)$$
This hopefully explains where the final expression comes from.
Alas, there is a mistake in the last ratio of integrals in the quoted text (presumably due to a cut & paste)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
which should be (when identifying the integrands differently in the two integrals to signify that the integrals are over two different parameter spaces)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta_0)dtheta_0}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta_1)dtheta_1}$$or (with another choice of representation of the integrands)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta)pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta)pi_{1}(theta)dtheta}$$
The only case when the notations $theta_0$ and $theta_1$ are important is when both null and alternative hypotheses are point hypotheses, i.e., when $Theta_0={theta_0}$ and $Theta_1={theta_1}$, since both symbols then take specific values, like $theta_0=3$ and $theta_1=-2$. In this specific case, the posterior is concentrated on ${theta_0,theta_1}$ and the Bayes factor writes $$ B^pi_{01} (x) = {f(x|theta_0)over f(x|theta_1)}$$ since
$$P(theta=theta_0|x)=frac{overbrace{pi(theta_0)}^{rho_0}f(x|theta_0)}{pi(theta_0)f(x|theta_0)+underbrace{pi(theta_1)}_{rho_1}f(x|theta_1)}$$
Thank you for pointing out this error, to be added to the list of typos.
1
Are there a few $f$s missing, and an accidentally squared $pi_1$?
– Taylor
2 hours ago
@Taylor: yes indeed, well spotted. Thx.
– Xi'an
1 hour ago
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
4
down vote
accepted
My deepest apologies, there is a typo in the final expression! Indeed, here is the complete text from my book (p.231):
Definition 5.5. The Bayes factor is the ratio of the posterior probabilities of the null and the alternative hypotheses over the
ratio of the prior probabilities of the null and the alternative
hypotheses, i.e., $$ B^pi_{01}(x) = {P(theta in Theta_ 0mid x)
over P(theta in Theta_1mid x)} bigg/ {pi(theta in Theta_ 0)
over pi(theta in Theta_ 1)}. $$ This ratio evaluates the
modification of the odds of $Theta_0$ against $Theta_1$ due to the
obs and can naturally be compared to $1$, although an exact
comparison scale can only be based upon a loss function. In the
particular case where $Theta_0={theta_0}$ and
$Theta_1={theta_1}$, the Bayes factor simplifies to the usual
likelihood ratio $$ B^pi_{01} (x) = {f(x|theta_0)over
f(x|theta_1)}. $$ In general, the Bayes factor depends on prior
information, but is still proposed as an ``objective'' Bayesian
answer, since it partly eliminates the influence of the prior modeling
and emphasizes the role of the observations. Actually, it can be
perceived as a Bayesian likelihood ratio since, if $pi_0$ is the
prior distribution under $H_0$ and $pi_1$ the prior distribution
under $H_1$, $B^pi_{01}(x)$ can be written as begin{equation}
B^pi_{01} (x) = {int_{Theta_0} f(x|theta_0)pi_0(theta)
,text{d}theta over int_{Theta_1} f(x|theta_1)pi_1(theta)
,text{d}theta}
=frac{m_0(x)}{m_1(x)},, end{equation} thus replacing the likelihoods with the marginals under both hypotheses.
The prior is thus defined as a mixture:
$$pi(theta)=pi(theta in Theta_ 0)timespi_0(theta)timesmathbb{I}_{Theta_0}(theta)+pi(theta in Theta_ 1)timespi_1(theta)timesmathbb{I}_{Theta_1}(theta)$$where
$$pi(theta in Theta_ 0)=rho_0qquadtext{and}qquadpi(theta in Theta_ 1)=1-rho_0stackrel{text{def}}{=}rho_1$$
are the prior weights of both hypotheses and
$$int_{Theta_0} pi_0(theta_0)text{d}theta_0=int_{Theta_1} pi_1(theta_1)text{d}theta_1=1$$Therefore
begin{align*}P(theta in Theta_ 0|x)=int_{Theta_0} pi(theta_0|x)text{d}theta_0&=int_{Theta_0} pi(theta_0|x)text{d}theta_0\
&= frac{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} pi(theta_1)f(x|theta_1)text{d}theta_1}\
&=frac{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}
end{align*}
and
$$P(theta in Theta_1mid x)=frac{int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}$$Hence,
$$frac{P(theta in Theta_ 0|x)}{P(theta in Theta_1mid x)}=frac{rho_0int_{Theta_0} pi_0(theta_0)f(x|theta_0)text{d}theta_0}{rho_1int_{Theta_1} pi_1(theta_1)f(x|theta_1)text{d}theta_1}=frac{rho_0}{1-rho_0}B^pi_{01}(x)$$
This hopefully explains where the final expression comes from.
Alas, there is a mistake in the last ratio of integrals in the quoted text (presumably due to a cut & paste)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
which should be (when identifying the integrands differently in the two integrals to signify that the integrals are over two different parameter spaces)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta_0)dtheta_0}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta_1)dtheta_1}$$or (with another choice of representation of the integrands)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta)pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta)pi_{1}(theta)dtheta}$$
The only case when the notations $theta_0$ and $theta_1$ are important is when both null and alternative hypotheses are point hypotheses, i.e., when $Theta_0={theta_0}$ and $Theta_1={theta_1}$, since both symbols then take specific values, like $theta_0=3$ and $theta_1=-2$. In this specific case, the posterior is concentrated on ${theta_0,theta_1}$ and the Bayes factor writes $$ B^pi_{01} (x) = {f(x|theta_0)over f(x|theta_1)}$$ since
$$P(theta=theta_0|x)=frac{overbrace{pi(theta_0)}^{rho_0}f(x|theta_0)}{pi(theta_0)f(x|theta_0)+underbrace{pi(theta_1)}_{rho_1}f(x|theta_1)}$$
Thank you for pointing out this error, to be added to the list of typos.
1
Are there a few $f$s missing, and an accidentally squared $pi_1$?
– Taylor
2 hours ago
@Taylor: yes indeed, well spotted. Thx.
– Xi'an
1 hour ago
add a comment |
up vote
4
down vote
accepted
My deepest apologies, there is a typo in the final expression! Indeed, here is the complete text from my book (p.231):
Definition 5.5. The Bayes factor is the ratio of the posterior probabilities of the null and the alternative hypotheses over the
ratio of the prior probabilities of the null and the alternative
hypotheses, i.e., $$ B^pi_{01}(x) = {P(theta in Theta_ 0mid x)
over P(theta in Theta_1mid x)} bigg/ {pi(theta in Theta_ 0)
over pi(theta in Theta_ 1)}. $$ This ratio evaluates the
modification of the odds of $Theta_0$ against $Theta_1$ due to the
obs and can naturally be compared to $1$, although an exact
comparison scale can only be based upon a loss function. In the
particular case where $Theta_0={theta_0}$ and
$Theta_1={theta_1}$, the Bayes factor simplifies to the usual
likelihood ratio $$ B^pi_{01} (x) = {f(x|theta_0)over
f(x|theta_1)}. $$ In general, the Bayes factor depends on prior
information, but is still proposed as an ``objective'' Bayesian
answer, since it partly eliminates the influence of the prior modeling
and emphasizes the role of the observations. Actually, it can be
perceived as a Bayesian likelihood ratio since, if $pi_0$ is the
prior distribution under $H_0$ and $pi_1$ the prior distribution
under $H_1$, $B^pi_{01}(x)$ can be written as begin{equation}
B^pi_{01} (x) = {int_{Theta_0} f(x|theta_0)pi_0(theta)
,text{d}theta over int_{Theta_1} f(x|theta_1)pi_1(theta)
,text{d}theta}
=frac{m_0(x)}{m_1(x)},, end{equation} thus replacing the likelihoods with the marginals under both hypotheses.
The prior is thus defined as a mixture:
$$pi(theta)=pi(theta in Theta_ 0)timespi_0(theta)timesmathbb{I}_{Theta_0}(theta)+pi(theta in Theta_ 1)timespi_1(theta)timesmathbb{I}_{Theta_1}(theta)$$where
$$pi(theta in Theta_ 0)=rho_0qquadtext{and}qquadpi(theta in Theta_ 1)=1-rho_0stackrel{text{def}}{=}rho_1$$
are the prior weights of both hypotheses and
$$int_{Theta_0} pi_0(theta_0)text{d}theta_0=int_{Theta_1} pi_1(theta_1)text{d}theta_1=1$$Therefore
begin{align*}P(theta in Theta_ 0|x)=int_{Theta_0} pi(theta_0|x)text{d}theta_0&=int_{Theta_0} pi(theta_0|x)text{d}theta_0\
&= frac{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} pi(theta_1)f(x|theta_1)text{d}theta_1}\
&=frac{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}
end{align*}
and
$$P(theta in Theta_1mid x)=frac{int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}$$Hence,
$$frac{P(theta in Theta_ 0|x)}{P(theta in Theta_1mid x)}=frac{rho_0int_{Theta_0} pi_0(theta_0)f(x|theta_0)text{d}theta_0}{rho_1int_{Theta_1} pi_1(theta_1)f(x|theta_1)text{d}theta_1}=frac{rho_0}{1-rho_0}B^pi_{01}(x)$$
This hopefully explains where the final expression comes from.
Alas, there is a mistake in the last ratio of integrals in the quoted text (presumably due to a cut & paste)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
which should be (when identifying the integrands differently in the two integrals to signify that the integrals are over two different parameter spaces)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta_0)dtheta_0}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta_1)dtheta_1}$$or (with another choice of representation of the integrands)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta)pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta)pi_{1}(theta)dtheta}$$
The only case when the notations $theta_0$ and $theta_1$ are important is when both null and alternative hypotheses are point hypotheses, i.e., when $Theta_0={theta_0}$ and $Theta_1={theta_1}$, since both symbols then take specific values, like $theta_0=3$ and $theta_1=-2$. In this specific case, the posterior is concentrated on ${theta_0,theta_1}$ and the Bayes factor writes $$ B^pi_{01} (x) = {f(x|theta_0)over f(x|theta_1)}$$ since
$$P(theta=theta_0|x)=frac{overbrace{pi(theta_0)}^{rho_0}f(x|theta_0)}{pi(theta_0)f(x|theta_0)+underbrace{pi(theta_1)}_{rho_1}f(x|theta_1)}$$
Thank you for pointing out this error, to be added to the list of typos.
1
Are there a few $f$s missing, and an accidentally squared $pi_1$?
– Taylor
2 hours ago
@Taylor: yes indeed, well spotted. Thx.
– Xi'an
1 hour ago
add a comment |
up vote
4
down vote
accepted
up vote
4
down vote
accepted
My deepest apologies, there is a typo in the final expression! Indeed, here is the complete text from my book (p.231):
Definition 5.5. The Bayes factor is the ratio of the posterior probabilities of the null and the alternative hypotheses over the
ratio of the prior probabilities of the null and the alternative
hypotheses, i.e., $$ B^pi_{01}(x) = {P(theta in Theta_ 0mid x)
over P(theta in Theta_1mid x)} bigg/ {pi(theta in Theta_ 0)
over pi(theta in Theta_ 1)}. $$ This ratio evaluates the
modification of the odds of $Theta_0$ against $Theta_1$ due to the
obs and can naturally be compared to $1$, although an exact
comparison scale can only be based upon a loss function. In the
particular case where $Theta_0={theta_0}$ and
$Theta_1={theta_1}$, the Bayes factor simplifies to the usual
likelihood ratio $$ B^pi_{01} (x) = {f(x|theta_0)over
f(x|theta_1)}. $$ In general, the Bayes factor depends on prior
information, but is still proposed as an ``objective'' Bayesian
answer, since it partly eliminates the influence of the prior modeling
and emphasizes the role of the observations. Actually, it can be
perceived as a Bayesian likelihood ratio since, if $pi_0$ is the
prior distribution under $H_0$ and $pi_1$ the prior distribution
under $H_1$, $B^pi_{01}(x)$ can be written as begin{equation}
B^pi_{01} (x) = {int_{Theta_0} f(x|theta_0)pi_0(theta)
,text{d}theta over int_{Theta_1} f(x|theta_1)pi_1(theta)
,text{d}theta}
=frac{m_0(x)}{m_1(x)},, end{equation} thus replacing the likelihoods with the marginals under both hypotheses.
The prior is thus defined as a mixture:
$$pi(theta)=pi(theta in Theta_ 0)timespi_0(theta)timesmathbb{I}_{Theta_0}(theta)+pi(theta in Theta_ 1)timespi_1(theta)timesmathbb{I}_{Theta_1}(theta)$$where
$$pi(theta in Theta_ 0)=rho_0qquadtext{and}qquadpi(theta in Theta_ 1)=1-rho_0stackrel{text{def}}{=}rho_1$$
are the prior weights of both hypotheses and
$$int_{Theta_0} pi_0(theta_0)text{d}theta_0=int_{Theta_1} pi_1(theta_1)text{d}theta_1=1$$Therefore
begin{align*}P(theta in Theta_ 0|x)=int_{Theta_0} pi(theta_0|x)text{d}theta_0&=int_{Theta_0} pi(theta_0|x)text{d}theta_0\
&= frac{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} pi(theta_1)f(x|theta_1)text{d}theta_1}\
&=frac{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}
end{align*}
and
$$P(theta in Theta_1mid x)=frac{int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}$$Hence,
$$frac{P(theta in Theta_ 0|x)}{P(theta in Theta_1mid x)}=frac{rho_0int_{Theta_0} pi_0(theta_0)f(x|theta_0)text{d}theta_0}{rho_1int_{Theta_1} pi_1(theta_1)f(x|theta_1)text{d}theta_1}=frac{rho_0}{1-rho_0}B^pi_{01}(x)$$
This hopefully explains where the final expression comes from.
Alas, there is a mistake in the last ratio of integrals in the quoted text (presumably due to a cut & paste)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
which should be (when identifying the integrands differently in the two integrals to signify that the integrals are over two different parameter spaces)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta_0)dtheta_0}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta_1)dtheta_1}$$or (with another choice of representation of the integrands)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta)pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta)pi_{1}(theta)dtheta}$$
The only case when the notations $theta_0$ and $theta_1$ are important is when both null and alternative hypotheses are point hypotheses, i.e., when $Theta_0={theta_0}$ and $Theta_1={theta_1}$, since both symbols then take specific values, like $theta_0=3$ and $theta_1=-2$. In this specific case, the posterior is concentrated on ${theta_0,theta_1}$ and the Bayes factor writes $$ B^pi_{01} (x) = {f(x|theta_0)over f(x|theta_1)}$$ since
$$P(theta=theta_0|x)=frac{overbrace{pi(theta_0)}^{rho_0}f(x|theta_0)}{pi(theta_0)f(x|theta_0)+underbrace{pi(theta_1)}_{rho_1}f(x|theta_1)}$$
Thank you for pointing out this error, to be added to the list of typos.
My deepest apologies, there is a typo in the final expression! Indeed, here is the complete text from my book (p.231):
Definition 5.5. The Bayes factor is the ratio of the posterior probabilities of the null and the alternative hypotheses over the
ratio of the prior probabilities of the null and the alternative
hypotheses, i.e., $$ B^pi_{01}(x) = {P(theta in Theta_ 0mid x)
over P(theta in Theta_1mid x)} bigg/ {pi(theta in Theta_ 0)
over pi(theta in Theta_ 1)}. $$ This ratio evaluates the
modification of the odds of $Theta_0$ against $Theta_1$ due to the
obs and can naturally be compared to $1$, although an exact
comparison scale can only be based upon a loss function. In the
particular case where $Theta_0={theta_0}$ and
$Theta_1={theta_1}$, the Bayes factor simplifies to the usual
likelihood ratio $$ B^pi_{01} (x) = {f(x|theta_0)over
f(x|theta_1)}. $$ In general, the Bayes factor depends on prior
information, but is still proposed as an ``objective'' Bayesian
answer, since it partly eliminates the influence of the prior modeling
and emphasizes the role of the observations. Actually, it can be
perceived as a Bayesian likelihood ratio since, if $pi_0$ is the
prior distribution under $H_0$ and $pi_1$ the prior distribution
under $H_1$, $B^pi_{01}(x)$ can be written as begin{equation}
B^pi_{01} (x) = {int_{Theta_0} f(x|theta_0)pi_0(theta)
,text{d}theta over int_{Theta_1} f(x|theta_1)pi_1(theta)
,text{d}theta}
=frac{m_0(x)}{m_1(x)},, end{equation} thus replacing the likelihoods with the marginals under both hypotheses.
The prior is thus defined as a mixture:
$$pi(theta)=pi(theta in Theta_ 0)timespi_0(theta)timesmathbb{I}_{Theta_0}(theta)+pi(theta in Theta_ 1)timespi_1(theta)timesmathbb{I}_{Theta_1}(theta)$$where
$$pi(theta in Theta_ 0)=rho_0qquadtext{and}qquadpi(theta in Theta_ 1)=1-rho_0stackrel{text{def}}{=}rho_1$$
are the prior weights of both hypotheses and
$$int_{Theta_0} pi_0(theta_0)text{d}theta_0=int_{Theta_1} pi_1(theta_1)text{d}theta_1=1$$Therefore
begin{align*}P(theta in Theta_ 0|x)=int_{Theta_0} pi(theta_0|x)text{d}theta_0&=int_{Theta_0} pi(theta_0|x)text{d}theta_0\
&= frac{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} pi(theta_1)f(x|theta_1)text{d}theta_1}\
&=frac{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}
end{align*}
and
$$P(theta in Theta_1mid x)=frac{int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}$$Hence,
$$frac{P(theta in Theta_ 0|x)}{P(theta in Theta_1mid x)}=frac{rho_0int_{Theta_0} pi_0(theta_0)f(x|theta_0)text{d}theta_0}{rho_1int_{Theta_1} pi_1(theta_1)f(x|theta_1)text{d}theta_1}=frac{rho_0}{1-rho_0}B^pi_{01}(x)$$
This hopefully explains where the final expression comes from.
Alas, there is a mistake in the last ratio of integrals in the quoted text (presumably due to a cut & paste)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
which should be (when identifying the integrands differently in the two integrals to signify that the integrals are over two different parameter spaces)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta_0)dtheta_0}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta_1)dtheta_1}$$or (with another choice of representation of the integrands)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta)pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta)pi_{1}(theta)dtheta}$$
The only case when the notations $theta_0$ and $theta_1$ are important is when both null and alternative hypotheses are point hypotheses, i.e., when $Theta_0={theta_0}$ and $Theta_1={theta_1}$, since both symbols then take specific values, like $theta_0=3$ and $theta_1=-2$. In this specific case, the posterior is concentrated on ${theta_0,theta_1}$ and the Bayes factor writes $$ B^pi_{01} (x) = {f(x|theta_0)over f(x|theta_1)}$$ since
$$P(theta=theta_0|x)=frac{overbrace{pi(theta_0)}^{rho_0}f(x|theta_0)}{pi(theta_0)f(x|theta_0)+underbrace{pi(theta_1)}_{rho_1}f(x|theta_1)}$$
Thank you for pointing out this error, to be added to the list of typos.
edited 1 hour ago
answered yesterday
Xi'an
52.2k688339
52.2k688339
1
Are there a few $f$s missing, and an accidentally squared $pi_1$?
– Taylor
2 hours ago
@Taylor: yes indeed, well spotted. Thx.
– Xi'an
1 hour ago
add a comment |
1
Are there a few $f$s missing, and an accidentally squared $pi_1$?
– Taylor
2 hours ago
@Taylor: yes indeed, well spotted. Thx.
– Xi'an
1 hour ago
1
1
Are there a few $f$s missing, and an accidentally squared $pi_1$?
– Taylor
2 hours ago
Are there a few $f$s missing, and an accidentally squared $pi_1$?
– Taylor
2 hours ago
@Taylor: yes indeed, well spotted. Thx.
– Xi'an
1 hour ago
@Taylor: yes indeed, well spotted. Thx.
– Xi'an
1 hour ago
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f378496%2fwhere-is-my-mistake-in-this-definition-of-bayes-factor%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown