How to infer difference of population proportion between two groups when proportion is small?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}
$begingroup$
I have a dataset where the issue is of this form.
There are two groups, Group A (N=5000) and Group B (N=1000). Let's say 5 people in Group A develop a certain disease, and only 2 in group B do.
Then the proportion for A is 5/5000 -> 0.001 and for B it is 2/1000 -> 0.002.
How can I test if the proportion between these two groups is statistically significant?
The tests I found online rely on the Central Limit Theorem, such that np>=10 and n(1-p) >= 10, which does not hold for my dataset. Are there any other approaches?
inference proportion
asked 6 hours ago
maxmax
1083
New contributor
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
I have a dataset where the issue is of this form.
There are two groups, Group A (N=5000) and Group B (N=1000). Let's say 5 people in Group A develop a certain disease, and only 2 in group B do.
Then the proportion for A is 5/5000 -> 0.001 and for B it is 2/1000 -> 0.002.
How can I test if the proportion between these two groups is statistically significant?
The tests I found online rely on the Central Limit Theorem, such that np>=10 and n(1-p) >= 10, which does not hold for my dataset. Are there any other approaches?
inference proportion
asked 6 hours ago
maxmax
1083
New contributor
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
$begingroup$
Use Fisher Exact Test per discussion in Answer.
$endgroup$
– BruceET
2 hours ago
add a comment |
$begingroup$
I have a dataset where the issue is of this form.
There are two groups, Group A (N=5000) and Group B (N=1000). Let's say 5 people in Group A develop a certain disease, and only 2 in group B do.
Then the proportion for A is 5/5000 -> 0.001 and for B it is 2/1000 -> 0.002.
How can I test if the proportion between these two groups is statistically significant?
The tests I found online rely on the Central Limit Theorem, such that np>=10 and n(1-p) >= 10, which does not hold for my dataset. Are there any other approaches?
inference proportion
asked 6 hours ago
maxmax
1083
New contributor
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
I have a dataset where the issue is of this form.
There are two groups, Group A (N=5000) and Group B (N=1000). Let's say 5 people in Group A develop a certain disease, and only 2 in group B do.
Then the proportion for A is 5/5000 -> 0.001 and for B it is 2/1000 -> 0.002.
How can I test if the proportion between these two groups is statistically significant?
The tests I found online rely on the Central Limit Theorem, such that np>=10 and n(1-p) >= 10, which does not hold for my dataset. Are there any other approaches?
inference proportion
inference proportion
asked 6 hours ago
maxmax
1083
New contributor
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 6 hours ago
maxmax
1083
New contributor
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 6 hours ago
maxmax
1083
New contributor
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 6 hours ago
maxmax
1083
asked 6 hours ago
maxmax
1083
1083
New contributor
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$begingroup$
Use Fisher Exact Test per discussion in Answer.
$endgroup$
– BruceET
2 hours ago
add a comment |
$begingroup$
Use Fisher Exact Test per discussion in Answer.
$endgroup$
– BruceET
2 hours ago
$begingroup$
Use Fisher Exact Test per discussion in Answer.
$endgroup$
– BruceET
2 hours ago
$begingroup$
Use Fisher Exact Test per discussion in Answer.
$endgroup$
– BruceET
2 hours ago
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
The traditional approach would be to do a 2-sample test for a difference in proportions: In Minitab, results of this test are as shown below. The warning about the first P-value from a normal approximation causes doubt,
roughly for the reasons you mention.
However, the result from Fisher's exact test uses an exact hypergeometric
probability. It also shows no significant difference.
Test and CI for Two Proportions
Sample X N Sample p
1 5 5000 0.001000
2 2 1000 0.002000
Difference = p (1) - p (2)
Estimate for difference: -0.001
95% upper bound for difference: 0.00143738
Test for difference = 0 (vs < 0):
Z = -0.67 P-Value = 0.250
* NOTE * The normal approximation may be
inaccurate for small samples.
Fisher’s exact test: P-Value = 0.330
A direct hypergeometric computation in R can be argued
as follows. Suppose an urn contains $5000$ tokens marked A and $1000$ marked B. Seven tokens are taken
at random without replacement, corresponding to disease.
What is the probability five or fewer of those are marked A?
The answer is
$$sum_{k=0}^5frac{{5000 choose k}{1000 choose 7-k}}{{6000 choose 7}} = 0.3302,$$
which agrees with the P-value from Fisher's exact test.
In R, the computation can be done in terms of a hypergeometric CDF:
phyper(5, 5000, 1000, 7)
[1] 0.330204
Here is a plot of this hypergeometric distribution. The P-value is the sum of the heights of the bars to the left of the vertical dotted line.
answered 2 hours ago
BruceETBruceET
7,0461721
$endgroup$
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "65"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
max is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f404331%2fhow-to-infer-difference-of-population-proportion-between-two-groups-when-proport%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
The traditional approach would be to do a 2-sample test for a difference in proportions: In Minitab, results of this test are as shown below. The warning about the first P-value from a normal approximation causes doubt,
roughly for the reasons you mention.
However, the result from Fisher's exact test uses an exact hypergeometric
probability. It also shows no significant difference.
Test and CI for Two Proportions
Sample X N Sample p
1 5 5000 0.001000
2 2 1000 0.002000
Difference = p (1) - p (2)
Estimate for difference: -0.001
95% upper bound for difference: 0.00143738
Test for difference = 0 (vs < 0):
Z = -0.67 P-Value = 0.250
* NOTE * The normal approximation may be
inaccurate for small samples.
Fisher’s exact test: P-Value = 0.330
A direct hypergeometric computation in R can be argued
as follows. Suppose an urn contains $5000$ tokens marked A and $1000$ marked B. Seven tokens are taken
at random without replacement, corresponding to disease.
What is the probability five or fewer of those are marked A?
The answer is
$$sum_{k=0}^5frac{{5000 choose k}{1000 choose 7-k}}{{6000 choose 7}} = 0.3302,$$
which agrees with the P-value from Fisher's exact test.
In R, the computation can be done in terms of a hypergeometric CDF:
phyper(5, 5000, 1000, 7)
[1] 0.330204
Here is a plot of this hypergeometric distribution. The P-value is the sum of the heights of the bars to the left of the vertical dotted line.
answered 2 hours ago
BruceETBruceET
7,0461721
$endgroup$
add a comment |
$begingroup$
The traditional approach would be to do a 2-sample test for a difference in proportions: In Minitab, results of this test are as shown below. The warning about the first P-value from a normal approximation causes doubt,
roughly for the reasons you mention.
However, the result from Fisher's exact test uses an exact hypergeometric
probability. It also shows no significant difference.
Test and CI for Two Proportions
Sample X N Sample p
1 5 5000 0.001000
2 2 1000 0.002000
Difference = p (1) - p (2)
Estimate for difference: -0.001
95% upper bound for difference: 0.00143738
Test for difference = 0 (vs < 0):
Z = -0.67 P-Value = 0.250
* NOTE * The normal approximation may be
inaccurate for small samples.
Fisher’s exact test: P-Value = 0.330
A direct hypergeometric computation in R can be argued
as follows. Suppose an urn contains $5000$ tokens marked A and $1000$ marked B. Seven tokens are taken
at random without replacement, corresponding to disease.
What is the probability five or fewer of those are marked A?
The answer is
$$sum_{k=0}^5frac{{5000 choose k}{1000 choose 7-k}}{{6000 choose 7}} = 0.3302,$$
which agrees with the P-value from Fisher's exact test.
In R, the computation can be done in terms of a hypergeometric CDF:
phyper(5, 5000, 1000, 7)
[1] 0.330204
Here is a plot of this hypergeometric distribution. The P-value is the sum of the heights of the bars to the left of the vertical dotted line.
answered 2 hours ago
BruceETBruceET
7,0461721
$endgroup$
add a comment |
$begingroup$
The traditional approach would be to do a 2-sample test for a difference in proportions: In Minitab, results of this test are as shown below. The warning about the first P-value from a normal approximation causes doubt,
roughly for the reasons you mention.
However, the result from Fisher's exact test uses an exact hypergeometric
probability. It also shows no significant difference.
Test and CI for Two Proportions
Sample X N Sample p
1 5 5000 0.001000
2 2 1000 0.002000
Difference = p (1) - p (2)
Estimate for difference: -0.001
95% upper bound for difference: 0.00143738
Test for difference = 0 (vs < 0):
Z = -0.67 P-Value = 0.250
* NOTE * The normal approximation may be
inaccurate for small samples.
Fisher’s exact test: P-Value = 0.330
A direct hypergeometric computation in R can be argued
as follows. Suppose an urn contains $5000$ tokens marked A and $1000$ marked B. Seven tokens are taken
at random without replacement, corresponding to disease.
What is the probability five or fewer of those are marked A?
The answer is
$$sum_{k=0}^5frac{{5000 choose k}{1000 choose 7-k}}{{6000 choose 7}} = 0.3302,$$
which agrees with the P-value from Fisher's exact test.
In R, the computation can be done in terms of a hypergeometric CDF:
phyper(5, 5000, 1000, 7)
[1] 0.330204
Here is a plot of this hypergeometric distribution. The P-value is the sum of the heights of the bars to the left of the vertical dotted line.
answered 2 hours ago
BruceETBruceET
7,0461721
$endgroup$
The traditional approach would be to do a 2-sample test for a difference in proportions: In Minitab, results of this test are as shown below. The warning about the first P-value from a normal approximation causes doubt,
roughly for the reasons you mention.
However, the result from Fisher's exact test uses an exact hypergeometric
probability. It also shows no significant difference.
Test and CI for Two Proportions
Sample X N Sample p
1 5 5000 0.001000
2 2 1000 0.002000
Difference = p (1) - p (2)
Estimate for difference: -0.001
95% upper bound for difference: 0.00143738
Test for difference = 0 (vs < 0):
Z = -0.67 P-Value = 0.250
* NOTE * The normal approximation may be
inaccurate for small samples.
Fisher’s exact test: P-Value = 0.330
A direct hypergeometric computation in R can be argued
as follows. Suppose an urn contains $5000$ tokens marked A and $1000$ marked B. Seven tokens are taken
at random without replacement, corresponding to disease.
What is the probability five or fewer of those are marked A?
The answer is
$$sum_{k=0}^5frac{{5000 choose k}{1000 choose 7-k}}{{6000 choose 7}} = 0.3302,$$
which agrees with the P-value from Fisher's exact test.
In R, the computation can be done in terms of a hypergeometric CDF:
phyper(5, 5000, 1000, 7)
[1] 0.330204
Here is a plot of this hypergeometric distribution. The P-value is the sum of the heights of the bars to the left of the vertical dotted line.
answered 2 hours ago
BruceETBruceET
7,0461721
answered 2 hours ago
BruceETBruceET
7,0461721
answered 2 hours ago
BruceETBruceET
7,0461721
answered 2 hours ago
BruceETBruceET
7,0461721
7,0461721
add a comment |
add a comment |
max is a new contributor. Be nice, and check out our Code of Conduct.
max is a new contributor. Be nice, and check out our Code of Conduct.
max is a new contributor. Be nice, and check out our Code of Conduct.
max is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Cross Validated!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f404331%2fhow-to-infer-difference-of-population-proportion-between-two-groups-when-proport%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Use Fisher Exact Test per discussion in Answer.
$endgroup$
– BruceET
2 hours ago