AAH

Normal user

13 Feb 2012, 17:24

Hi,
We just finish a SQUEAC in Haiti region.
Our food security coordinator asking a excellent question (we had the same question in Mali) and we don’t have any answer for now could you try to give us an answer?
Our coverage being 12, 4 % we have a reliable interval of [6 - 24 %] normally the reliable intervals are "equal" like [0 - 24 % or 6 - 18 %] what is the reason of this disproportion in SQUEAC methodology?
Thank you!
C.

ACF Senior Technical Advisor

Normal user

15 Feb 2012, 18:53

I always thought that the reason for this is that the theoretical distribution of the population parameter estimated is not symmetric, but maybe I am wrong... In terms of SQUEAC it means that the coverage rate estimated in the prior is not symmetric. If you look to the "blue" curve of the prior the right tail (or the right half) is larger than the left for this example.

CMAM Advisor

Technical expert

16 Feb 2012, 06:34

I just guess an answer and I am sure that experts will be able to give a more "mathematical" explanation. Contrary to the frequentist approach, in Bayesian we talk about “credibility interval” and not “confidence interval” this is because the Bayesian approach uses the prior results which are based on “beliefs’ about coverage . In SQUEAC, the prior may be very asymmetric (as José suggested) and I presume that this asymmetric effect (symmetric credibility interval) will be stronger if your sample size (for the likelihood) was small (was it the case?). Any other explanations?

Frequent user

16 Feb 2012, 10:18

First some terminology ... SQUEAC uses a Bayesian approach to estimating coverage. This yields a point estimate and a [b][i]credible interval[/i][/b]. This is different from a frequentist [b][i]confidence interval[/b][/i]. A credible interval give the range of values that contains (with 95% probability) the true value. A confidence interval gives the range in which the estimated value would occur 95% of the time with repeated sampling from the same population with the same method and is [b]not[/b] an indication as to whether or not the confidence interval estimated from the current sample contains the true value.
Most people intuitively misinterpret a frequentist confidence interval as though it were a Bayesian credibility interval.
A properly calculated credible interval (or confidence interval) around an estimate of a proportion is only symmetrical when the estimated proportion is 50%. A good model for calculating CIs around an estimate of a proportion is the binomial distribution. Here is a table of 95% CIs around estimates of 10%, 20%, 30%, 40%, and 50% with a sample size of 100:
[code]
estimate 95% CI
-------- ----------------
10% 4.90% -> 17.62%
20% 12.67% -> 29.18%
30% 21.24% -> 39.98%
40% 30.33% -> 50.28%
50% 39.83% -> 60.17%
-------- ----------------
[/code]
Note that the CI becomes more symmetrical as the estimate approaches 50% but that at 20% the CI is reasonably symmetrical.
The CIs given above are known as "exact" confidence intervals. Before we all had powerful computers on our desks or in our pockets it was very difficult to calculate these so we used an approximate method based on the normal distribution. This works well when you have estimates between about 20% and 80% with samples sizes of about [i]n[/i] = 50 or more. A well used rule-of-thumb is that the normal approximation is safe when:
[code]
sample size * estimate > 5
sample size - sample size * estimate > 5
[/code]
are true.
The normal distribution is a symmetrical distribution so the approximate method will always yield a symmetrical CI. With the same estimates and sample size as used above the approximate method yields:
[code]
estimate 95% CI
-------- ----------------
19% 4.12% -> 15.88%
20% 12.16% -> 27.84%
30% 21.02% -> 38.98%
40% 30.40% -> 49.60%
50% 40.20% -> 59.80%
-------- ----------------
[/code]
Note that the CI is now always symmetrical. This is a result of using an approximate method. If you are used to seeing symmetrical CIs then it is very likely that you (or your software) is using approximate methods. You should also note that the 95% CIs generated by the approximate method are narrower than those generated by the exact method. Both of these effects (i.e. symmetry and improved precision) are [b][i]errors[/i][/b] of the approximate method. When the rule of thumb (above) is met than these errors are not large.
BTW : Another weakness of the normal approximation is that the normal distribution is not constrained to range between 0% and 100%. This means that with low or high proportions and / or small samples sizes it can give CIs that include numbers below 0% and above 100% (i.e. impossible values). This has come up in these posts
[url]http://www.en-net.org.uk/question/290.aspx[/url]
[url]http://www.en-net.org.uk/question/333.aspx[/url]
The hand calculation methods in the SQUEAC handbook are approximate (they use a normal approximation) and will yield "equal tailed" 95% credible intervals. The BayesSQUEAC calculator uses a bootstrap aggregation estimator (this is described in the technical appendix to the SQUEAC handbook) which is a reasonably exact method and will only yield an "equal tailed" CI when the estimate is 50%.
In summary ... There is no "disproportion" in the SQUEAC method. Equal tailed CIs are an artefact of using an approximate method.
I hope this answers your question.