Menu ENN Search
Language: English Français

Coverage estimate calculation - Final stage - Help again

This question was posted the Coverage assessment forum area and has 14 replies. You can also reply via email – be sure to leave the subject unchanged.

» Post a reply

Géraldine LE CUZIAT

ACF

Normal user

14 Dec 2011, 18:56

Dear all,

I hope you are doing fine.
Well, I still have an issue and need a bit of guidance. We are in the very final stage of the SQUEAC investigation. I have just compiled the No of current SAM cases attending the programme (n=59), No of current SAM cases not attending the programme (n=82). The total no of current SAM cases makes 141 - which is way too high for a denominator and can not be entered in the Bayes software !!
any suggestion ?
Thanks in advance for your support,
Géraldine

Mark Myatt

Consultant Epideomiologist

Frequent user

15 Dec 2011, 10:39

We can change the software to handle larger sample sizes. It is open-source so you (or anyone else) can change that. The source code is here.

You need to change line 963 which starts:

    scale .frameControls.likeN -variable likeN -from 1 -to 96

change the 96 to anything you like and recompile using a tclkit.

If there is a demand for a larger limit then I will make the change and compile for Windows and OS-X.

In the mean time I suggest that you use the hand calculation method as outline in this section of the SQUEAC handbook.

If you post the data here we could go through it as a "worked example".

Saul Guerrero

Director of Nutrition

Technical expert

16 Dec 2011, 12:45

I think the idea of doing these calculations on paper here would helpful.

Having said that, I also think that this is not an isolated event. I know that MSF-S had this exact same issue in India about a year ago. As we continue to do more and more of these in places with high caseloads, the need for a higher denominator may come up again.

Just a thought.

S

Mark Myatt

Consultant Epideomiologist

Frequent user

16 Dec 2011, 16:58

OK. Give me an upper limit and I will make the changes, compile for Windows and OS-X and put it all up on the web.

So ... let me know and I will do what needs to be done.

Saul Guerrero

Director of Nutrition

Technical expert

16 Dec 2011, 17:06

Hi Mark - thanks for offering to do this (I have spent some time today trying to find a tclkit to do the changes..)

The highest number of cases found by MSF in India (in one area) was 140. In Myanmar it was 141. I think that setting the upper limit at 150 could be sufficient, but to avoid having to do it yet again in the future, perhaps we could do 200?

You know best though.

Hope that helps

S

Lio

CMAM advisor

Technical expert

19 Dec 2011, 10:59

A quick thought,
Yes, when the caseload is likely to be high, it is quite frequent that the final sample is larger than the required sample size; therefore adjustment in the software is welcome. However, having a very large sample size is not necessary and will not really improve the precision. In my opinion, when large caseload is expected (large number of SAM Kids or Kids in the programme) are foreseen, it would save time and resources to reduce the number of villages to survey in the same quadrant or in the same area (depending what is the sampling method)

Mark Myatt

Consultant Epideomiologist

Frequent user

19 Dec 2011, 18:19

I have made a new version (2.01) with an upper limit of 192 for the denominator sample size. You can get it from here. Let me know if this is OK,

Géraldine LE CUZIAT

ACF

Normal user

20 Dec 2011, 08:48

Mingalaba Mark,

Thanks for this and apologies for the late reply - but the connection in Yangon is a headache. I could not see your post before.
Well, I have a problem as the denominator (n = 203) of the period coverage still exceeds the revised version. I tried to change the code manually but the connection is not stable enough. Can you change again up to 250 ?
Thanks again !!
all the best for the coming year and Christmas,
Géraldine

Mark Myatt

Consultant Epideomiologist

Frequent user

20 Dec 2011, 09:51

Finding a sample size larger than original required is not uncommon in a first survey using active and adaptive case-finding with a given set of survey staff. This is because the survey staff get better (quicker and better) quite quickly over time. Another reason for this is that you may do small-area surveys in phase II to investigate possible low coverage areas and these areas are not low coverage but low prevalence (i.e. low program case-numbers from these areas reflect low prevalence not low coverage). This can lead you to underestimate prevalence over a wide-area and to choose to select a larger number of villages than is needed to get the required sample size.

Probably the most common reason for getting a much larger sample size than required is unrecognised low coverage. People get to thinking that their program has good coverage. If they have low case numbers they can easily believe that this is due to low prevalence rather than to low coverage. From "inside" a program the two situations:

    Low prevalence = Low case numbers

    High prevalence + Low coverage = Low case numbers

can be difficult to distinguish from each other and wishful thinking can cloud judgement so that evidence of low coverage is ignored (there is an example of this in the case-studies in the SQUEAC handbook).

The sample size prompting this question is four or five times larger than required. This suggests a large miscalculation on the part of the investigator. Everyone makes mistakes. It would be interesting to hear Géraldine's take on this.

A sample size such as n = 96 is conventional for coverage surveys of child survival interventions (e.g. EPI surveys are design to have an effective sample size, after accounting for expected design effects, of n = 96). I am not in favour in increasing the sample size limit in the BayesSQUEAC software since SQUEAC is supposed to be a low resource method. A sample size of much about n = 50 will usually be a waste of resources (I have done a few SQUEACs and cannot recall ever going above that). I have, however, increased this to n = 192 at Saul's request.

A note on sample size and precision : A larger sample size will improve precision but not as much as you might think. Moving (e.g.) from a sample sise of 100 to a sample size of 200 does not double precision (or half the with of the credible or confidence interval). Instead, precision increase with the square root of sample size. This means that the doubling the sample size increases precision by about 1.4 times (i.e. the square root of 2). Moving (e.g.) from a sample size of 100 to a sample size of 1000 improves precision by only about 3.2 times (i.e. the square root of 10).

Mark Myatt

Consultant Epideomiologist

Frequent user

20 Dec 2011, 10:28

I have made a new version with an upper limit on the denominator of 256. You can download it it from here. Check if this is OK.

Géraldine LE CUZIAT

ACF

Normal user

10 Jan 2012, 09:32

Dear Mark and all,

Apologies for being silent, as I was off for Christmas and New Year’s Eve.
I will try to answer as best I can. I strictly applied the SQUEAC methodology as per handbook. I have gone through all the calculations and could not find where the mistake could be coming from. I may have overestimated/underestimated some information as this was supervised in remote from Yangon (and I never went to the field), but data interpretation was the result of fruitful teamwork collaboration and a strong technical guidance from Saul.

To give a bit of background information, ACF directly implements OTPs programme in Maungdaw (MGD) townships in the Northern Rakhine State with protracted high rates of acute malnutrition. The last figures available (December 2010) indicated a GAM rate of 19% and a prevalence of SAM of 2.9% in Maungdaw. From the period of January to October 2011, ACF admitted 3881 SAM children.

STAGE 1
Based on the information collected and analysed in Stage 1, the investigation concluded that coverage is likely to be not homogenous throughout MGD Township. Two primary factors affecting coverage were identified:
1. Spatial distribution and distance (no admissions & no facility in the northern part of the township)
2. Awareness about the programme

After discussing with the team and Saul, we concluded that high and low coverage classification was not enough to reflect the picture in MGD. Although, we knew that this risked being too precise, we added a 3rd level to see if we could pick up a difference between mid and high coverage.

The hypothesis became:
• Coverage is low (less than 20%) in northern areas that do not have a nutrition facility providing CMAM services.
• Coverage is medium (between 20 and 50%) in southern areas that have nutrition facilities providing CMAM but no community awareness about programme.
• Coverage is high (higher than 50%) in central areas that have long-running ACF nutrition facilities providing CMAM.
The first two hypotheses were proven – meanwhile the last one was refuted.

STAGE 2
The Prior was constructed with a mode of 38.5% with main positive feedbacks including high caseload of self/spontaneous referrals, timely treatment seeking, good malnutrition awareness and existing treatment programme, long running programme implementation and adequate programme performance indicators. The following negative aspects were pinpointed: distance, poor spatial OTP distribution, travel authorisation required for the patients to move within the township and high cost/opportunity ratio.

The following results were calculated following SQUEAC methodology.
a = 12.74
ß = 20.35
n Likelihood = 60
11 villages in 7 quadrats

STAGE 3
The findings of the wide-area survey are displayed below:
Number of current (SAM) cases : 145
Number of current (SAM) cases attending the programme: 60
Number of current (SAM) cases not attending the programme: 85
Number of recovering cases attending the programme: 63

At the end, the Prior, the Likelihood and the Posterior look fine with good overlapping.

A combination of factors may explain the higher sample size (n=145 – has gone up compared to previous posts – n=141) found compared to the initial sample size (n= 60):
- High SAM prevalence
- Higher opportunity/cost than expected in the earlier stage. People know about malnutrition and treatment programme but make a decision not to attend.

I will not be able to give the final coverage estimates, as the report validation is pending. However, we learnt a lot in terms of programme performance and ways forward.

Hope this helps,
Géraldine

Ernest Guevarra

Valid International

Technical expert

10 Jan 2012, 15:44

Dear Géraldine,

Thanks for your update. Your posts launched the coverage assessment forum well and started a lively discussion.

I just have two questions I wanted to ask you regarding the SQUEAC you implemented and that we are currently discussing about in this forum.

1. I am curious, what do you think is the reason why you suddenly got more cases than you expected? I remember, this post started when you asked for help on what to do if you don't reach your sample size and then after a few days, you shared good news that you were actually finding more than what you needed.

2. What is your case definition for this SQUEAC survey that you just did? Is it MUAC = 115 only or does it include a weight for height criteria? If the latter, was your case finding methodology active and adaptive case finding?

Thanks again for sharing.

Mark Myatt

Consultant Epideomiologist

Frequent user

11 Jan 2012, 16:39

Géraldine,

Thanks for your detailed response. Have you considered writing this up as a case-study for publication in Field Exchange? I think you should.

Géraldine LE CUZIAT

ACF

Normal user

12 Jan 2012, 03:07

Dear Ernest & all,

Thanks for your questions. I think I understand where the misunderstanding comes from. I should have given a bit more of background information, as I was reading the posts again and realised how confusing this might be.

We ran TWO separate SQUEAC investigations at the same time in TWO different (neighbouring) Townships, respectively Maungdaw and Butidaung. This was not the original plan, but this was the right move because the findings and the coverage patterns ended up being very different.

My first post ‘stage 3 – sample size issue’ was for Buthidaung Township, where we faced difficulty in reaching the expected sample size in stage 3; initially, we did not think this would be a problem – we expected a high caseload as a mere reflection of high prevalence in the area (like Maungdaw). We finally got more cases (n=82) than the required sample size (n=52), but an in-depth analysis of the outcomes reveals unforeseen findings. Unfortunately, I am not in a position to disclose them in a public forum.

My second post ‘Coverage estimate calculation –final stage” was about Maungdaw where we expected a high caseload and we got it – even bigger than what the software can handle.

The case definition in both Townships used was for marasmus and kwashiorkor cases - MUAC < 115m and/or bilateral oedemas. Combined active & adaptive with house-to-house case finding methodology was used to ensure exhaustive coverage of targeted areas.

My SQUEAC mission is about to finish but I really enjoyed the whole process!
Thanks again for your on-line support,
Géraldine

Mark Myatt

Consultant Epideomiologist

Frequent user

12 Jan 2012, 12:07

Géraldine,

Thanks for the clarification. Most of the SQUEAC material available to data has come from people involved with the development of the method. The major exception to this is this Field Exchange article which was written before the addition of the "stage 3" survey method. I think we need more user-produced documentation. I think, therefore, that it would be very useful for you to document your experiences using the new SQUEAC method in a Field Exchange article.

Mark

Back to top

» Post a reply