Public Health Nutritionist

Normal user

25 Nov 2013, 11:07

I have tried to check different SQUEAC investigation reports. I am observing inconsistencies in weighing and scoring system. Regardless of the number of boosters and barriers, the score was given differently. Some score between 0 and 3, some between 0 and 4, some between 0 and 5 and some between 0 and 10 and so on. I have seen report with 27 barriers with maximum score of each 3. I have also seen report of 9 barriers with each maximum score of 4. I think, such under or over scoring value could twist or distort the estimation of prior. if the posterior is distorted then it affect the posterior. So, is there a general rule how to set the maximum scoring system. I am not sure whether or not it is true but was told the maximum score should be determined by 100 divided by the number of boosters or barriers. For example if the number of barriers is 20, then the maximum score of each barrier or booster should be 5. If they are 10, then the maximum score of each should be 10. But the question will be what if the number of barriers and boosters are different like booster 10 and barrier 20. In this case should we set the scoring for booster and barrier separate?
Thanks

CMAM Advisor

Technical expert

25 Nov 2013, 11:54

Dear Kiross,
thank you for your observation. You are right, general rule should be that the maximum score cannot be higher than 100 divided by the number of barriers or boosters because, hypothetically, you may have only barriers or only boosters and coverage cannot be higher than 100 or lower than 0. Saying this, I personally never used a score higher than 5; the teams can chose if they are more comfortable giving score 1,2,3,4,5 or 1,3,5. However, barriers and booster should have the same scoring scale. Based on the "rule" of dividing 100, you will use the higher of the two, if you have 20 boosters and 10 barriers, you will use 20 (so maximum of 5). As far as I know, there is no particular rule to set a score if not the one you mentioned. You can also use the same score for each barrier or booster (not weighted prior), and we do with the current way of calculating the prior, but we also know that barriers and boosters don't have the same impact on coverage, reason why we set a scale;

Brixton Health

Frequent user

25 Nov 2013, 12:22

The weights given to barriers and boosters can vary. If (e.g.) you have several minor barriers and one major barrier such as repeated RUTF stock-outs then you might give a weight of 2 or 3 to the minor barriers and a truly massive weight (e.g. 30) to the "show-stopper" barrier. The weights given to each barrier and booster are supposed to reflect the strength of effect of each barrier and booster on coverage. The effect of a particular barrier may vary from program to program so the weight associated with a particular barrier may vary from program to program. A few years ago I did a SQUEAC in a program run by a predominantly Christian organisation that had done very little to sensitise, mobilise, or accommodate the Moslem community. In this setting the population of the program area was over 70% Moslem. This was therefore a big barrier to coverage. If the population had been 7% Moslem then it may not have been such an important barrier to coverage. It would still be nasty prejudice but not such an important barrier to coverage.
BTW : If you set all weights equal then you have an unweighted method.
In my experience it is the tendency to want coverage to be good that results in barriers being ignored or given low weights that creates a bias rather than the range of weights that are used.
Some rules ...
Don't use methods like this is a dumb mechanistic manner.
The magnitude of the scores should reflect the expected magnitude of effect.
It is better to underestimate magnitude of effects if you intend to use the sums of weights as credible limits for the prior mode.
Limit scores so that coverage is not below zero or above 100 (see page 77 of the SQUEAC / SLEAC Technical Reference).
Use several methods to work on the prior. I tend to use unweighted and weighted scores and a histogram prior approach and then triangulate. Then I think about it some more and do it all again.
Set realistic strengths for the prior. On a first SQUEAC it is common to agree on a prior mode and then set wide credible limits on the prior mode (e.g. plus or minus 25%).
Be realistic about the prior mode. Don't set your prior by wishful thinking.
WRT "distortion" ... (I think you mean "bias") ... this is why we check for prior-likelihood conflict. If there is a prior-likelihood conflict then we have bias and have to start again. The latest version of BayesSQUEAC provides a formal test for a prior-likelihood conflict.
I hope this helps.

Normal user

25 Nov 2013, 12:23

Dear Lio,
i am facing the same situation where i have boosters (16) more than the barriers(14) based on the brainstorming exercise. now the question is how to weight them either from 0 to 4 or 0 to 6 or for the maximum one which affecting the coverage. what should be the case for estimating prior.

Brixton Health

Frequent user

25 Nov 2013, 15:01

Following the SQUEAC / SLEAC Technical Reference (p. 77). The maximum weight would be:
```
100 / 16 = 6
```

This is a mechanistic approach as you may have (as in my previous post) a lot of minor barriers and one major barrier that should have a very large weight. Having a maximum weight of 6 will do the job of giving you a candidate value for the prior mode most of the time.

CMAM Advisor

Technical expert

25 Nov 2013, 16:30

Thank you Mark for replying to this; indeed I am in the field with limited access to internet. The idea is always using the higher number (barriers or boosters) to divide the 100; in you case you mention 14 and 16 so you have to use 16 which makes indeed 6. The most important point to understand is, as Mark clearly explained, to be able to give the correct weight of the barrier/booster on coverage; all information collected during the SQUEAC exercise should guide the team to understand which one are the "main" barriers and the "main" boosters; this is important when it comes the time to reform the programme: we have to identify and work on the elements which will have the bigger impact; A scale 1 to 5 (or 6) is large enough to see the difference between the various barriers/boosters and point out the most important to address; having a larger scale (10 or more) will probably make the exercise too heavy and I am not sure that we have enough information to do such detailed analysis. Good luck.