Age Ratio Check in SMART

Dear Julian,

The test used to analyze the age ratio in plausibility check ENA report is a Chi².
You would notice that the expected ratio is not 1 (as sex ratio) but 0.8 considering 24 months (=6-29 month) / 30 months (=30-59 month) = 0.8.
So the Chi² is testing if the sample's age ratio is near 0.8 as expected.

The original idea of the SMART methodology is to permit Nutritionist or Health worker to perform a strong robust cross-sectional survey (with international standard results) with basic knowledge on epidemiology and statistics. So, you don't necessarily need to understand the whole statistical fundaments under plausibility check report.

For better understanding of data quality process, see:
ANNEX 7.1: SMART: Ensuring data quality – is the survey result usable? by M. Golden
From training toolkit on smartmethodology.org

Hope this is helpful

Hi Julian

The expected proportion of children for the Plausibility Check were compiled by Michael Golden based on extensive age distribution research as noted by Damien.

ENA computes the expected number of children for each month of the total population based on these expected proportions.

By looking at the expected proportions of children, multiplying them by the months at risk (12 months in the first four age categories, and 6 months in the last one), and then dividing them according to the age ratio grouping (6-29/30-59 months).

Looking at the table below, if 1,000 children are expected in the 6-17 month age group. Due to infant and child mortality 975 children are expected in the 18-29 months

Age Group Options in ENA

Expected Proportion of children for plausibility check of sampling
]
Age Groups (m) Male Female
6 - 17 ----------------- 1000, 1000
18 - 29 ---------------- 975, 975
30 - 41 ----------------- 945, 945
42 - 53 ----------------- 930, 930
54 - 59 ---------------- 920, 920

Thus, for the different age ranges we get the following:

6-17 = 12 months. 1,000*12=12,000.
18-29 = 12 months. 975*12=11,700.
30-41 = 12 months. 945*12=11,340.
42-53 = 12 months. 930*12=11,160.
54-59 = 6 months. 920*6=5,520.

Based on the above, we expect to have an age ratio of 6-29 months/30-59 months = 12000 + 11700 / 11340 + 11160 + 5520 = 23,700/28,020 = 0.8458 ˜ 0.85.

Where do the expected numbers of children expressed in the statistical test for sex and age ratios come from?

When adding the whole proportions obtained for each age ranges above, we obtain:
12000 + 11700 + 11340 + 11160 + 5520 = 51720, or 100% of an “ideal” sample.

For each age range, we expect to obtain the following proportions:
6 – 17 months: 12000 / 51720 ˜ 23.2% (expressed as a simple proportion, it is 23.2/100 = 0.232)
18 – 29 months: 11700 / 51720 ˜ 22.6% (0.226)
30 – 41 months: 11340 / 51720 ˜ 21.9% (0.219)
42 – 53 months: 11160 / 51720 ˜ 21.6% (0.216)
54 – 59 months: 5520 / 51720 ˜ 10.7% (0.107)

Check: 23.2% + 22.6% + 21.9% + 21.6% + 10.7% = 100%.

Hope this is helpful.

Thanks

Hi All,

The Overall Sex Ratio and Overall Age Distribution are the two key Plausibility Check criteria useful in assessing the representativeness of a survey sample (detection of any selection bias) in relation to the expected age/sex distribution of the U5 population.

The values in ENA for SMART Options Tab have been set on the assumption of having equal number of boys and girls (aged 6-59 months) in a stable population size with similar birth rate and a death rate of about 1/10,000/day. Following extensive analyses of mortality trends across various countries, these figures in the Options Tab portray an expected exponential decrease in mortality for children from 0 to 59 months. For sake of comparison between surveys within and across countries, these figures should not be adjusted when generating the Plausibility Check.

A significant difference in the Overall Sex Ratio and Overall Age Distribution (the chi-square test shows a p-value <0.05), may be indicative of either:

1) Sampling problem, or

2) Significant deviation from normal demographics for this population (birth, death and differential migration rates)

These tests shouldn’t be solely relied upon to invalidate a survey, but rather be intuitively utilized to draw attention on the possible causes of deviation from what is expected based on the survey context at hand. Further details on the SMART Plausibility Check Chapter can be found here: http://smartmethodology.org/survey-planning-tools/smart-methodology/

Thanks

The comments about SMART are incorrect in this thread as in nearly all previous threads concerning SMART. Again, there are key issues which are wrong or misrepresented.

The zero to 5 mortality rate is of course counted from BIRTH and not from 6 months of age. This means that 5q0 and the 5q0.5 will be completely different. Because the deaths that occur in the first 6 months of life have been neglected in the posts made the calculations are simply wrong. If the data from www.childmortality.org and Wang et al (Age-specific and sex-specific mortality in 187 countries, 1970-2010: a systematic analysis for the Global Burden of Disease Study 2010. The Lancet 380, 2071-94, 2012) are consulted you will find that globally about 50% of deaths are neonatal and a further 30% post-neonatal (1 to 12months). In sub-Saharan Africa because the 5q1 mortality is increased the relative proportion of deaths that are neonatal falls to about 30%. Naturally, the mortality from 6 to 59 months will be substantially less than the mortality from birth to 59 months; but the difference is less in developing than in developed countries. To do the calculations correctly it is necessary to start at birth, abstract that part of the curve that encompasses the 6 to 59 month old children and then average this over the age ranges of interest.

As many of the surveys are conducted in areas under stress we have used the following equation to calculate the rate of loss of children as 10,000 -364.4xLN(day). The data are adjusted to having 1000 children in the first age range. The mortality rate from 6 months to 59.9 months using this equation is 0.616/10,000/d.

We do NOT recommend REJECTION of a survey on the basis of any one criterion as repeatedly asserted. There needs to be several parameters where a survey’s results deviate from what is normally found with a well conducted survey before it accumulates sufficient points to be “problematic”. Problematic surveys then require that the report justifies the deviation, before acceptance.

Although there are more males than females at birth (51-52:48-49), the mortality rate of males is higher than females from birth (globally 5q0 mortality ratio male:female is about 56:44), progressively evening up the sex ratio. Thus the assumption of equal numbers of boys and girls during the period 6-59months is justified. Well conducted surveys nearly all have about equal numbers of boys and girls providing empirical confirmation. Your theoretical reasons for a “legitimate” change in the sex ratio sufficient to alter analysis are not justified empirically. Again, in the very few countries where sex ratio in children 6-59 months deviates meaningfully from 1:1 the surveys would not be penalized for age distribution in the sample if it’s in the expected direction and is justified by reliable country demographic data.

There is one typo in the default age distribution numbers on ENA Options page– the second figure (for the age group 18-29 months) should be 965 and not 975. It will be corrected in the next updated version of ENA. Correspondingly, the expected age ratio of 6-29 months to 30-59 months in the Plausibility check report will be adjusted from 0.85 to 0.84. These changes will have a trivial effect on which surveys would be flagged for justification because of the age distribution in the sample.

This will conclude SMART input on the topic.

Results of population census results of Ethiopia (2007) shows the following proportion of child population for each age band:

Ethiopia
Age band Number %
0-11.9 months 1,775,454 16%
12-23.9months 1,964,606 18%
24-35.9months 2,294,205 21%
36-47.9months 2,263,614 21%
48-59.9months 2,499,143 23%
10,797,022

Tigray Region
Age band Number %
0-11.9 months 119,603 19%
12-23.9months 113,812 18%
24-35.9months 128,436 20%
36-47.9months 136,531 22%
48-59.9months 132,480 21%
630,862


Afar Region
Age band Number %
0-11.9 months 14,291 10%
12-23.9months 21,581 15%
24-35.9months 32,306 23%
36-47.9months 31,872 22%
48-59.9months 42,327 30%
142,377

Amhara region
Age band Number %
0-11.9 months 428,329 18%
12-23.9months 446,651 19%
24-35.9months 476,654 20%
36-47.9months 484,432 21%
48-59.9months 501,857 21%
2,337,923


Oromia Region
Age band Number %
0-11.9 months 732,201 16%
12-23.9months 825,025 19%
24-35.9months 978,883 22%
36-47.9months 911,247 20%
48-59.9months 1,003,961 23%
4,451,317


Somali Region
Age band Number %
0-11.9 months 31,626 7%
12-23.9months 62,901 14%
24-35.9months 116,666 26%
36-47.9months 85,482 19%
48-59.9months 153,024 34%
449,699

SNNP Region
Age band Number %
0-11.9 months 370,781 16%
12-23.9months 413,107 18%
24-35.9months 467,578 20%
36-47.9months 522,512 22%
48-59.9months 567,277 24%
2,341,255