Karnataka govt. unveils digital grievance portal for gig workers

 The allegation that many of India’s official nation-wide surveys are based on unsound data collection frameworks is misplaced

how sound data collection procedures in India ?

There was controversy recently over how sound data collection procedures in India are, especially when it comes to some of the important national level surveys. 

An article (by Shamika Ravi, Member of the Economic Advisory Council to the Prime Minister)

1  which was published in a leading national daily on July 7, 2023, 

2 cast doubts on the soundness of data collection procedures of some of the surveys such as the 

• National Sample Survey (NSS), 
• National Family Health Survey (NFHS) 
• and Periodic Labour Force Survey (PLFS). 

3 ‘a major sampling overhaul’ 

The writer suggest ‘a major sampling overhaul’ so that the survey estimates reflect the ground reality in the country. 

4 In support of her argument, the writer said that from 2011-12 till 2019-21, out of 11 surveys listed in her article, 

‘every survey (except NFHS-4 of 2015-16) underestimates the proportion of the urban population or overestimates the rural population significantly’. 

5 Consequently, the estimates from these surveys ‘systematically underestimate the improvements across the country’.

 इस बात पर फिर से ज़ोर देना ज़रूरी है कि सूचीबद्ध सभी सर्वेक्षण वैज्ञानिक नमूना डिज़ाइन को अपनाते हैं।

1 It is important to re-emphasise the point that all the surveys listed do adopt scientific sample designs. 

2 However, there can be no denying the fact that there is always a scope for improvement of the sample designs. 

3 deliberations in the meetings 

In fact, the sample designs of the NSS have undergone periodic revisions after due deliberations in the meetings of the NSS round-specific working groups and in last 

with final approval by the National Statistical Commission (and by the governing council of the National Sample Survey Office earlier).

 These committees and bodies have been chaired by or have had as their members some of the most eminent economists, statisticians and demographers that India has ever produced.

Bias in population estimation

under astimation or over astimation of population 

1 On the issue of an underestimation of the proportion of the urban population or an overestimation of the proportion of rural population in surveys (as pointed out by the writer in her article on July 7), 

2 not aimed at estimating the number of households or population 

it is important to remember that the sample designs of the NSS or the PLFS are not aimed at estimating the number of households or population. 

3 Instead, they are meant primarily to estimate the major socio-economic indicators that relate to the subjects of interest. 

4 The estimates of the number of households or population are auxiliary information. 

5 using projected population figures based on the Census 

Data users appropriately adjust the survey-based estimates separately for the rural and urban areas by using projected population figures based on the Census. 

6 based on the NSS 

•  rates and ratios relating to the major survey characteristics for rural or urban areas based on the NSS, broadly reflect the ground realities. 

• This fact has been acknowledged by the National Statistical Commission (Rangarajan, 2001).

• estimates of the number of households based on the NSS are in close agreement with the Census-based number of households. 

 

outdated sampling ? or not  :

1  first, these surveys primarily depend on the population census

list of villages and towns/urban blocks (available once in 10 years only) for sampling purpose which, in any case, is complete in coverage. 

2 And second, for sampling of urban blocks

 the NSS and PLFS use the latest list of Urban Frame Survey (UFS) blocks 

(i.e., the counterpart of the list of urban census enumeration blocks) 

covering all the towns of the country — this partially corrects the frame for urbanisation taking place subsequent to the census by way of State government notifications. 

On the issue of rural-urban classification of geographical areas, all these surveys treat census towns as part of the urban sampling frame.

Systematic bias in response rate

अंतरराष्ट्रीय स्तर पर इसी तरह के सर्वेक्षणों में भी इसी समस्या का सामना करना पड़ता है। हालाँकि, सर्वेक्षण पद्धति ऐसे परिवारों को यथासंभव हद तक समान परिवारों द्वारा प्रतिस्थापित करने का प्रावधान करती है।

Room for improvement

नमूना डिज़ाइन और डेटा गुणवत्ता एक सर्वेक्षण के दो अलग-अलग घटक हैं। दोनों महत्वपूर्ण हैं. जब नमूना डिज़ाइन की बात आती है, तो इन सर्वेक्षणों में वैज्ञानिक नमूना डिज़ाइन को अपनाकर आम तौर पर बहुत सावधानी बरती जाती है।

Sample design and the data quality are two different components of a survey

Both are important. When it comes to sample design, a lot of care is taken generally by adopting a scientific sample design in these surveys. 

However, on the issue of sampling frame, given the apprehension over inadequate representation of rich households, it may be worth exploring whether a list of such households can be developed by tapping alternative sources and covering a representative sample of them in conjunction with the traditional survey of the residual population.

it is good to examining the coverage of the UFS frame, given the extent of underestimation of the urban population. Setting up a methodological study unit to undertake other similar studies oriented towards improvement of the survey design may also be a step in the right direction.

what can we do ? 

The aspect of 

• training of field personnel, 
• field inspection, 
• concurrent data validation 
• and publicity measures may be strengthened to improve the quality of primary data, which is most crucial in any survey.

Finally, while there is always scope for the improvement of the survey results

 criticising all large-scale official surveys on the ground/basis that they do not capture improvements adequately is akin to throwing the baby out with the bathwater.