Earlier today the Lowy Institute put out the results of a survey it conducted on “India’s views of the world ahead”. While the report contains some excellent insights (including Indians’ perception of various countries), the problem is that it doesn’t establish what people’s priorities are.
For example, there is a question that asks people how important it is that “India has the largest navy in the Indian Ocean”. Some 94% of respondents think it is important, but neither the question nor the answer acknowledges the cost of being the largest navy in the Indian Ocean. Of course, having the largest navy in the Indian Ocean is a great thing to have, but what about the cost?
This is the problem with “uni-directional surveys” – where questions are independent of each other and no relation between factors is established. For example, everyone wants low taxes, high level of government-sponsored welfare, full employment, good wages and a strong military. The reason differences between political parties occur is because it is impossible to have all of it at the same time, and different parties have different positions on the trade-offs.
Table 24 of the Lowy survey illustrates this. The question is about domestic policy goals, and respondents are asked about the importance of each. Is it of any surprise that over 90% of respondents think each and every one of these goals is important?
In order to capture trade-offs, I propose a different kind of survey. One where the respondent is told “The government suddenly gets an extra Rs. 100 which it has to spend on either strengthening our military or providing food security. What do you choose?”. The survey I propose will have a series of such “binary” questions, where respondents have to allocate the government budget between various programs. That way, the true preferences of the respondents can be captured.
One last point on the presentation of the above table. The survey uses a “4 point Likert scale” (“not at all important”, “not very important”, “fairly important”,”very important”) to record responses. First off, marketing research theory recommends that such scales have an odd number of choices (3 and 5 are the recommended numbers). Secondly, the report has chosen to group the first two choices under “total not important” and the latter two under “Total important”. As you can see from the table, these “total” columns are presented in boldface, thus drawing attention. Consequently, given the amount of information in each table, no one really looks at the columns not in bold face. In other words, the Likert scale could have had only two points (important – not important)!