Call: No more perceptions, reach for the facts, in: Civis Mundi Digitaal #91, november 2019 door Michel van Hulten
In a few weeks Transparency International’s Secretariat in Berlin will publish the 2019 edition of the annual Corruption Perceptions Index (CPI). In order to prepare for surprises in the new results, I analyse the results as published in the 2018-CPI. How valid are the qualifications found? See for the full TI-report: https://www.transparency.org/cpi2018
One general remark is easily made in advance without any hesitation as since its inception in 1995, the CPI does not deal with ‘facts’, but with ‘perceptions’, rightfully not concealed in its name since 1996. The CPI does not qualify countries on the basis of known facts. And it says so explicitly. The CPI reproduces what people think is the reality.
In other words, the CPI deals with the ‘gut feelings’ of selected individuals about the social and economic behavior of people / decisionmakers / consumers in their environment. The relationship between these ‘gut feelings’ and real life, real actions and activities of those observed (on whom is reported), is unclear (to say the least).
A second general remark is also easy to give and maintain as an opinion. It is that the top rankings in the list of ‘perception of corruptness’ of countries (or is it of their populations (?) are scored by modern, rich countries (most of them in Western Europe). Scores booked at the bottom end of the list (more than half way down a list of close to 180 countries) are mostly African, in any case are ‘underdeveloped economies’. Europe figures among all those countries named in the bottom half of the ranking only with (rank number) 138 Russia, 120 Ukraïna, 117 Moldova, 105 Armenia, 99 Albania, 93 Macedonia, 93 Kosovo.
Never ever a country – once listed in the top quarter of the ranks – went down to the second half of the ranking list, or conversely, never ever a country from the bottom quartile went up to the top half of the list. In other words, the listing is quite stable. Ranking changes are minimal.
Nevertheless, the annual data are always offered with enthusiastic comments by the reporting entity about those changes as clear results of the fight against corruption; improvements which should impress the readers; us. Or reflect deteriorations in behavior, policies, public administration which have occurred recently.
As an example it was interesting to read the enthusiastic article published by the TI-Secretariat about the gain of 7 points by Gambia between 2017 and 2018 on which I reflected in nr. 85 of Civis Mundi (June 2019), see The Gambia’s seven-point improvement in CPI comes after end of decades-long autocratic rule →https://www.civismundi.nl/?p=artikel&aid=5211 by Kaunain Rahman.
I quote from the article:
“Overcoming corruption Challenges
While still far below the global average of 43 out of 100, with a score of 37 out of 100 on the latest Corruption Perceptions Index (CPI) The Gambia has gained seven points since 2017. Although in general we should not read too much into a single year-on-year change, even of this magnitude, the increase reflects significant changes in the way the country is tackling corruption and strengthening democracy.
From 1996 to 2017, the small and densely populated West African nation of The Gambia suffered under the violent and repressive regime of its president, Yahya Jammeh. Systemic corruption and kleptocracy crippled private enterprise and robbed the Gambian people of vast sums, undermining an already fragile and shock-prone economy.
In 2017, the year Jammeh finally left office after losing an election which he then declared void, the country scored just 30 out of 100 on the Corruption Perceptions Index, and ranked 130 out of 180 countries.
A year later, The Gambia is inside the top 100 countries for the first time.”
So, what has made the difference? According to the article it is the fall from his official function and flight from The Gambia of the former President Yahya Jammeh, and the restauration of democracy. As the President went into exile only 21 January 2017, it is quite unlikely that this single event in 2017 caused the changes in scores and the rising in rank of Gambia on the TI-list in such a short time-span. If perceptions have any value they are persistent and do not change in some months with so many as happened in The Gambia. What else did cause these changes? I do not know either.
Selection of sources?
In this analysis I will try to prove that the results as presented in the ranking under the title ‘Corruption Perceptions Index 2018’ are rather doubtful as the sources used for making the amalgam of the results coming from the 13 sources that have been used, do not stand scientific proof of their validity.
I do not enter in seeking an answer to the question why it is that in the course of the years since 1995, not every year the same sources are used but that some drop out and new ones enter.
I will also not ask why it is that precisely these sources, like the 13 used in 2018 and analysed below, have been chosen as building blocks for the TI ranking list? Why these and not others? Are these 13 of superior quality? The reporting coming annually from Berlin also does not inform us whether other sources have been accessed and considered as possible building blocks. Neither why eventually they have not been admitted in establishing the CPI? There could have been good reasons anyhow.
Corruption? Various subjects?
I will show in the analysis below that the 13 sources do not deal all with the same subject of research (which nevertheless is named equally in all sources ‘corruption’ as if it is one subject). Moreover the data do not stem all from the year 2018 (worse: nearly nothing in the data stems from 2018, but is older, back as far as even 2012). The researchers involved are quite varied in their personal capacities. Those subjected to research show a similar variety but are also uniform: lack of women, lack of blue collar workers, academic or at least business school education and/or experience in the top layers of the business world, Northwestern Europeans or their likes from countries like Canada, Australia, New Zealand, Hong Kong, Singapore.
In some sources we find explicitly mentioned that the results are not what the researchers found but that what is published in the sources, is presented after interventions and declarations of acceptability only after some sort of ‘overseeers’ have overlooked the preliminary results. However, nowhere is mentioned that the results as presented after those responsible for the published results agreed, have been falsified.
13 Sources
I quote the introduction from the TI-report ‘CPI-2018’:
“Since its inception in 1995, the Corruption Perceptions Index, Transparency International’s flagship research product, has become the leading global indicator of public sector corruption. The index offers an annual snapshot of the relative degree of corruption by ranking countries and territories from all over the globe. In 2012, Transparency International revised the methodology used to construct the index to allow for comparison of scores from one year to the next. The 2018 CPI draws on 13 surveys and expert assessments to measure public sector corruption in 180 countries and territories, giving each a score from zero (highly corrupt) to 100 (very clean).”
Note that in this Introduction from the TI-report the CPI is self-qualified as ‘global indicator of public sector corruption’. Are the ‘perceptions’ upgraded to ‘facts’? Also it is clearly said that the indicator does not include similar behavior in the private sector. Are all the researchers involved in making the 13 sources, and all respondents in the various field researches executed for these sources, aware of this distinction in offering their ‘perceptions’ as data? What does this mean for the ‘petty corruption’ as occurs in offering places in schoolclasses for pupils and offering beds in hospitals for the sick. What if private institutions are compared to state-run schools and hospitals? Bribes paid in private institutions are not included in the ‘perceptions’? This seems to me is highly improbable. Do perceptions include bribe-paying in order to fill up a vacancy in the police-force (by definition public sector) if nepotism (a family-affair) plays a role? The applicant may pay his uncle (a private sector action?), but what has this to do with the public institution ‘police-force’? Is this private or public corruption? And how is this seen by the general public or by specialist-researchers?
13 data sources were used to construct the Corruption Perceptions Index (CPI) 2018
- African Development Bank Country Policy and Institutional Assessment 2016
- Bertelsmann Stiftung Sustainable Governance Indicators 2018
- Bertelsmann Stiftung Transformation Index 2017-2018
- Economist Intelligence Unit Country Risk Service 2018
- Freedom House Nations in Transit 2018
- Global Insight Business Conditions and Risk Indicators 2017
- IMD World Competitiveness Center World Competitiveness Yearbook Executive Opinion Survey 2018
- Political and Economic Risk Consultancy Asian Intelligence 2018
- The PRS Group International Country Risk Guide 2018
- World Bank Country Policy and Institutional Assessment 2017
- World Economic Forum Executive Opinion Survey 2018
- World Justice Project Rule of Law Index Expert Survey 2017-2018
- Varieties of Democracy (V-Dem) 2018
Each one of these will be scrutinized with regard to the socio-economic phenomenon ‘corruption’.
First question: Corruption?
Do these sources explicitly deal with ‘corruption’, and subsidiary with ‘perceived corruption’, or do they deal with substitutes for ‘corruption’?
Source 1
African Development Bank Country Policy and Institutional Assessment 2016 – CPIA
Corruption questions
Experts are asked to assess: “Transparency, accountability and corruption in the public sector”.
“This criterion assesses the extent to which the executive can be held accountable for its use of funds and the results of its actions by the electorate and by the legislature and judiciary, and the extent to which public employees within the executive are required to account for the use of resources, administrative decisions, and results obtained. Both levels of accountability are enhanced by transparency in decision-making, public audit institutions, access to relevant and timely information, and public and media scrutiny. National and sub-national governments should be appropriately weighted.”
“Each of three dimensions are rated separately:
(a) the accountability of the executive to oversight institutions and of public employees for their performance
(b) access of civil society to information on public affairs
(c) state capture by narrow vested interests
For the overall rating, these three dimensions receive equal weighting.
Scores : The rating scale ranges from 1 (very weak) to 6 (very strong).
Country coverage: 54 African countries are covered.
The data is publicly available online at https://cpia.afdb.org/?page=data “
Comments
The questions do not aim directly for ‘corruption’.
The overall CPIA is assessed in 5 clusters covering Economic Management, Structural Policies, Social Inclusion / Equity, Governance, Infrastructure & Regional Integration. All 5 together make 18 distinctions stretching from ‘Fiscal Policy’ to ‘Regional Integration’. Of these 5 clusters and 18 distinctions, only the ‘Governance’ cluster makes one distinction on corruption among its five constituent parts as ‘Transparency, Accountability and Corruption in the Public Sector’. That is 1/18th or 1/3rd of 1/18th of the CPIA?
Moreover, in the CPI-2018 is mentioned that this source covers 54 African countries while in fact the website of the CPIA gives data only for 37 African countries. Missing are a. o. South-Africa, Botswana, Angola, Mozambique and all countries bordering the Mediterranean from Morocco to Egypt. This is correct as the CPIA only presents data on IDA-entitled countries (a World Bank criterion). Why is the figure ‘54’ given?
Conclusion: the CPIA is not specifically a source giving data about ‘(perceptions of) Corruption’; the coverage in number of countries is far less than mentioned in the CPI-2018 report.
Not mentioned in the TI report on the CPI-2018 is that the same source – the World Bank CPIA – is used for this source 1 and for source 10 (see below) which both contribute to the CPI-2018, as here analysed. Unnoted remains also the remarkable difference between the two that the African Development Bank (source 1) used the CPIA-2016 edition, and the World Bank (source 10) used the CPIA-2017 edition. Moreover, the AfDB quotes from the CPIA-report 18 distinctions in 5 clusters (source 1), and the WB quotes 16 criteria grouped in 4 clusters (source 10). Nothing in the CPI-2018 report of Transparency International shows that the authors are aware of these discrepancies and of the double use of the same source.
Source 2
Bertelsmann Stiftung Sustainable Governance Indicators 2018
Corruption question(s)
Experts are asked to assess: Corruption prevention
“D4.4 To what extent are public officeholders prevented from abusing their position for private interests?”
“This question addresses how the state and society prevent public servants and politicians from accepting bribes by applying mechanisms to guarantee the integrity of officeholders: auditing of state spending; regulation of party financing; citizen and media access to information; accountability of officeholders (asset declarations, conflict of interest rules, codes of conduct); transparent public procurement systems; effective prosecution of corruption.”
Scores are given from:
a low of 1, where “Public officeholders can exploit their offices for private gain as they see fit without fear of legal consequences or adverse publicity.”
to a high of 10, where “Legal, political and public integrity mechanisms effectively prevent public officeholders from abusing their positions.”
The questionnaire can be accessed here: http://www.sgi-network.org/2018/Questionnaire
Comments
Note that ‘Experts are asked to assess: Corruption prevention’ and not explicitly ‘Corruption’.
See also the more detailed analysis of source 2 as I made and published in Civis Mundi nr 88, August 2019 → https://www.civismundi.nl/index.php?p=artikel&aid=5372 under the title CPI 2018: My summary and evaluation of this one source out of the 13 used.
Source 3
Bertelsmann Stiftung Transformation Index 2018
Corruption questions
Experts are asked to assess:
“Q3.3 To what extent are public officeholders who abuse their positions prosecuted or penalized?”
Assessments range from:
a low of 1, where “Officeholders who break the law and engage in corruption can do so without fear of legal consequences or adverse publicity.”
to a high of 10, where “Officeholders who break the law and engage in corruption are prosecuted rigorously under established laws and always attract adverse publicity.”
“Q15.3 To what extent does the government successfully contain corruption?” Assessments range
from a low of 1, where “The government fails to contain corruption, and there are no integrity mechanisms in place.”
to a high of 10, where “The government is successful in containing corruption, and all integrity mechanisms are in place and effective.”
The data is publicly available online at http://www.bti-project.org/en/home/.
Comments
Note that the experts are asked not about ‘corruption’ but about ‘repression of corruption’.
Source 4
Economist Intelligence Unit Country Risk Service 2018
Corruption questions
Specific guiding questions include:
Are there clear procedures and accountability governing the allocation and use of public funds?
Are public funds misappropriated by ministers/public officials for private or party political purposes?
Are there special funds for which there is no accountability?
Are there general abuses of public resources?
Is there a professional civil service or are large numbers of officials directly appointed by the government?
Is there an independent body auditing the management of public finances?
Is there an independent judiciary with the power to try ministers/public officials for abuses?
Is there a tradition of a payment of bribes to secure contracts and gain favours?
Scores
Scores are given as integers on a scale from 0 (very low incidence of corruption) to 4 (very high incidence of corruption).
The score is a generalised composite measure of corruption that includes an assessment of all areas covered by the indicative questions.
Data is available to subscribers of EIU Country Risk Service → http://www.eiu.com
Comments
For an outsider like me it is difficult to judge whether and how the EIU deals with ’Corruption’ as long as this source is not open. The ‘guiding questions’ as mentioned above deal in any case with the public domain, and wording indicates proxies of ‘(Perceptions of) Corruption’. It is interesting to note that in the so-called ‘guiding questions’ not one contains the word corruption (indeed one has ‘bribe’), while under the heading ‘Scores’ qualifications are given as ‘incidence of corruption’.
Source 5
Freedom House Nations in Transit 2018
Corruption questions
The Freedom House experts are asked to explore a range of indicative questions, including:
Has the government implemented effective anti-corruption initiatives?
Is the country’s economy free of excessive state involvement?
Is the government free from excessive bureaucratic regulations, registration requirements, and other controls that increase opportunities for corruption?
Are there significant limitations on the participation of government officials in economic life?
Are there adequate laws requiring financial disclosure and disallowing conflict of interest?
Does the government advertise jobs and contracts?
Does the state enforce an effective legislative or administrative process—particularly one that is free of prejudice against one’s political opponents—to prevent, investigate, and prosecute the corruption of government officials and civil servants?
Do whistleblowers, anti-corruption activists, investigators, and journalists enjoy legal protections that make them feel secure about reporting cases of bribery and corruption?
Are allegations of corruption given wide and extensive airing in the media?
Does the public display a high intolerance for official corruption?
Scores
Ratings run from 1 (lowest level of corruption) to 7 (highest level of corruption) and allow for half-point and quarter-point intermediate scores (e.g. 3.25).
The score is a generalised composite measure of corruption that includes an assessment of all areas covered by the indicative questions.
More information can be found here: https://freedomhouse.org/report/nations-transit-methodology
The data is publicly available → https://freedomhouse.org/report/nations-transit/nations-transit-2018
Comments
“The reports focus on democratic progress and setbacks”. “Corruption” is only one of seven thematic areas listed as “national democratic governance; electoral process; civil society; independent media; local democratic governance; judicial framework and independence; and corruption”.
Source 6
Global Insights Business Conditions and Risk Indicators 2017
Corruption questions
Experts are asked to assess:
“The risk that individuals/companies will face bribery or other corrupt practices to carry out business, from securing major contracts to being allowed to import/export a small product or obtain everyday paperwork. This threatens a company’s ability to operate in a country, or opens it up to legal or regulatory penalties and reputational damage.”
“The data for CPI 2018 from IHS Global Insight was accessed through the World Bank World Governance Indicators portal, as IHS Global Insight stopped providing data to Transparency International in 2015. This can be accessed through: http://info.worldbank.org/governance/wgi/index.aspx#doc-sources
Detailed data is also available to customers of IHS’ Country Intelligence: http://www.ihs.com/products/global-insight/country-analysis/ “
Comments
In the TI-report is not given the reason why IHS Global Insight stopped providing data to Transparency International in 2015, which now had to be found at the World Bank portal.
The World Bank Worldwide Governance Indicators (WGI) project “reports [ … ] for six dimensions of governance: Voice and Accountability; Political Stability and Absence of Violence; Government Effectiveness; Regulatory Quality; Rule of Law; Control of Corruption”. Note: only one dimension among six deals with ‘Corruption’ and then in a limited sense: only with the ‘control’. This is not an open source. ‘Detailed data’ are only available to “customers of IHS’ Country Intelligence”.
Source 7
IMD World Competitiveness Yearbook 2018
Corruption question
Survey respondents were asked:
“Bribery and corruption: Exist or do not exist”.
Scores
Answers are given on a 1 to 6 scale which is then converted to a 0 to 10 scale where 0 is the highest level of perceived corruption and 10 is the lowest. https://www.imd.org/uupload/imd.website/wcc/Survey_Explanation.pdf
https://worldcompetitiveness.imd.org/
https://www.imd.org/wcc/research-methodology/
Comments
This source is very direct in approach of ‘corruption’: does corruption exist?
This is not an open source. Data is available to customers of IMD WCY, package of online services.
Source 8
Political and Economic Risk Consultancy Asian Intelligence 2018
Corruption question
“How do you grade the problem of corruption in the country in which you are working?”
Scores
Answers to the question were scaled from 0 (not a problem) to 10 (a serious problem).
The data is available to subscribers under http://www.asiarisk.com/subscribe/exsum1.pdf
Comments
A direct approach of the subject ‘corruption’. Alas, data only accessible to subscribers.
Source 9
The PRS Group International Country Risk Guide 2018
Corruption questions
“This is an assessment of corruption within the political system. The most common form of corruption met directly by businesses is financial corruption in the form of demands for special payments and bribes connected with import and export licenses, exchange controls, tax assessments, police protection, or loans. The measure is most concerned with actual or potential corruption in the form of excessive patronage, nepotism, job reservations, exchange of favours, secret party funding and suspiciously close ties between politics and business.”
Scores
The corruption scores are given on a scale of 0 (highest potential risk) to 6 (lowest potential risk) on a monthly basis. Half points are given. The monthly ratings are then aggregated using a simple average to create a single country score.
Data is available to customers of the PRS International Country Risk Guide at www.prsgroup.com
Comments
A direct approach of the subject ‘corruption’. Alas, data only accessible to subscribers.
Source 10
World Bank Country Policy and Institutional Assessment 2017
Corruption questions
Experts are asked to assess: “Transparency, accountability and corruption in the public sector”.
“This criterion assesses the extent to which the executive can be held accountable for its use of funds and the results of its actions by the electorate and by the legislature and judiciary, and the extent to which public employees within the executive are required to account for the use of resources, administrative decisions, and results obtained. Both levels of accountability are enhanced by transparency in decision making, public audit institutions, access to relevant and timely information, and public and media scrutiny. A high degree of accountability and transparency discourages corruption, or the abuse of public office for private gain. [Why italics, see below]. National and sub-national governments should be appropriately weighted.”
Each of three dimensions are rated separately:
(a) accountability of the executive to oversight institutions and of public employees for their performance;
(b) access of civil society to information on public affairs; and
(c) state capture by narrow vested interests.
For the overall rating, these three dimensions receive equal weighting. http://pubdocs.worldbank.org/pubdocs/publicdoc/2015/6/559351435159340828/cpia14-webFAQ14.pdf
The data is publicly available online at: http://data.worldbank.org/data-catalog/CPIA
Comments
See first the ‘comments’ as given above under ‘source 1’.
The questions do not aim directly for ‘corruption’.
I add here that I find it remarkable that under ‘Questions asked’ under source 1 is missing the phrase
“A high degree of accountability and transparency discourages corruption, or the abuse of public office for private gain.” This is remarkable as this phrase is present in the similar quote from the World Bank-report, see above in this paragraph under ‘Corruption questions’ in italics.
Source 11
World Economic Forum (WEF) Executive Opinion Survey 2018
http://reports.weforum.org/global-competitiveness-report-2018/appendix-b-the-executive-opinion-survey-the-voice-of-the-business-community/
Corruption questions
Survey respondents were asked:
“In your country, how common is it for firms to make undocumented extra payments or bribes connected with the following:
- a) Imports and exports
- b) Public utilities
- c) Annual tax payments
- d) Awarding of public contracts and licenses
- e) Obtaining favourable judicial decisions ”
More details on the Executive Opinion Survey can be found online under:
http://reports.weforum.org/global-competitiveness-report-2018/appendix-b-the-executive-opinion-survey-the-voice-of-the-business-community/
and
This survey-source presents 148 questions under 15 sections. The Report presents four ‘overarching components’ (Enabling environment, Human capital, Markets, Innovation ecosystem) with 12 pillars.
All this in order to answer the main question: ‘What is competitiveness?’ as “the set of institutions, policies and factors that determine the level of productivity of a country”.
For the purpose of my analysis of the sources used for establishing the CPI-2018 only a few of the 148 questions are directly relevant, all three in pillar 1.
In ‘Pillar 1, Institutions, para 1.01, Business costs of organized crime’, the response to the survey question “In your country, to what extent does organized crime (mafia-oriented racketeering, extortion) impose costs on businesses?” (I do this because the word ‘extortion’ is often used in business circles as their word for ‘corruption’; it shifts the burden of guilt from one to the other participant in a quid pro quo action, from the payer to the payee).
Also in ‘pillar 1, 1.14, Incidence of corruption’, as outcome is used the score of the country as is the outcome in the Corruption Perceptions Index, source: Transparency International.
(I note here the circularity that TI uses for the CPI the findings of the WEF and this WEF survey makes use of the CPI constructed by TI).
The 3rd question is less relevant for this analysis as it restricts the concept of ‘conflict of interest’ in a rather narrow sense i.e. “the protection of shareholders against directors’ misuse of corporate assets for personal gain.”
The Report and an interactive data platform ‘The Global Competitiveness Report 2018’ are available at www.weforum.org/gcr.
Comments
It is clear that for clarification of the position of ‘corruption’ in society as is the aim of the CPI, the WEF Survey has a minimal value. Corruption is a small element in its methodology. I do not really understand why the results of this Survey are being used for the establishment of the CPI by TI.
Source 12
World Justice Project Rule of Law Index 2017-2018
Corruption questions
Index 2: Absence of corruption
“A total of 53 questions are asked of experts on the extent to which government officials use public office for private gain. These questions touch on a variety of sectors within government including the public health system, regulatory agencies, the police, and the courts.
Individual questions are aggregated into four sub-indices:
2.1 Government officials in the executive branch do not use public office for private gain
2.2 Government officials in the judicial branch do not use public office for private gain
2.3 Government officials in the police and the military do not use public office for private gain
2.4 Government officials in the legislature do not use public office for private gain
Only the scores provided by the experts were considered for the CPI calculations. The four sub-indicators are then averaged to create a single score.
The data is publicly available online under http://data.worldjusticeproject.org/ . For CPI calculations, the disaggregated expert survey data is provided to Transparency International by the WJP.
Comments
The main purpose of this World Justice Project is to answer the question: “how the rule of law is experienced in practical, everyday situations by the general public worldwide?”
Note that “only the scores provided by the experts were considered for the CPI calculations.” Why the results gained from “the households” are neglected for the CPI remains unclarified. In my view this makes the WJP-data less valuable for the CPI. Why TI makes this choice is not clarified.
Source 13
Varieties of Democracy Project 2018
Corruption questions
“Question: How pervasive is political corruption? (v2x_corr)
The directionality of the V-Dem corruption index runs from less corrupt to more corrupt (unlike the other V-Dem variables that generally run from less democratic to more democratic situation). The corruption index includes measures of six distinct types of corruption that cover both different areas and levels of the political realm, distinguishing between executive, legislative and judicial corruption. Within the executive realm, the measures also distinguish between corruption mostly pertaining to bribery, and corruption due to embezzlement. Finally, they differentiate between corruption in the highest echelons of the executive (at the level of the rulers/cabinet) and in the public sector at large. The measures thus tap into several distinguishable types of corruption: both petty and grand; both bribery and theft; both corruption aimed at influencing law making and that affects implementation.
Aggregation: The index is arrived at by taking the average of (a) public sector corruption index (b) executive corruption index (c) the indicator for legislative corruption and (d) the indicator for judicial corruption. In other words, these four different government spheres are weighted equally in the resulting index.”
“V-Dem draws on theoretical and methodological expertise from its worldwide team to produce data in the most objective and reliable way possible. Approximately half of the indicators in the V-Dem dataset are based on factual information obtainable from official documents such as constitutions and government records. The remainder consists of more subjective assessments on topics like democratic and governing practices and compliance with de jure rules. On such issues, typically five experts provide ratings for the country, thematic area and time period for which they have information.
To address variation in coder ratings, V-Dem works closely with leading social science research methodologists and has developed a state of the art Bayesian measurement model that, to the extent possible, addresses coder error and issues of comparability across countries and over time. V-Dem also provides upper and lower point estimates, which represent a range of probable values for a given observation. When the ranges of two observations do not overlap, there is relative confidence that difference between them is significant. V-Dem is continually experimenting with new techniques and soliciting feedback from experts throughout the field. In this sense, V-Dem is at the cutting edge of developing new and improved methods to increase both the reliability and comparability of expert survey data. V-Dem also draws on the team’s academic expertise to develop theoretically informed techniques for aggregating indicators into mid- and high-level indices. “
V-Dem data to be accessed through: https://www.v-dem.net/en/data/data-version-8/
and the codebook is available at:
https://www.v-dem.net/media/filer_public/64/ad/64ad9308-45fa-473e-8e2b-e1c0c4e421e6/v-dem_codebook_v8.pdf
Comments
This source deals with ‘political corruption’ and offers hundreds of access points. I did not succeed in finding what elements in this research and reporting do directly contribute to the establishment of the CPI 2018. Under 5.7 we find ‘Governance indicators’ specified in ‘5.7.1 Control of Corruption – estimate’ and ‘5.7.2 Control of corruption – standard error.
Summary answer at the first question posed above
Summarizing my remarks made above about the use of the concept and the word ‘Corruption’, I conclude my answer on the first question as follows.
1. Out of the 13 sources, those numbered 4, 6, 7, 8 and 9, are not open sources. You have to be a member or subscriber to get access to ‘data’ and or ‘detailed data’(see source 6). To say the least, it is remarkable that an institution that calls itself ‘Transparency International’ lacks at this point so much transparency.
- Specifically giving scores and rankings on ‘(perceptions of) corruption’ in only a few of the sources can be found. Most of the sources deal predominantly with other subjects in which at best ‘corruption’ is embedded (see e.g. source 5), on the contrary see e.g. source 7 in which ‘corruption’ is very directly approached with the question: “Bribery and corruption: Exist or do not exist”.
- Source 1 and 10 use data from the same research report (CPIA) but from different years (2016, 2017). This is not mentioned in the CPI-report. Why was this double usage acceptable? Why was it not mentioned?
- Some sources (see e.g. 2) deal explicitly with prevention of corruption, others (see e.g. 3) with repression. All results seem to be of equal value for the CPI-outcome. A very broad approach is chosen in source 12 where ‘absence of corruption’ is covered with 53 questions.
In the opening section of this analysis I posed as ‘First question’:
“Do these sources explicitly deal with ‘corruption’, and subsidiary with ‘perceived corruption’, or do they deal with substitutes for ‘corruption’?”
It is clear from reading all 13 sources that the answer must be that a direct approach of corruption is rather missing and that the subject is covered mostly by asking questions and collecting answers that deal with substitutes for ‘corruption’.
More doubts
More doubts about the validity of the CPI reports, in this case from the Year 2018, can be listed. I did so already in my article “CPI 2018: comments, questions, validity?” in: Civis Mundi, #82, April 2019 (II), see https://www.civismundi.nl/index.php?p=artikel&aid=5069 .
Also in that article the interested reader will see my suggestion to add as a source for the CPI by TI the Mo Ibrahim Index of African Governance, http://iiag.online/ and as an additional source the report produced by the TI-Secretariat annually, Title: Exporting Corruption, progress report 2018 Assessing enforcement of the OECD Anti-Bribery Convention, by Gillian Dell and Andrew McDevitt, ISBN of full report: 978-3-96076-097-9. See:
https://www.transparency.org/files/content/publication/Download_a_short_version_of_the_report.pdf
In that April article in Civis Mundi I mentioned my forthcoming articles in following editions of e-Civis Mundi and that I would focus my analysis of the CPI 2018 on four subjects:
1. Who does the work in collecting and processing the data used in classifying the countries?
This article was published in Civis Mundi Digitaal #84, mei 2019 (II). Title: “CPI 2018: Whose perceptions, and who does the work of collecting and processing the source data?” See: https://www.civismundi.nl/index.php?p=artikel&aid=5178
My conclusion in that article was:
“All the foregoing together leads me to the conclusion that we get a ‘perception of the presence of corruption’ as worded by a small bandwith of experts in this field, mostly male, white collar workers, between 35 and 55, with MBA or doctorate in economics or finance, in leading business or academic-research positions, often temporary residents in the country from which they report, employed by companies who make their living from studies in this field. It is not the ‘ordinary people’ involved in the work of these reviewers and investigators. Also, if their perceptions are captured in the right way, and are processed into an amalgam which represents their views correctly, still there is no way in which we can confirm that these perceptions are the same as we would get if we directed those same questionnaires to women, youth, blue collar workers, lower educated levels in society, the poor, expatriates, in short the powerless, etc. No one will doubt that asking trade-unionists about corruption will deliver other results than are received asking CEO’s and professors in business administration. Eventually, why should we pay more attention to the perceptions of those belonging to the first selection and not to the second?”
For the details on which this conclusion is based, please turn to the article which gives details per source.
2. How precise is ‘2018’? When have data been collected and processed?
According to the text of the CPI-2018 report would most CPI-2018-data originate from 2017 or the early months of 2018. This is also what I reported in my article “When? ‘2018’? And what about The Gambia?”, see https://www.civismundi.nl/index.php?p=artikel&aid=5211. The article on Gambia which I used is authored by Kaunain Rahman, I am responsible for thye cooments and questions.
However, it is too superficial working to rely on the tekst as is in the source. If the source is consulted deeper, much earlier dates show up.
The basic data TI got from the 13 sources and used for the aggregated figures in the 2018 CPI, stem from calender dates between at least from Sept. 2016 to maximum August 2018. But if you enter the data as far as possible in the CPI-2018 report, you will find that the oldest input dates as far back as 2013. Moreover, sources 1 – 6, 10 and 12 ended input for CPI 2018 before the year 2018 began.
Of all sources 11 out of 13 have as ‘Year of Publication’ 2018, nrs 1 and 12 have 2017. The total CPI 2018 Report is published by TI International Secretariat on Tuesday 29 January 2019 as: “CPI 2018”.
Data collection time per source in short:
| Number Source |
Dates of collecting and processsing field data |
| 1 | Sept – Nov 2016 |
| 2 | Nov 2016 – Nov 2017 |
| 3 | ……… – 31 Jan 2017 |
| 4 | ……… – Sept 2017 |
| 5 | Jan 2017 – 31 Dec 2017 |
| 6 | 2017 |
| 7 | Febr 2018 – April 2018 |
| 8 | Jan 2018 – Maart 2018 |
| 9 | Sept 2017 – Aug 2018 |
| 10 | 2017 |
| 11 | Jan 2018 – April 2018 |
| 12 | 2017 |
| 13 | ……. |
-
What definition for ‘Corruption’ has been used and which questions have been asked?
This number 3 of the specific subjects which I announced as to be dealt with separately has been the subject of analysis in the first half of this article.
For details please refer to that first half.
- How ‘global’ is the coverage of the report?
Global coverage
- ADB
“54 African countries are covered” is what the CPI-2018 says. But the CPIA website gives data for 37 African countries. Missing are a.o. South Africa, Botswana, Angola, Mozambique and all countries bordering the Mediterranean
- Bertelsmann SGI
- U and OECD countries.
- Bertelsmann Stiftung Transformation Index 2018
- countries and territories are scored.
- Economist Intelligence Unit Country Risk Service 2018
- countries/territories were scored in 2018.
- Freedom House Nations in Transit 2018
- countries/territories were ranked in 2018.
- Global Insights Business Conditions and Risk Indicators 2017
- countries/territories worldwide are scored.
- IMD World Competitiveness Yearbook 2018
- countries/territories around the world were scored in 2018.
- Political and Economic Risk Consultancy Asian Intelligence 2018
- Asia Pacific countries/territories plus the Unites States were surveyed in 2018.
- The PRS Group International Country Risk Guide 2018
The ICRG provides ratings for 140 countries on a monthly basis.
- World Bank Country Policy and Institutional Assessment 2017
- ountries were scored in the CPIA 2017.
- World Economic Forum Executive Opinion Survey 2018
In 2018 the survey captured the views of 12,274 responses from business executives in 140 economies.
- World Justice Project Rule of Law Index 2017-2018
- untries were scored in the 2017 Rule of Law Index.
- Varieties of Democracy Project 2018
177 countries were scored in the 2018 update of the index.
Final
I hope that all readers will agree that my arguments are strong enough so that they can agree with already so many others and with me, that the qualifications given by TI and the resulting ranking order as published under the title Corruption Perceptions Index cannot pass the quality check, and therefore are unreliable and harmful to the fight against corruption. The first and foremost need is for improvements to arrive at a better Index as has been suggested already several times. And TI should not shy away any longer from all the factual knowledge which has been built up since its inception in the last decade of the 20th century. We know now much more than we did in 1993 and 1995. Forget the perceptions, reach for the facts.
*****