The Construction of Indicators and Its Impact on the Study of Contemporary International Political Phenomena

Indicators are an important tool used by researchers and policymakers to measure and track trends in political, economic, social, and environmental issues. An indicator quantifies and simplifies complex phenomena in order to support decision-making and evaluate progress towards goals. Indicators can be used to monitor government performance, guide policy, benchmark countries, and alert decision-makers to potential problems.

The construction and selection of indicators involves theoretical, methodological, and political considerations. How indicators are constructed can significantly impact how issues are understood and addressed. In recent decades, there has been a huge expansion in the development and use of indicators across all fields of international politics and policymaking. This has been driven by demands for evidence-based policymaking, the data revolution, and the need to measure progress towards national and international development goals.

This article provides an overview of how indicators are constructed, and their growing influence in shaping understanding of contemporary international issues. It highlights the responsibilities and challenges for researchers in developing robust, meaningful indicators. The impact of indicators on different areas of international policymaking is explored, including development, environment, governance, peace and security. The conclusion reflects on improvements in indicator construction and use, as well as risks from reliance on quantification and indicators for decision-making.

Defining Indicators

Indicators can be defined as metrics based on verifiable data that convey information about complex phenomenon. They provide comprehension of where we are, where we are going and how far we are from an objective or endpoint (Saisana and Tarantola, 2002). Indicators simplify and quantify complex realities to support decision-making, track performance, measure progress, and alert actors to potential issues.

The OECD (2008, p.13) defines indicators as “a parameter, or a value derived from parameters, which points to, provides information about, or describes the state of a phenomenon/environment/area, with a significance extending beyond that directly associated with its value”. Indicators provide insight into areas of interest that cannot be directly measured or observed. They should be relevant to the phenomenon and provide a representative picture of reality.

Indicators can be distinguished from other statistical data by their ability to summarize or simplify relevant information. Data only becomes an indicator when its significance extends beyond its direct relevance to policymakers and analysts. The information conveyed can be quantitative or qualitative. Indicators are used when there is a need to compare performance or progress across time, geographic locations, and domains of interest. They allow issues to be quantified so they can be systematically monitored and evaluated.

Characteristics of Good Indicators

Indicators must balance simplicity with accurately representing complex realities. As Boulanger (2008, p.3) notes, indicators “use a manageable number of data to convey as much information as possible”. Several criteria are generally used to assess useful indicators (Dale and Beyeler 2001; Heink and Kowarik 2010):

• Relevance: The indicator measures key aspects of the phenomenon of interest. It is clearly connected to the goals and objectives it measures.
• Validity: The indicator accurately measures the phenomenon it aims to represent.
• Reliability: The indicator produces consistent results when repeated.
• Simplicity: The indicator is easy to interpret and understand.
• Timeliness: Data is available frequently enough to inform decisions.
• Comparability: Common indicators should be used to enable comparison.
• Policy relevance: The indicator informs political decision-making and tracks outcomes of policy.
• Methodological transparency: The methodology for constructing the indicator is clear and replicable.
• Data transparency: The underlying data is accessible and verifiable.
• Construct validity: The methodology aligns with theoretical concepts.
• Sensitivity: The indicator detects changes in the phenomenon being measured.

Good indicators use objective raw data and have clear, replicable methodologies. They balance simplicity in communication with accurately conveying meaning about complex realities. Indicators may focus on inputs (resources), outputs (direct products of activities) or impacts (changes in system state) (Saisana and Tarantola 2002). Input and output indicators are easier to measure but tell less about real-world changes. Impact indicators best represent broader meaning but are harder to construct. Trade-offs between simplicity and representation must be managed in constructing useful indicators.

Approaches for Constructing Indicators

Indicators can be constructed through a range of quantitative and qualitative methods. This section outlines common approaches.

Composite Indicators

Composite indicators aggregate individual indicators into a single index using a theoretical framework. The most well known is the Human Development Index. Composite indicators can compare complex phenomena across countries and summarize multidimensional realities. However, they depend heavily on the methodology and weighting of components.

Composite indicators are constructed using five main steps (Nardo et al. 2005):

1. Develop a theoretical framework to structure indicators into dimensions and sub-dimensions. This determines the indicators that will be combined.
2. Standardize individual indicators to make them comparable. Common techniques include min-max normalization, z-scores, and ranking.
3. Apply weighting to individual indicators if they should not contribute equally to the composite indicator. Weighting depends on theoretical relevance. Equal weighting is most common.
4. Aggregate the individual indicators using mathematical functions. Linear aggregation is simplest but other techniques like geometric aggregation can handle trade-offs between components.
5. Validate the composite indicator and test sensitivity to assumptions. Robustness checks are essential.

Composite indicators require balancing comprehensive representation with risk of “concealing serious failings in some dimensions and amplifying small changes in others” (Bandura, 2005, p.313). The methodology determines how different components of wellbeing are valued and aggregated. Criticisms include lack of transparency, sensitivity of results to methodological choices, and failure to capture systemic interactions between dimensions (Freudenberg, 2003). Composite indicators summarize broad meaning but involve many subjective decisions.

Social Indicators

Social indicators are statistical time series tracking social phenomenon like health, education, employment, and crime. They provide standardized metrics that can be compared temporally and spatially. Social indicators are often used to evaluate social progress and monitor socioeconomic conditions.

In the 1960s, social indicators arose from growing belief that economic indicators like Gross Domestic Product failed to represent quality of life. Social indicators aimed to supplement economic metrics and support social policy evaluation. The U.S. Department of Health, Education, and Welfare published early compilations of social indicators. The OECD played an important role developing standardized social indicators for cross-country comparison.

Social indicators can be categorized into nine broad areas (Carley, 1981):

• Health status
• Public safety
• Social participation
• Income and poverty
• Work and quality of working life
• Education and learning
• Housing and neighbourhood quality
• Leisure and recreation
• Natural environment

Key challenges in using social indicators include developing objective metrics for complex social qualities, ensuring comparability of data across nations, and handling cultural relativism in defining social progress (Noll, 2002). Social indicators continue providing essential data but have faced criticism for attempting to quantify subjective qualities like happiness.

Expert Assessment

Expert assessment indicators draw on skilled judgments to evaluate complex qualitative realities. Experts assign numerical or categorical scores based on systematic assessment frameworks. Expert assessment commonly includes:

• Delphi method: Iterative survey of experts aiming for consensus.
• Peer review: Judges with relevant expertise conduct systematic review.
• Composite measures: Combining data sources with expert judgment.

Examples include Freedom House’s Freedom in the World index, the Corruption Perceptions Index by Transparency International, and the Environmental Performance Index developed at Yale and Columbia Universities.

Expert assessments compensate for lack of reliable quantitative data on many social and political issues. However, subjective biases are a key limitation. Efforts are made to standardize assessment frameworks and use multiple experts to improve reliability. Expert indicators tend to have smaller sample sizes but provide otherwise unavailable insight into complex issues like human rights protection and corruption.

Public Opinion

Public opinion indicators are based on large-scale surveys of individual perceptions and experiences. By aggregating individual views, they capture broad social patterns. Examples include Gallup polls on public confidence in institutions, Pew Global Attitudes surveys about social and political values, and Afrobarometer surveys on lived experiences in Africa. These provide window into public perspectives on quality of governance and social conditions.

Perception-based indicators are important but face validity challenges in accurately representing realities on the ground. Sampling methods and question framing shape results. Cultural biases and subjectivity of self-reported data are limitations. Triangulation with objective indicators provides a more balanced picture. As imperfect proxies, public opinion indicators remain useful in bringing public views into policy debates.

Big Data and New Metrics

The data revolution has enabled developing indicators from new types of unstructured digital data, such as social media, satellite imagery, online searches, mobile phone records, and sensors. Derived indicators include population displacement from satellite images, infectious disease activity from digital disease surveillance, consumer confidence from sentiment on social media, and traffic patterns from location data in phone networks.

These new metrics provide real-time, granular insights complementing traditional indicators. However, biases in big data like social media reflect uneven digital access. New capabilities spark debates on privacy and ethical use. The sheer volume of data requires advanced analytical methods like machine learning. Big data indicators remain a nascent field with great potential if limitations are addressed.

Trends in Indicator Construction

Several key trends have progressed indicator construction, including greater adoption of:

• Expert-based assessment: Compensates for lack of objective data where measurement is difficult.
• Perception-based data: Captures subjective qualities like public opinion through surveys.
• Multidimensional indicators: Composite indicators combining many dimensions of wellbeing.
• Disaggregation: Indicators broken down by geography, demographics and units like gender to uncover disparities.
• Objective benchmarks: Compares country performance on common indicators.
• Quantitative modeling: Relates multiple indicators to reveal systemic interconnections.
• Geospatial data: Geo-located, high-resolution data like satellite imagery to map environmental change.
• Big data sources: Real-time, granular data from online activity and sensors.

This expands possibilities for representing complex realities through indicators. It is driven by greater data availability, computing power, and demand for nuanced insights to guide decision-making.

Uses of Indicators in International Politics and Policy

Indicators now feature prominently across all domains of international politics and policymaking. They are embedded in decision-making processes and agreements. This section reviews uses of indicators across major issue areas like development, environment, governance, and peace.

Development Indicators

Indicators are central to defining and tracking progress on international development. Global development efforts have relied on indicators for:

• UN Millennium Development Goals (MDGs, 2000-2015)
• Sustainable Development Goals (SDGs, 2015-2030)
• Human Development Index by UN Development Program
• World Bank World Development Indicators

These frameworks monitor dozens of internationally standardized indicators on poverty, health, education, gender equity, and sustainability. They enable accountability for development commitments. Disaggregated data identifies which groups are being left behind.

Indicators like GDP per capita, life expectancy, literacy rates, child and maternal mortality have been especially influential in framing development priorities. These simple metrics focus policy attention and become symbolic of progress.

However, development indicators have been critiqued for oversimplifying complex processes into quantified outcomes without capturing real-world change. The indicators themselves start shaping development visions (Merry, 2011). Reductionist indicators struggle to represent interconnected and context-specific development realities.

Environment and Sustainability Indicators

Indicators are widely used for environmental monitoring and sustainability policy, including:

• Greenhouse gas concentrations
• Species population trends
• Forest coverage and land use change
• Water quality and scarcity
• Renewable energy share
• Ecological footprint

Globally comparable and consistent environment indicators are critical for understanding transnational issues like climate change, biodiversity loss, and ecosystem health. They enable monitoring planetary boundaries and limits.

Satellite observation data has expanded environment indicator availability and granularity. However,Indicator choice and measurement conventions embed certain values and ideas of sustainable development (Lorenzoni et al, 2000). Critics argue a narrow set of quantified indicators dominate over more complex understandings of ecosystems and human-nature relations.

Governance Indicators

Indicators are increasingly used to monitor standards and performance in governance. This includes World Bank Worldwide Governance Indicators, Bertelsmann Stiftung Sustainable Governance Indicators, and Varieties of Democracy metrics. Major dimensions tracked include:

• Voice and accountability
• Political stability and violence
• Government effectiveness
• Regulatory quality
• Rule of law
• Control of corruption

Governance indicators aim to measure institutional quality and legitimacy. This data supports reform initiatives and policy conditionality. It facilitates comparative benchmarking across countries. However, governance resists straightforward quantification. Important context is lost reducing complex political processes like democracy to scorecards. The subjective nature of underlying data also faces criticism. Governance indicators feature prominently in policy debates despite these limitations.

Peace and Security Indicators

Measuring peace and conflict relies heavily on indicators like:

• Deaths from organized violence
• Refugee populations
• Terrorism incidents
• Military expenditure
• Weapons imports/exports
• Human rights violations

These are used in frameworks like the Global Peace Index and for monitoring violence prevention goals like those in the UN Sustainable Development Goals. Data comes from a mix of event data, expert coding, and public perception surveys.

Challenges include counting indirect deaths from conflict, inconsistent reporting, and representing complex concepts like positive peace. Lack of data hampers indicator availability in countries suffering conflicts. Macro-level indicators often miss micro-dynamics in societies that shape peace and conflict. Data constraints hinder stronger indicators in this critical area.

Rise of Indicators in Public Policy

The expanded use of indicators across international issue areas reflects several governance trends:

Evidence-based policymaking: Technocratic desire for measurable indicators to inform policy and track outcomes against objectives. This technocratic approach promises neutral, rational and data-driven governance.

New public management: Managerial worldview relying on quantitative metrics, monitoring, evaluation and performance management. Indicators support accountability and benchmarking.

Audit culture: Governance model that emphasizes oversight, formal measurement of outputs, and procedural compliance over professional judgment.

Transnational governance: Need for internationally comparable metrics to coordinate action on global issues. Indicators also enable policy convergence across countries.

These trends all drive demand for formal indicators. However, critics argue this marginalizes unquantified knowledge, imposes reductive frameworks, and advances particular policy agendas under guise of neutral metrics. Use of indicators reflects power dynamics in global governance. The impacts of growing reliance on indicators should be continually assessed.

Methodological and Political Dimensions in Indicator Construction

The methodology used to construct indicators embeds important assumptions, value judgments, and models of causality. Although indicators aim for scientific objectivity, the subjectivity of social science means indicator design involves making many theoretical and methodological choices. These choices ultimately carry political implications by determining how problems are represented.

Methodological Choices and Assumptions

All stages in indicator construction contain methodology choices and assumptions:

What to measure: The selection of metrics structures understanding of the issue. It leaves out issues not amenable to quantification.

Proxy measures: Indicators approximate desired but unmeasurable qualities through proxy measures, which embed judgments on appropriateness.

Data sources: Choice of data sources shapes results, like expert vs. perception surveys. Samples may not represent populations.

Weighting: Weightings applied when aggregating indicators into composite indexes embed subjective judgments of importance.

Standardization: Normalizing raw data into standardized metrics removes context which can distort outcomes.

Aggregation: Mathematical techniques used to combine indicators determine how they interact and substitute. Linear aggregation implies perfect substitutability between components.

Validation: Testing indicator robustness and sensitivity to choices is important but cannot identify all biases.

Decisions made at each stage shape the concepts, models, and meanings represented by the final indicator. Transparency on these methodological choices and limitations is important for interpreting indicators. Even transparency cannot remove the inherent subjectivity in quantifying social phenomena.

Political and Normative Implications

Indicators do not neutrally reflect reality. They shape how issues are problematized and goals are articulated. The authority of numbers further legitimizes these representations, enabling them to define policy agendas (Merry, 2011).

Choices in indicator construction contain implicit values and worldviews:

• What is considered worth measuring reflects value judgments on proper goals.
• Metrics claim neutrality but embed cultural perspectives on how to understand issues.
• Simplification of complex processes loses meaning.
• Aggregation conventions impose models of how dimensions substitute and interact.
• Ratings and rankings embed normative evaluation.

As Rose (1991, p.675) notes “To measure is to know, to quantify is to have power over.” Indicators are not apolitical tools. They may advance particular agendas and shape which policy solutions are considered viable. Use of indicators can be analyzed critically through the lens of power relations in governance processes.

Impacts on Policymaking

The impacts indicators can have on policymaking include:

Issue attention: Indicators highlight certain issues and priorities over others. Availability of consistent data on some topics but not others skews attention and resources.

Policy framing: How indicators represent issues shapes how problems are conceptualized and goals are articulated in policy.

Policy convergence: Standardized indicators encourage policy convergence as countries react to comparative benchmarking. But uniform models may not fit diverse contexts.

Unintended consequences: Indicators can be manipulated and lead to perverse outcomes when targets are pursued at the expense of real progress.

Path dependency: Reliance on indicators creates policy inertia since programs are built around existing metrics. New approaches are neglected if not measured.

Blaming metric not policy: Indicators used in performance management redirect attention towards improving measured numbers instead of solutions.

Over-simplification: Reduction of complex social phenomena to limited quantified indicators can undermine development of holistic policy.

Indicators do not provide a neutral, technocratic basis for public policy. They shape agendas and priorities in ways imbued with subjectivity and power dynamics. This risks problems arising from indicators being positioned as apolitical.

Positivist Critique of Social Indicators

Indicators have been critiqued by positivists for relying too much on subjective qualitative methods at odds with sciences. Positivism argues the social sciences should emulate the natural sciences by focusing on directly observable and quantifiable phenomena tested through experimental methods to uncover predictive laws.

Positivist criticisms of many social indicators include:

• Perception-based data relies on subjective self-reporting rather than verifiable facts.
• Expert assessment incorporates unscientific subjective opinions rather than empirical measurement.
• Composite indicators aggregate potentially incompatible data into single artifacts with no empirical meaning.
• Normalization of raw data into standardized metrics strips away context needed for scientific analysis.
• Weighting schemes are arbitrary assumptions rather than empirically derived.
• Aggregation rules impose untested theoretical models and subjective judgments on combining data.
• Testing for validity and sensitivity of indicators fails to meet Popper’s falsification criteria for rigorous science.

Positivism insists indicators demonstrate strict scientific validity to produce objective knowledge. But social realities involve meaning-making not governed by predictive laws. Positivism undermines indicators for raising awareness, guiding values-based policy, and representing voices of the marginalized.

Interpretivist Critique of Social Indicators

In contrast to positivists, interpretivists criticize indicators for falsely portraying subjective social realities as measurable objective facts governed by universal laws. Key arguments include:

• Meaningful qualities of social experience and culture cannot be reduced to abstract quantified indicators.
• Indicators impose general models that poorly represent unique local conditions and complex system dynamics.
• Focusing governance on narrow measurable indicators overlooks unquantifiable social and ethical values central to wellbeing.
• Benchmarking countries by indicators ignores incommensurable cultural differences in how progress is envisioned.
• People internalize simplified indicators as reflections of truth rather than socially constructed models.
• Indicators only measure phenomena amenable to quantification, marginalizing other forms of knowledge.

For interpretivists, subjective shared meanings, beliefs and motivations cannot be captured by abstract indicators. They risk imposing reductionist frameworks insensitive to local contexts, cultures, values and systems dynamics. Overreliance on indicators undermines pluralistic, ethical and humanistic policymaking.

Towards Reflexive Use of Indicators in Policymaking

Positivist and interpretivist critiques highlight how indicators construct styled representations of social issues that carry normative and political implications.

This does not negate the usefulness of indicators for raising awareness, guiding policy and enabling accountability. But it emphasizes need for reflexivity on the limits and assumptions entailed in quantification of complex social realities (Merry, 2011). Those involved in producing, analyzing and using indicators should maintain critical perspective by:

• Rejecting view of indicators as neutral, objective representations of truth
• Acknowledging and mitigating methodological and data limitations
• Assessing biases, assumptions and framing effects embodied in indicators
• Considering what meaningful issues and perspectives are marginalized by reliance on quantification
• Recognizing how indicators can advance particular agendas and impose external policy models
• Exploring multiple types of indicators and data to develop holistic understanding
• Engaging multiple stakeholder voices in selection and interpretation of indicators
• Combining indicators with contextual qualitative information and local knowledge
• Focusing policy on transforming structures and systems, not just improving indicators
• Evaluating both intended and unintended consequences of using indicators to guide policy

Thoughtful use of indicators should be embedded within inclusive, ethical, and culture-sensitive policy deliberation and implementation. Indicators support but do not substitute for collective political processes and local knowledge in promoting just social progress.

Conclusion

Indicators have become ubiquitous across international politics and policymaking to monitor progress, benchmark performance and inform decisions. This articles has shown how indicators construct styled representations of social reality through choices in measurement, data sources, standardization, aggregation and framing. The methodology and use of indicators is not neutral but imbued with models, assumptions and values which carry political implications.

Growing reliance on indicators for governance is tied to demands for evidence-based policymaking, results-based management, and greater accountability. Indicators enable tracking commitments on global priorities like development, environment, governance, and peace. However, overdependence on simplified metrics risks displacing unquantified knowledge, neglecting unintended consequences, imposing external policy models, and undermining pluralism.

Indicator quality continues improving through better data, methods for composite indexes, disaggregation by social factors, and engagement of local knowledge. But indicators should supplement rather than substitute for inclusive policy deliberation and collective political processes. Indicators support but do not replace humanistic, ethical and context-sensitive policymaking. Their impacts and limitations must be continually evaluated. When carefully used, indicators can play constructive role in creating more just, sustainable and peaceful societies. But reflexivity is essential for indicators to enable, not distort, social progress.

References

Bandura, R. (2005). Measuring country performance and state behavior: A survey of composite indices. United Nations Office of Development Studies (UNODS).

Boulanger, P. M. (2008). Sustainable development indicators: a scientific challenge, a democratic issue. SAPIENS, 1(1).

Carley, M. (1981). Social measurement and social indicators: Issues of policy and theory. George Allen & Unwin.

Dale, V. H., & Beyeler, S. C. (2001). Challenges in the development and use of ecological indicators. Ecological indicators, 1(1), 3-10.

Freudenberg, M. (2003). Composite indicators of country performance: a critical assessment. OECD science, technology and industry working papers, 2003/16.

Heink, U., & Kowarik, I. (2010). What are indicators? On the definition of indicators in ecology and environmental planning. Ecological Indicators, 10(3), 584-593.

Lorenzoni, I., Leiserowitz, A., De Franca Doria, M., Poortinga, W., & Pidgeon, N. F. (2006). Cross‐National Comparisons of Image Associations with “Global Warming” and “Climate Change” Among Laypeople in the United States of America and Great Britain1. Journal of Risk Research, 9(3), 265-281.

Merry, S. E. (2011). Measuring the world: Indicators, human rights, and global governance. Current Anthropology, 52(S3), S83-S95.

Nardo, M., Saisana, M., Saltelli, A., Tarantola, S., Hoffman, A., & Giovannini, E. (2005). Handbook on constructing composite indicators: methodology and user guide. OECD publishing.

Noll, H. H. (2002). Social indicators and quality of life research: Background, achievements and current trends. In Advances in sociological knowledge over half a century. Springer, Boston, MA. 151-181.

OECD (2008). Handbook on Constructing Composite Indicators: Methodology and User Guide. OECD Publishing, Paris. https://doi.org/10.1787/9789264043466-en.

Rose, N. (1991). Governing by numbers: Figuring out democracy. Accounting, organizations and society, 16(7), 673-692.

Saisana, M., & Tarantola, S. (2002). State-of-the-art report on current methodologies and practices for composite indicator development. EUR 20408 EN, European Commission-JRC: Italy.