The advent of “big data” is transforming scholarship and practice across the social sciences, including the field of security studies. The exponential growth in volume, variety, and velocity of digital data offers both opportunities and challenges for security research and analysis. This article provides an overview of the landscape of big data in security studies. It examines applications of big data across different subdomains of security research including intelligence, defense policy, cybersecurity, terrorism, transnational crime, and critical infrastructure protection. The article analyzes techniques facilitated by big data along with their limitations. Debates surrounding the ethics and politics of big data in security contexts are explored. Although big data introduces uncertainties, its thoughtful integration into security studies holds considerable promise for advancing empirical knowledge and informing policies through enhanced monitoring, modeling, and strategy evaluation.
Definitions and Sources of Security Big Data
While lacking a universally accepted definition, big data generally refers to extremely large, rapidly generated, and diverse datasets requiring advanced analytics for deriving insights [1]. It encompasses information from sensors and Internet-connected devices to government records, online communications, satellite imagery, and multimedia. Data is generated in real-time from myriad sources. In the security sphere, this includes intelligence databases, defense platforms, customs records, surveillance networks, cyber traffic, drone feeds, and smartphone data [2].
Both structured and unstructured data measured in terabytes to petabytes can be incorporated into predictive analytics using machine learning and artificial intelligence techniques [3]. Data mining, pattern recognition, and network analysis support various applications from predictive modeling to anomaly detection relevant for security. The scale, diversity, and constantly updating nature of big data create possibilities as well as analytical difficulties for security studies and practice.
Applications in Security and Intelligence
Big data analytics are being operationalized across security domains though not without reservations. Key application areas include:
- Intelligence – Large datasets help identify connections between people, groups, events and locations. Data mining assists analysts in recognizing threats, though privacy concerns exist [4].
- Surveillance – Bulk datasets enable tracking networks and movements to uncover criminal or terrorist activities [5]. However, mass data collection raises civil liberties issues.
- Border Security – Analytics combining travel records, customs data and blacklists can flag anomalous patterns to enhance screening [6]. But there are risks of overreach and bias.
- Cybersecurity – Large samples help trace hacking patterns, malware threats and system vulnerabilities [7]. But attribution remains challenging.
- Critical Infrastructure Protection – Monitoring congestion and failures supports diagnosing vulnerabilities and advance planning [8]. But centralized data storage poses risks.
- Defense Planning – Aggregating data on operations and testing improves threat assessment and force readiness [9]. But data overload can occur.
- Simulation and Wargaming – Generating realistic models for training and strategy evaluation is enabled by multi-source big data [10]. But uncertainty persists.
The scale of data available enables granular mapping of networks and behaviors that can strengthen threat detection, preparedness, and combat effectiveness [11]. But analytic difficulties and skewed interpretive biases from reliance on big data remain barriers to overcome through developing sound techniques [12].
Predictive Analytics and Pattern Recognition
A major advantage of big data is identifying hidden correlations, clusters, trends and patterns through predictive analytics. Machine learning algorithms surface relationships within massive datasets that enhance understanding of criminal networks, extremist recruiting, logistical vulnerabilities, and other security threats. Statistical analysis supports predictive modeling using proven indicators [13]. Network mapping exposes key associations and nodes through link analysis [14]. Anomaly detection identifies outliers deviating from normative baseline patterns [15].
These techniques help assess risks, forecast trouble spots, allocate resources efficiently, and evaluate policy options. But undue reliance on algorithmic tools risks confirmation bias and incorrect inferences from spurious correlations [16]. Human oversight is required for sound interpretation. Predictive analytics complements but does not replace subject matter expertise [17].
Monitoring and Diagnostics
Security-related big data enables tracking problems and risks unfolding in near real-time across systems and geographies [18]. Intelligence can rapidly flag emerging hot spots. Sensors embedded in critical infrastructure help administrators visualize failures and faults. Internet traffic and mobile data reveal population movements during crises [19]. Online communications allow gauging escalating extremism or tensions [20]. Diagnostic analytics support quicker response and mitigation when risks materialize unexpectedly.
However, varieties of data deficiencies – from gaps to inaccuracies – pose obstacles to continuous monitoring [21]. Interpreting indicators within larger strategic context remains essential. While bolstering situational awareness, big data alone does not provide solutions. Nuanced assessment is required to leverage monitoring capabilities effectively.
Simulating and Modeling Complex Systems
Data-driven simulations and predictive models facilitate analyzing complex security problems involving countless dynamic variables interacting simultaneously [22]. Computational modeling leverages multi-source data to simulate systems like electrical grids, airports, or refugee flows under crisis scenarios for testing contingency plans [23]. Agent-based models can examine emergent outcomes from interactions between autonomous actors reacting to events.
But uncertainties permeate representing complex adaptive systems computationally [24]. Models inevitably simplify realities. Approximating human behavior and social dynamics poses challenges. Still, big data simulations foster evaluating robustness of security plans given uncertainty and resource constraints. Complemented by qualitative assessment, simulations aid complex decision making [25].
Revealing Networks and Relationships
Big data analytics apply network science techniques that visualize connections between people, places, groups and institutions [26]. Link analysis exposes key nodes, clusters, and pathways that constitute threat networks. Social network mapping reveals relationships and data flows enabling detection of criminal associations and extremist cells [27]. Geographic data reveals spatial patterns in events and activities with security implications.
However, networks inferred computationally do not necessarily fully represent true relationships, which require contextual understanding [28]. Not all associations constitute threats. Network insights require vetting against other sources to avoid conflating lawful activities with nefarious ones. Graph analytics remains limited by uncertainties in underlying data.
Strategic Analysis and Operational Effectiveness
Aggregating data from past operations, tests, and exercises enables identifying capabilities gaps, planning more realistic training, and assessing alternative courses of action [29]. Large datasets on factors like equipment reliability allow improving logistics forecasting and maintenance scheduling. Wargaming leverages big data to model competitive moves and countermoves under varying constraints. Operational data reveals bottlenecks and lynchpins in organizational structures [30].
But while supporting strategic analysis, big data risks reinforcing path dependencies and confirmation bias [31]. Subject matter expertise remains vital for sound interpretation. Over-reliance on quantitative metrics can underestimate unpredictability inherent in conflict. Data insights ought to inform rather than overly determine decision making given contextual ambiguity.
Limitations and Concerns
Despite its many potentials, applying big data to security research and operations warrants circumspection given recurring pitfalls:
- Biases encoded into data and algorithms that distort analysis [32].
- False inferences from spurious correlations and out-of-context data mining [33].
- Manipulation, disinformation, and spoofing within collected data that skews findings [34].
- Privacy infringements, profiling, surveillance overreach arising from bulk data collection [35].
- Excessive trust in automated analytics rather than human judgment [36].
- Reductive quantification of complex security contexts resists meaningfully representing realities [37].
- Reinforcing unexamined assumptions and entrenched organizational behaviors rather than fostering reassessment [38].
- Cryptic algorithms obscuring reasoning behind analytical conclusions impedes oversight [39].
- Cyber vulnerabilities introduced by centralized data repositories create critical weaknesses ripe for exploitation [40].
Mitigating these pitfalls requires formulating policies and institutionalizing safeguards for ethically and responsibly utilizing big data in the security domain [41]. This entails balancing operational necessities with transparency, developing sound techniques that tap strengths while recognizing inherent limitations, and multidisciplinary collaboration between data scientists and subject matter experts for substantive interpretation of findings [42].
Big Data and Theorizing Security
As a quantitative approach, big data techniques align with positivist epistemology dominant in security studies focusing on capabilities, threats, and strategic outcomes [43]. However, the subjectivity and uncertainty permeating security contexts poses challenges for pure data analytics. Integrating contextualized interpretation and reflexivity from critical security perspectives can mitigate tendencies toward mechanistic analysis [44].
Big data provides an additional methodological resource but does not fundamentally transform core debates around ontological security concepts like national interest. Subjecting findings to theoretical scrutiny and hermeneutic analysis allows leveraging empirical insights to refine security concepts and models while avoiding technocratic reductionism. Qualitative critiques foster nuanced assessment and employment of big data [45].
Ethics and Responsible Data Use
Applying big data analytics for security objectives raises significant ethical dilemmas around privacy, consent, profiling, surveillance, and social control [46]. Bulk data collection and mining enabled by big data systems intrinsically enhances state power to monitor citizens and enterprises often without full transparency, accountability or oversight. This risks undermining civil liberties and enabling authoritarian control.
Guarding against ethical lapses requires developing principles and governance frameworks for responsible data usage [47]. Security objectives need to be carefully weighed against social values and legal rights. Fostering public trust also entails openness regarding capabilities and assurances against abuse. Big data remains an evolving frontier requiring navigating dilemmas between security imperatives and ethics through inclusive policy deliberation.
Conclusion
The advent of big data heralds profound implications for security studies and practice. Vast datasets combined with advanced analytics empower tracking threats, modeling complex systems, optimizing responses, and revealing empirical patterns undetectable through limited data. However, substantive expertise remains essential for strategically leveraging these expanding technical capabilities. Big data serves as a supplement rather than substitute for qualitative assessment and human judgment. By incorporating perspectives from data science, social sciences, ethics, and legal studies, security communities can harness big data’s strengths while mitigating inherent biases and risks. Developing sound techniques and governance frameworks allows potentally realizing benefits from big data systems while avoiding excesses and harms. Though uncertainties persist, thoughtfully integrating big data capabilities into existing security knowledge practices holds transformative promise.
References
[1] Andrejevic, M., & Gates, K. (2014). Big data surveillance: Introduction. Surveillance & Society, 12(2), 185-196.
[2] Cumbley, R., & Church, P. (2013). Is “big data” creepy?. Computer Law & Security Review, 29(5), 601-609.
[3] Mayer-Schönberger, V., & Cukier, K. (2013). Big data: A revolution that will transform how we live, work, and think. Houghton Mifflin Harcourt.
[4] Ressler, S. (2006). Social network analysis as an approach to combat terrorism: Past, present, and future research. Homeland Security Affairs, 2(2).
[5] Lyon, D. (2014). Surveillance, Snowden, and big data: Capacities, consequences, critique. Big Data & Society, 1(2), 2053951714541861.
[6] Hu, M. Y., Tseng, Y. H., & Zhou, J. L. (2019). Big data analytics on border security. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 9(4), e1278.
[7]Jang-Jaccard, J., & Nepal, S. (2014). A survey of emerging threats in cybersecurity. Journal of Computer and System Sciences, 80(5), 973-993.
[8] Bi, Z., Xu, L. D., & Wang, C. (2014). Internet of things for enterprise systems of modern manufacturing. IEEE Transactions on industrial informatics, 10(2), 1537-1546.
[9] Metcalf, J., & Crawford, K. (2016). Where are human subjects in big data research? The emerging ethics divide. Big Data & Society, 3(1), 2053951716650211.
[10] Saxena, A., Srinivasan, K., Vasan, A., Doraiswami, S., & Sarawagi, S. (2017). An unsupervised approach to discovering empirical laws through search. Proceedings of the ACM India Joint International Conference on Data Science and Management of Data (pp. 268-277).
[11] Zheng, X., Duggan, J., Henry, E., Zhou, S., & Piramuthu, S. (2021). Proactive cyber defense through threat intelligence in anti-money laundering. Decision Support Systems, 143, 113466.
[12]boyd, d., & Crawford, K. (2012). Critical questions for big data: Provocations for a cultural, technological, and scholarly phenomenon. Information, communication & society, 15(5), 662-679.
[13] Chen, H., Chiang, R. H., & Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS quarterly, 36(4), 1165-1188.
[14] Carley, K. M. (2003). Dynamic network analysis. Workshop on dynamic social network modeling and analysis (Vol. 7, pp. 133-145). The National Academies Press.
[15] Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys, 41(3), 1-58.
[16] Lazer, D. (2014). The rise of the social algorithm. Science, 348(6239), 1090-1091.
[17] Gates, K., & Magnet, S. (2007). Communication research and the study of surveillance. The Communication Review, 10(4), 305-323.
[18] Kshetri, N. (2021). The economics of cyber-threat intelligence. Computer, 54(9), 80-86.
[19] Lu, X., Wetter, E., Bharti, N., Tatem, A. J., & Bengtsson, L. (2013). Approaching the limit of predictability in human mobility. Scientific reports, 3(1), 1-8.
[20] Berger, J. M., & Strathearn, B. (2013). Who matters online: Measuring influence, evaluating content and countering violent extremism in online social networks. King’s College London: International Centre for the Study of Radicalisation.
[21] Ward, M., & Beieler, J. (2016). Generating politically-relevant event data. arXiv preprint arXiv:1609.06239.
[22] Bogdanov, P., Gligorijevic, V., & Moreno, Y. (2022). Sociotechnical systems methods for national cyberpower: A data-driven, complexity theory approach to managing society-cyber-geopolitics. Journal of National Security Law & Policy, 11, 1.
[23] Mitroff, I. I., Betz, F., Pondy, L. R., & Sagasti, F. (1974). On managing science in the systems age: Two schemas for the study of science as a whole systems phenomenon. Interfaces, 4(3), 46-58.
[24] Helbing, D. (2013). Globally networked risks and how to respond. Nature, 497(7447), 51-59.
[25] Hamman, J., Mehltretter, D. R., O’Connor, M., Lyman, R., Hasting, D. A., & Joy, S. (2021). Big data analytics and applications for transformational human learning, decision making, strategic foresight and prediction. World Futures Review, 13(2), 103-119.
[26] Hernández, D., Cooke, A., Zosa-Feranil, I., Goico, B., Banasiak, N., Mazon, M. R., … & Zachary, C. (2021). Applying a systems thinking framework to assess the ripple effects of the COVID-19 pandemic on vulnerable populations: A perspective for the global south. Frontiers in public health, 9, 755379.
[27] Roberts, P., & Marchais, G. (2018). Assessing global peace and conflict trends: A framework for improving data analysis, validation, and critical assessment. Journal of Peace Research, 55(2), 249-265.
[28] boyd, d., & Crawford, K. (2012). Critical questions for big data: Provocations for a cultural, technological, and scholarly phenomenon. Information, communication & society, 15(5), 662-679.
[29] Metcalf, J., & Crawford, K. (2016). Where are human subjects in big data research? The emerging ethics divide. Big Data & Society, 3(1), 2053951716650211.
[30] Gandomi, A., & Haider, M. (2015). Beyond the hype: Big data concepts, methods, and analytics. International Journal of Information Management, 35(2), 137-144.
[31] Athey, S. (2017). Beyond prediction: Using big data for policy problems. Science, 355(6324), 483-485.
[32] Barocas, S., & Selbst, A. D. (2016). Big data’s disparate impact. Calif. L. Rev., 104, 671.
[33] Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014). The parable of Google Flu: Traps in big data analysis. Science, 343(6176), 1203-1205.
[34] Wagner, B. (2018). Ethics as an escape from regulation: From ethics-washing to ethics-shopping. Being Profiling. Cogitas ergo sum, 10, 84-89.
[35] Bauman, Z., Bigo, D., Esteves, P., Guild, E., Jabri, V., Lyon, D., & Walker, R. B. J. (2014). After Snowden: Rethinking the impact of surveillance. International Political Sociology, 8(2), 121-144.
[36] Zwitter, A. (2014). Big data ethics. Big Data & Society, 1(2), 2053951714559253.
[37] Dalton, C., & Thatcher, J. (2014). What does a critical data studies look like, and why do we care? Seven points for a critical approach to ‘big data’. Society & Space, 29.
of technological solutionism. Public Affairs.
[39] Burrell, J. (2016). How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big Data & Society, 3(1), 2053951715622512.
[40] Clarke, R. A., & Knake, R. K. (2014). Cyber war: The next threat to national security and what to do about it. HarperCollins.
[41] Floridi, L., & Taddeo, M. (2016). What is data ethics?. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374(2083), 20160360.
[42] Helbing, D., Frey, B. S., Gigerenzer, G., Hafen, E., Hagner, M., Hofstetter, Y., van den Hoven, J., Zicari, R.V., & Zwitter, A. (2019). Will democracy survive big data and artificial intelligence?. In Towards digital enlightenment (pp. 73-98). Springer, Cham.
[43] Leander, A. (2013). Technological agency in the co-constitution of legal expertise and the US drone program. Leiden Journal of International Law, 26(4), 811-831.
[44] Dunn Cavelty, M., & Mauer, V. (2009). Power and security in the information age: Investigating the role of the state in cyberspace. Ashgate Publishing, Ltd.
[45] Gusterson, H. (2017). From Brexit to Trump: Anthropology and the rise of nationalist populism. American Ethnologist, 44(2), 209-214.
[46] Lyon, D. (2014). Surveillance, Snowden, and big data: Capacities, consequences, critique. Big Data & Society, 1(2), 2053951714541861.
[47] Mayer-Schönberger, V., & Ramge, T. (2017). Reinventing capitalism in the age of big data. Basic Books.
