AI-Driven Resilience in Cloud-Native Big Data Platforms Against Cyberattacks

Abstract

As cloud-native big data platforms such as Kubernetes, Apache Spark, and Databricks become the cornerstone of modern digital infrastructure, they increasingly face advanced cyber threats that exploit their complexity, dynamism, and scale. Traditional security mechanisms, which rely on static rules and perimeter defenses, fail to adapt to the ephemeral and distributed nature of these environments. This paper explores how Artificial Intelligence (AI), particularly machine learning (ML), anomaly detection, and reinforcement learning, can augment cyber resilience across cloud-native platforms. A layered AI-augmented architecture is proposed, covering telemetry ingestion, behavioral feature engineering, ML-based detection, and automated response orchestration. A real-world case study from a global retail enterprise demonstrates the practical efficacy of this approach, with measurable improvements in detection latency, false positive reduction, and incident response. Key evaluation metrics and datasets are discussed, alongside limitations such as adversarial AI, data imbalance, and explainability concerns. Finally, future directions including federated learning, graph neural networks, digital twin simulations, and AI-driven zero-trust frameworks are outlined to guide the evolution of proactive, intelligent defense systems in cloud-native infrastructures.