Secure AI Summit 2024 (Powered by Cloud Native)

Malicious human and AI actors can infiltrate AI supply chains, compromising the integrity and reliability of the resultant AI systems through training data tampering, software or model backdoors, model interference, or new runtime attacks against the model or its hosting infrastructure. This talk examines the importance of securing the data, models, and pipelines involved at each step of an AI supply chain. We evaluate the efficacy of emerging industry best practices and risk assessment strategies gathered from the FINOS AI Readiness Working Group, TAG Security Kubeflow joint assessment, and case studies with air-gapped and cloud-based AI/ML deployments for regulated and privacy-protecting workloads. In this talk, we: - threat model an AI system, from supply chain, through training and tuning, to production inference and integration - implement quantified security controls and monitoring mechanisms for an AI enterprise architecture - mitigate the risks associated with adversarial attacks on AI systems - address compliance and regulation requirements with actionable remediations - look to accelerate AI adoption while balancing minimum viable security measures

The intersection of AI, cybersecurity, and open-source software (OSS) is pivotal for growth and development of companies and society. We discuss the four apparent corners of this intersection to help inform a growing ecosystem: (1) OSS underpins AI systems, and routinely faces security risks. Tools like Scorecard help consumers understand risks in the supply chain of OSS used in AI systems. (2) Furthermore, open-sourcing AI components accelerates OSS growth, requiring secure practices. Tools like sigstore can help secure these newly released open sourced AI components entering the OSS supply chain. (2) AI also revolutionizes OSS security by automating vulnerability management, enhancing development lifecycles. (4) Lastly, AI's role is evolving; it now contributes to OSS, influencing both upstream creation and downstream use, marking a significant shift in open-source development. These four corners and the challenges within are crucial in shaping the future of technology.

Several mission-critical software systems rely on a single, seemingly insignificant open-source library. As with xz utils, these are prime targets for sophisticated adversaries with time and resources, leading to catastrophic outcomes if a successful infiltration remains undetected. A parallel scenario can unfold for the open-source AI ecosystem in the future, where a select few of the most powerful large language models (LLM) are repeatedly utilised, fine-tuned for tasks ranging from casual conversation to code generation, or compressed to suit personal computers. Then, they are redistributed again, sometimes by untrusted entities and individuals. In this talk, Andrew Martin and Vicente Herrera will explain methods by which an advanced adversary could leverage access to LLMs. They will show full kill chains based on exploiting the open-source nature of the ecosystem or finding gaps in the MLOps infrastructure and repositories, which can lead to vulnerabilities in your software. Finally, they will show both new and existing security practices that should be in place to prevent and mitigate these risks.

This presentation will explore the common journey of software development companies in securely adopting AI technology within a cloud-native environment. It will unpack the challenges that development and platform teams face as they integrate AI into their systems. After initial resistance to using external LLM APIs, confidence grew and companies began building solutions with non-public data and open-source models. However, questions persist: Is my new shiny AI/LLM app secure and safe? This session will discuss practical approaches to challenges such as data privacy in Retrieval-Augmented Generation architectures, the complexities of AI-agent architectures where actions are performed across integrated systems, and the general security hardening of AI/LLM applications. We will share insights from our practice, which began by defining a threat model for AI-based systems aligned with the OWASP Top 10 for LLM Applications, and progressed to incorporating solutions into our cloud-native platform. Both offensive and defensive approaches were implemented, including the integration of tools like garak (LLM Vulnerability Scanner) and NVIDIA NeMo Guardrails into our cloud-native stack.