Secure AI Summit 2024 (Powered by Cloud Native)

Malicious human and AI actors can infiltrate AI supply chains, compromising the integrity and reliability of the resultant AI systems through training data tampering, software or model backdoors, model interference, or new runtime attacks against the model or its hosting infrastructure. This talk examines the importance of securing the data, models, and pipelines involved at each step of an AI supply chain. We evaluate the efficacy of emerging industry best practices and risk assessment strategies gathered from the FINOS AI Readiness Working Group, TAG Security Kubeflow joint assessment, and case studies with air-gapped and cloud-based AI/ML deployments for regulated and privacy-protecting workloads. In this talk, we: - threat model an AI system, from supply chain, through training and tuning, to production inference and integration - implement quantified security controls and monitoring mechanisms for an AI enterprise architecture - mitigate the risks associated with adversarial attacks on AI systems - address compliance and regulation requirements with actionable remediations - look to accelerate AI adoption while balancing minimum viable security measures

The rapid advancement of Generative AI has lowered the barrier for creating sophisticated malware, making less experienced hackers capable of propagating attacks in a matter of minutes. This new type of threat highlights the need to develop suitable tools to reduce detection time to a similar timeframe. This talk introduces Kestrel as a Service (KaaS), empowering threat hunters with reusable threat hunting flows from the Kestrel language, effortlessly deployable in the cloud. Augmented by predictive AI model plugins, Kestrel optimizes threat detection, accelerating response times in case of attacks. Kestrel provides a layer of abstraction to stop the repetition involved in cyber threat hunting. Kestrel contains two main components, 1) A threat hunting language for a human to express what to hunt and 2) A machine interpreter that deals with how to hunt. The key objective is to use these components to hunt faster.

The intersection of AI, cybersecurity, and open-source software (OSS) is pivotal for growth and development of companies and society. We discuss the four apparent corners of this intersection to help inform a growing ecosystem: (1) OSS underpins AI systems, and routinely faces security risks. Tools like Scorecard help consumers understand risks in the supply chain of OSS used in AI systems. (2) Furthermore, open-sourcing AI components accelerates OSS growth, requiring secure practices. Tools like sigstore can help secure these newly released open sourced AI components entering the OSS supply chain. (2) AI also revolutionizes OSS security by automating vulnerability management, enhancing development lifecycles. (4) Lastly, AI's role is evolving; it now contributes to OSS, influencing both upstream creation and downstream use, marking a significant shift in open-source development. These four corners and the challenges within are crucial in shaping the future of technology.

Several mission-critical software systems rely on a single, seemingly insignificant open-source library. As with xz utils, these are prime targets for sophisticated adversaries with time and resources, leading to catastrophic outcomes if a successful infiltration remains undetected. A parallel scenario can unfold for the open-source AI ecosystem in the future, where a select few of the most powerful large language models (LLM) are repeatedly utilised, fine-tuned for tasks ranging from casual conversation to code generation, or compressed to suit personal computers. Then, they are redistributed again, sometimes by untrusted entities and individuals. In this talk, Andrew Martin and Vicente Herrera will explain methods by which an advanced adversary could leverage access to LLMs. They will show full kill chains based on exploiting the open-source nature of the ecosystem or finding gaps in the MLOps infrastructure and repositories, which can lead to vulnerabilities in your software. Finally, they will show both new and existing security practices that should be in place to prevent and mitigate these risks.

Achieving and maintaining a Zero Trust architecture in cloud-native environments remains a complex challenge. K8sGPT, a cutting-edge AI-powered tool, is revolutionizing system management and streamlining the path to Zero Trust. By providing detailed guidance, integrating with system events, and working alongside tools like Istio and Kyverno, K8sGPT simplifies policy enforcement and network security, empowering operators to implement a robust Zero Trust model confidently.

While AI presents an opportunity to innovate across domains, we are learning that it also presents unknown threat vectors that are constantly evolving. So what does threat-modeling look like for today's AI applications? Some frameworks are emerging like the OWASP LLM risks or MITRE ATLAS framework that lists attack TTPs for AI applications. However these are just baseline frameworks that need customizing for each organization. Furthermore, secure behavior of AI applications needs continuous verification as they are by nature, indeterministic and are often built on top of 3rd party models which are untrusted black boxes. AI apps should be actively breached to test how secure organizational data, IP, and internal APIs are when connected through them, much like the way the resilience of dynamic microservices is actively tested using chaos experiments. This talk will describe how to bring proactive Chaos-testing to AI security using Secops-Chaos - an open source framework that helps encode TTPs as security focused chaos experiments, with hands-on demos of how to map some of the MITRE ATLAS TTPs to AI apps running as containers within Kubernetes environments.

This presentation will explore the common journey of software development companies in securely adopting AI technology within a cloud-native environment. It will unpack the challenges that development and platform teams face as they integrate AI into their systems. After initial resistance to using external LLM APIs, confidence grew and companies began building solutions with non-public data and open-source models. However, questions persist: Is my new shiny AI/LLM app secure and safe? This session will discuss practical approaches to challenges such as data privacy in Retrieval-Augmented Generation architectures, the complexities of AI-agent architectures where actions are performed across integrated systems, and the general security hardening of AI/LLM applications. We will share insights from our practice, which began by defining a threat model for AI-based systems aligned with the OWASP Top 10 for LLM Applications, and progressed to incorporating solutions into our cloud-native platform. Both offensive and defensive approaches were implemented, including the integration of tools like garak (LLM Vulnerability Scanner) and NVIDIA NeMo Guardrails into our cloud-native stack.

Today GRC team struggles to instill the culture of Continuous Control Monitoring. Typically, they utilize mechanisms such as security questionnaire, email or Sharepoint to gather evidence. These aids help them in assessing compliance, preparing for audits and managing vendor risk assessments. However, they encounter difficulties in collecting data and evidence due to lack of standardization, technical complexity, repetitiveness and insufficient time and resources. We can support our hard working GRC teams and equip them the necessary tools by employing LLM in the following way: - Creating a machine readable controls framework in YAML from the policy document. - Generating a dynamic graph of policies, controls and frameworks based on the YAML - Designing a dynamic evaluation questionnaire for users to assess the effectiveness of these policies - Deploying this questionnaire using well known tools like Google forms for continuous controls monitoring - Implementing CEL (Common Expression Language) to calculate the compliance score dynamically based on the evaluation responses. - Integrating the final results into reports and dashboards for the steering governance committee.

Cato Networks has recently released a new data loss prevention (DLP) capability, enabling customers to detect and block documents being transferred over the network, based on sensitive categories, such as tax forms, financial transactions, patent filings, medical records, job applications, and more. Many modern DLP solutions rely heavily on pattern-based matching to detect sensitive information. However, they don't enable full control over sensitive data loss. Take for example a resume of a job applicant. While the document might contain basic PIIs, such as the candidate's phone number and address, it's the application itself that concerns the company's DLP policy. Unfortunately, pattern-based methods fall short when trying to detect the document category. Many sensitive documents don't have specific keywords or patterns that distinguish them from others, and therefore, require full-text analysis. In this case, the best approach is to apply data-driven methods and tools from the domain of natural language processing (NLP), specifically, large language models (LLM).

The Oligo Security team recently identified ShellTorch, a chain of 4 vulnerabilities that allow a full chain of Remote Code Execution (RCE), with a new CVE-2023-43654 having a CVSS score of 9.8, and found tens of thousands of vulnerable instances publicly exposed in Torchserve, which is part of the PyTorch ecosystem (one of the most widely adopted OSS frameworks for AI in the world), open to unauthorized access and insertion of malicious AI models.   In this talk, we’ll dive into the research team’s identification of the TorchServe vulnerabilities enabling a total takeover of impacted systems.  With the growing popularity of AI and LLMs, securing these applications and their tooling stacks is becoming increasingly important.  Come to this session to unpack this newly discovered high-severity exploit from the researchers themselves, which enables the viewing, modifying, stealing, and deleting of AI models and sensitive data on a targeted TorchServe server, with a live demo of its reproduction, and steps you can take immediately to mitigate the risk.