From the viewpoint of malicious actors: Discover how cybercriminals are utilizing AI and taking advantage of its vulnerabilities to infiltrate systems, users, and even other AI applications.

The Truth About Cybercriminals and AI

According to Etay Maor, Chief Security Strategist at a leading company, “AI will not replace humans anytime soon. However, individuals who are adept at using AI are poised to take over roles from those who aren’t familiar with the technology.” He further notes that attackers are also integrating AI into their operations.

Despite this, the discussion surrounding AI’s impact on cybercrime is often inflated beyond reality. Articles frequently alarm readers with sensational claims about AI dangers, using terms like “Chaos-GPT” and “Black Hat AI Tools,” and even asserting they could threaten humanity. Yet, these write-ups tend to evoke fear rather than deliver a nuanced portrayal of actual risks.

In clandestine online discussions, many purported “AI cyber tools” turned out to be merely repackaged versions of basic public LLMs without any sophisticated features. Frustrated users even criticized them as fraudulent.

The Actual Use of AI by Hackers

In reality, cybercriminals are still learning how to deploy AI effectively. They face similar challenges as legitimate users, such as hallucinations and limited functionalities. Experts predict it may take several years before they can truly utilize generative AI for hacking purposes.

Currently, generative AI tools are mostly employed for simpler tasks, like crafting phishing emails and generating code snippets to facilitate cyberattacks. Furthermore, there have been instances where attackers input compromised code into AI systems for evaluation, attempting to “normalize” such code as non-malicious.

AI Misuse: The Emergence of GPTs

GPTs, which were launched by OpenAI recently, enable users to create tailored versions of ChatGPT by adding specific instructions and external APIs while integrating various knowledge sources. This allows developers to create specialized applications, like customer service bots and educational resources, and offers monetization opportunities through a marketplace.

Exploiting GPTs

However, GPTs come with their own set of security risks. One major concern is the potential exposure of confidential instructions, proprietary information, or API keys embedded in customized GPTs. Malicious individuals could employ techniques such as prompt engineering to replicate a GPT and exploit its monetization functions.

Attackers may use prompts for various purposes, such as asking the customized GPT to list uploaded files or debugging information, or even to produce downloadable links for specific files.

“Even safeguards implemented by developers can be bypassed, allowing for complete extraction of knowledge,” advises a Threat Intelligence Researcher from a leading security firm.

To mitigate these risks, users should:

• Avoid uploading sensitive information
• Implement instruction-based protections, though these may not be fully secure. “It’s crucial to consider all potential attacker strategies,” remarks the expert.
• Rely on protections offered by OpenAI.

Risks and Threats from AI

Multiple frameworks are available to aid organizations looking to develop AI-based software:

• NIST Artificial Intelligence Risk Management Framework
• Google’s Secure AI Infrastructure
• OWASP Top 10 for Large Language Models
• OWASP Top 10 for LLM Applications
• The recently launched MITRE ATLAS initiative.

Vulnerabilities in LLMs

There are six primary components of Large Language Models that cybercriminals may target:

1. Prompt – Manipulations like prompt injections that affect the AI’s output.
2. Response – Improper use or leakage of sensitive data in responses generated by AI.
3. Model – Theft or manipulation of the AI model itself.
4. Training Data – Infusion of harmful data to change the AI’s behavior.
5. Infrastructure – Attacks on the servers and services supporting the AI.
6. Users – Misleading or exploiting individuals relying on AI outputs.

Examples of AI Risks

To illustrate the potential dangers of AI manipulation, here are several examples:

• Manipulating Customer Service Chatbots – A car dealership utilized an AI chatbot for customer inquiries, which a researcher managed to manipulate into agreeing to overly favorable conditions for purchasing a vehicle.
• Legal Repercussions from AI Errors – Air Canada faced lawsuits after its AI chatbot provided erroneous information regarding refund policies, ultimately leading to liability issues.
• Leaking Confidential Information – Employees at Samsung inadvertently shared sensitive data when they employed ChatGPT for code analysis. Uploading confidential material to third-party AI services poses significant risks.
• AI Used in Fraud Schemes – Cybercriminals deployed deepfake technology in a high-stakes heist, convincing a bank in Hong Kong to transfer $25 million using fabricated visual representations of trusted officials.

Conclusion: The Impact of AI on Cyber Crime

AI serves as a potent weapon in the arsenal of both defenders and attackers in the cyber realm. As malicious actors continue to explore AI’s capabilities, gaining insight into their thought processes, tactics, and challenges is essential for organizations aiming to protect their AI systems from exploitation.

For more insights, consider watching a comprehensive masterclass on AI risks.

If you found this article informative, it is a contributed piece from a trusted partner. Stay connected with us for more exclusive content on various platforms.