APPLY FOR PARTNERSHIP
Thank you for your interest. Kindly fill out the details below and we will contact you back soon.
Contact Us
Please complete this form to be contacted by one of our experts.
Throughout this year, there has been much buzz surrounding generative AI tools such as OpenAI ChatGPT and Google Bard. Many new and established tech companies are now integrating generative AI into their solutions. Most companies use the OpenAI Platform through the GPT API or host private OpenAI Large Language Model (LLM) instances on Microsoft Azure. Grammarly’s latest product, Grammarly Go, is an example of the latter.
It’s clear that generative AI is here to stay via text-based LLM tools, image-generation AI tools like Midjourney and Dall-E, and various audio-generative music and voice replication tools. While these technologies will certainly have positive impacts, there is also a risk of bad actors exploiting them for personal gain. In this article, I will highlight some cybersecurity and data security implications of these tools.
Organisation Wide Cybersecurity Directives
Cybersecurity Risks from LLM Tools
The cybersecurity and data risks from LLMs come in four areas — cybercriminals using them to perform more sophisticated attacks — vulnerabilities in LLM-generated code — the leaking of personally identifiable information (PII) — the loss of company secrets.
Improved Phishing and Other Social Engineering Attacks
Since ChatGPT’s release in 2022, cybersecurity researchers and cybercriminals have examined its capabilities to enhance cyberattacks. Demonstrations of the technology reveal how easy it is for anyone to use ChatGPT to generate convincing emails and other texts for phishing schemes.
These more convincing texts for phishing and other social-engineering-based attacks pose threats in multiple ways. For instance, non-native speakers of a particular language can use ChatGPT to create more sophisticated output than they could create themselves or with a tool like Google Translate. Bad actors can use this feature to generate convincing phishing emails in many major languages, even if it’s not their native tongue.
Additionally, criminals can leverage ChatGPT’s chatbot user interface to craft highly targeted phishing attacks by repeatedly asking for modifications to the generated text. This iterative approach is particularly effective in spear-phishing attacks aimed at associates of high-profile individuals like business executives and politicians. And for Business Email Compromise (BEC) attacks aimed directly at executives and other high-profile individuals.
Furthermore, ChatGPT can produce human-like text for social media posts, fake press releases, web pages, YouTube video descriptions, and other collateral that cybercriminals can use to build a fake online presence to aid their attempts to trick people. The more natural emails generated present a technical challenge, as they may evade current content checkers looking for spam and malware.
In addition to using LLMs to improve their language use, cybercriminals can also use them to create executable code for attacks or build attack chains. According to Bleeping Computer, ChatGPT was used to generate JavaScript code that could detect credit card numbers, expiration dates, CVV digits, billing addresses, and other payment information entered into a web page. SC Magazine also reports how ChatGPT was used to generate code to scan text on a website for social security numbers. Security researchers have observed many examples of LLM-generated code being discussed and shared by cybercriminals on the Dark Web and in other cybercriminal forums.
What’s concerning is that some individuals who lack the skills to write malware-related code themselves are turning to ChatGPT for assistance. This increases the threat level from cybercriminals who may not have been able to perform attacks previously.
Researchers from Checkpoint demonstrated in a series of blogs late in 2022 that they could use an iterative process with ChatGPT to build an attack chain (blog 1 here and blog 2 here). The attack chain created was not particularly complex. Still, it demonstrated how engaging with an LLM can enhance the potency of the resulting code and make it more harmful.
With the assistance of ChatGPT and its output, individuals lacking the expertise to construct an attack chain can now create one and refine it through iteration. As a result, the disparity in skills required to carry out cyberattacks is diminishing.
Code Generation Vulnerabilities
Many developers have started using LLMs to assist in creating or updating source code. The code generated by these systems is often subpar and frequently contains known cybersecurity vulnerabilities. New York University and the University of Calgary researchers discovered that GitHub Copilot’s “AI pair programmer” produced code with known MITRE ATT@CK framework vulnerabilities in 40% of the 1,689 tests they ran.
Copilot is an LLM that relies on open-source code from GitHub for training. It uses the same OpenAI technology as ChatGPT, which means ChatGPT is likely to have a similar vulnerability rate when generating code. Other LLMs, such as Google Bard, will likely introduce known and unknown cybersecurity vulnerabilities into any generated code.
While tools are available to check source code for common vulnerabilities, it’s important for those using LLM-based code assistants to understand that the code generated must undergo both automated checks and expert human code reviews before it gets deployed to production systems.
Leaking of PII
Organisations invest a significant amount of effort in safeguarding the data they control. A data breach due to ransomware or any other cyberattack can lead to serious financial and reputational consequences.
As publicly accessible LLMs like ChatGPT become more widely available, information workers will use them to generate personalised emails for customer queries and other tasks. However, this process may involve the inclusion of PII or other sensitive data in the prompt entered into the LLM to generate the desired response. Although many LLM providers claim that they do not share entered information with other users of their tool or retain it beyond each user session, human error and software glitches are inevitable. The use of LLMs carries an unquantifiable risk of them retaining restricted or sensitive data that can be stolen by hackers or accidentally exposed in response to someone else’s use of the LLM.
Recently, TechRadar reported that 100,000 ChatGPT accounts had been stolen and sold on the Dark Web. This incident highlights that OpenAI and its data are susceptible to cyberattacks.
All organisations should create and enforce an acceptable Use Policy (AUP) for LLMs and other AI tools for their staff. Just as most organisations now have AUPs for Internet use, company laptop use etc., the time has come for AI AUPs to be standard. I’ll write more about this topic in my next article.
Loss of Company Secrets
It’s important to note that using LLMs can pose risks not only to PII but also to company intellectual property. Samsung Semiconductor demonstrated this when their developers used ChatGPT to modify some source code. The developers input confidential data into the LLM and the proprietary source code they were working on, resulting in the exposure of confidential data and the proprietary source code. As a result, Samsung has prohibited the use of ChatGPT.
Several other companies have banned or restricted the use of LLMs internally to prevent similar incidents. The companies banning use include Apple, Deutsche Bank, Northrop Grumman, and Verizon. While JPMorgan Chase, Accenture, and Amazon have restricted the use of LLMs.
Conclusion
The use of LLM technology can have both positive and negative implications. Unfortunately, the history of cybersecurity has shown that malicious individuals will exploit any technology available to them, and LLMs are no exception.
The ease of text generation using LLMs means that cybercriminals can easily raise their game by creating more sophisticated phishing emails and social engineering attacks. Over time, these bad actors may also learn to use ChatGPT or other LLMs to generate more complex malware code and attack chains.
To combat the threat of LLM-enhanced attacks, existing security measures and user training should be improved. It is critical to educate individuals to be wary of any suspicious emails requesting personal information. Taking an over-cautious approach is better than falling victim to an LLM-generated attack email.
As cyber attackers increasingly use LLMs to their advantage, cyber defenders must also employ AI systems to counter these attacks. This ongoing technological race between the two sides will require the expertise of human cybersecurity professionals who can focus on emerging threats, attack vectors, and effective defence strategies.
Halodata, our partners, and vendors have the expertise and solutions to enable your organisation to defend against current and emerging cybersecurity threats. Contact us today to start a conversation about boosting your cybersecurity posture.
It’s Time to Get Started with Halodata
Request Demo
Contact Us
Please complete this form to be contacted by one of our experts.
