Generative artificial intelligence (AI)

Artificial intelligence (AI) is technology that allows computers to perform tasks and make decisions like a human. AI tools make these decisions by learning and analysing large amounts of data to look for patterns and make predictions.

Generative AI is different to traditional AI – it uses data to generate new content, like text, images, or audio. A prime example of a generative AI tool is Chat-GPT.

When fraudsters use generative AI, it can make it harder to identify a fraud or scam attack. Below are some of the known ways fraudsters could use this to target you, as well as the steps you can take to protect yourself and your business.

Enhanced email phishing

Many scam emails are easy to spot. They’re usually filled with spelling mistakes and other errors. While most scam emails are still basic, fraudsters could use generative AI to create more sophisticated emails, which are perfectly written and even copy the tone of voice of trusted people or businesses. This could make phishing emails harder to spot.

Voice spoofing

Voice spoofing (or voice cloning) uses generative AI to copy a person’s voice. The copied voice can then say certain phrases or act as a chatbot. Compared to other scams, voice spoofing is rare. However, this technology could help fraudsters to deepen their scams. For example, attaching a voice note to a phishing email could make the scam look more convincing if the copied voice is from a trusted person.

Deepfake video

A deepfake video uses generative AI to copy the appearance and voice of a person. Deepfake videos can be convincing, usually showing the copied person saying things they’ve never said. Like voice spoofing, deepfake scams are rare – you’re more likely to see a deepfake of a celebrity than someone you know personally.

Increased accessibility

Generative AI makes it possible for fraudsters to potentially increase the scale and sophistication of fraud and malware attacks. It does this by lowering the level of technical or cultural knowledge required to commit sophisticated fraud attacks, enabling fraudsters to work across international markets more easily. For malware, generative AI systems could be used to identify vulnerabilities in a business' software and systems, leading to an increase in security breaches.

Protect yourself from generative AI fraud

  • Verify requests by calling a trusted number

    If you receive information or payment requests from someone you know, you should verify it’s genuine. Even if you receive an unexpected request through a phone/video call from someone you know, it might be cloned. We recommend calling them back using a trusted number.
  • Stay vigilant to deepfakes

    Although rare, deepfake videos can be identified by looking out for irregular blinking, blurred personal silhouette, strangely lit teeth and jewellery, sound to lip sync lag and inconsistent lighting.
  • Make sure staff understand the threat of phishing

    Educate staff so they’re aware of the risks and know how to spot signs of phishing.
  • Check you have the latest security updates and patches

    Weaknesses in hardware and software are exploited by criminals. Publishers release stronger updates or patches and it’s important to apply these as soon as you can.
  • Don’t overshare personal information

    Don’t overshare personal information such as pet names, birthdays, and family connections. Criminals may use this to launch personalised and sophisticated fraud attacks using generative AI.
  • Don’t open unexpected attachments

    Open an attachment within an email or text which you’re not expecting, even if the content doesn’t feel suspicious. If you need to open it, use up-to-date anti-virus security to scan the file before downloading.
  • Don’t be put off seeking a second opinion

    If you’re unsure whether a request is genuine, don’t action it and refer it to a colleague or manager.

Further Resources

Take 5 – Stop, Challenge and Protect

NCSC – National Cyber Security Centre advice and guidance on a range of cyber topics

Stop Scams UK – Working together to stop scams at source