Home Global Phishing Simulators May Not Be Effective in Training Users According to a New Study

Phishing Simulators May Not Be Effective in Training Users According to a New Study


Embedded phishing training in organization-run simulations does not function well, according to a new study conducted at an unprecedented scale. Crowdsourcing phishing detection, on the other hand, is not.

When it comes to hacking into a company’s network, the quickest method to get started is to send phishing emails to employees. They’re the weakest link in your network’s chain.

As a result, phishing simulators (also known as phishing tests) are becoming more common in businesses.

Those simulations imitate real phishing emails that arrive in employees’ inboxes but include no harmful payload. They display a genuine phishing website and collect data on who clicked with or without supplying credentials, how many people reported the phishing page to security, and so on.

Professional phishing simulation services are available, or companies can construct their own for free using tools like GoPhish.

The purpose of phishing simulation, regardless of the method, remains the same: better understand employees’ behaviour within the firm and raise awareness of that vital issue.

The computer science department of ETH Zurich, a Swiss public university concentrating on science, technology, and engineering, recently published a paper on the subject. The study lasted 15 months in a big organisation (about 56,000 individuals employed, with about 14,000 employees targeted by the study), making it the largest study reported to date in terms of scope and length.

The strategy involved sending phishing emails that directed users to a phishing page or emails that contained a malicious file that enticed users to perform a harmful action when launched, such as submitting credentials or enabling macros on an attachment.

The phishing emails could include short or thorough cautions (Figure A), whereas others did not contain any warnings at all.

Figure A Two warnings in simulated phishing emails: short and long, Source: ETH Zurich, Dept of Computer Science

A reporting button embedded in the employee’s email client might potentially be used to report the phishing attempts. Prior to the trial, the button was introduced and publicised in internal business news.

After a user takes a risky action, the simulation may direct them to an educational page that explains what happened in detail, what they should have looked for to avoid phishing, and future prevention suggestions. A second instructional film, additional quizzes, and phishing learning material were also offered, although the user was not obligated to view or read them. That informative page was not received by all users.

The study analysed what kind of computer usage, gender and age range would perform the dangerous action (Figure B).

Figure B Percentage of dangerous actions performed out of all phishing emails sent, divided by different demographics. Source: ETH Zurich, Dept of Computer Science

Computer usage

Employees with a specialized usage of computers (e.g., branch workers who mostly use a single dedicated software) clicked on more phishing links and performed more dangerous actions than the other categories of users.

Age range

The youngest employees clicked more on dangerous links than the oldest ones. Employees in the 50-59 age range were also more prone to fall for phishing.


According to the study, the combination of gender and computer use was significant, but gender by itself was not. The study lasted 15 months and found that a small percentage of employees, particularly the youngest, fall for phishing many times.

It also indicated that if employees are constantly exposed to phishing, many of them will eventually fall for it. “A rather large fraction of the overall employee base will be vulnerable to phishing when exposed to phishing emails for a sufficiently lengthy time,” according to ETH researchers.

The cautions in the phishing emails appear to have been useful in preventing people from clicking on the links, but extensive warnings were not more effective than brief ones.

Surprisingly, those who did receive the instructive page after falling for a phishing ruse clicked on subsequent phishing pages more frequently. The researchers cautioned that this result could only be applied to this specific technique of offering volunteer training, and that other ways might provide different results.

In the post-experiment questionnaire filled out by the employees, the researchers attempted to determine the source of this remarkable discovery. One probable cause is a false sense of security associated with the deployed training method: 43% of respondents chose “Seeing the training web page made me feel protected,” and 40% chose “The organisation is protecting me against nasty emails.”

Future research will need to determine whether this is due to a misunderstanding of the training page (for example, employees believing they were protected against an actual phishing attack) or overconfidence in the company’s IT staff.

Users continued to report phishing emails over time, according to the study, and there was no “reporting fatigue” in the firm. Reporting was used by many people. The reporters who demonstrated the best expected computer abilities were the most engaged. When reporting users received favourable responses, they felt encouraged as well.

Users sent ten percent of the reports within five minutes of getting the email. The majority of the reports, between 30 and 40 percent, were sent within 30 minutes (Figure C).

Figure C Source: ETH Zurich, Dept of Computer Science

Yet for such crowdsourcing to be effective, employees still need a convenient and easy way to report phishing cases. A button in their email client seems to be a good option.

Related Articles

We use cookies to improve user experience, and analyze website traffic. For these reasons, we may share your site usage data with our analytics partners. By clicking “Accept Cookies,” you consent to store on your device all the technologies described in our Cookie Policy. Accept Read More

ESPC on the go