[This is a Guest Diary by Joshua Woodward, an ISC intern as part of the SANS.edu BACS program]

Just listening to a lecture is boring. Is there a better way?

I recently had the opportunity to engage in conversation with Jonathan, a lead analyst at Rapid7, where our discussion led to the internal technical training that he gives to their new analysts. He saw a notable ineffectiveness in the training sessions and was "dissatisfied with the new analysts' ability to remember and apply the knowledge when it was time to use it." The new analysts struggled to recall and apply the knowledge from the classroom training and often "had to be retaught live," resulting in inefficiencies and frustration. After reflecting on the root cause of this issue, Jonathan suspected that the traditional approach to learning, such as classroom lectures and workshops, was at the heart of the problem. These more passive learning approaches failed to engage the participants, leading to disinterest in the training and lower knowledge retention. Drawing inspiration from a method that was effective for him, Jonathan decided to adopt a more active and engaging approach: Capture the Flag (CTF) competitions.

Capture the Flag (CTF) Competitions

Capture the Flag competitions can offer exposure to a wide range of cybersecurity concepts or drill into a particular skill set through carefully crafted puzzles. CTFs foster an active learning environment by encouraging participants to apply their critical thinking skills and knowledge in a practical context. The gamified nature of CTFs leads to more excitement and motivation to participate, and active engagement and problem-solving allows a deeper understanding and retention of cybersecurity concepts.

Considerations

Traditional training excels at comprehensively covering topics in a structured matter, while CTFs offer a better environment to apply skills practically and can be built to mimic real-world scenarios. However, the nature of CTFs may not be suitable for teaching specific skills in a predetermined manner, as participants may creatively approach challenges from various angles. Participants will only learn what is needed to solve the challenge. Carefully crafted challenges can offset this disadvantage to some extent, but they may not fully address this drawback. Despite the limitations, CTFs shine at getting participants to retain knowledge because they foster active learning. Participants are effectively teaching themselves in a hands-on manner that will help them remember and gain experience in the topic.

How puzzles are designed greatly influences the effectiveness of CTFs. Developing good challenges is a very time-consuming process. A senior analyst can teach a lecture in an ad-hoc matter, but all CTFs require a large preparation time. Jonathan mentioned that there are "a lot of competing requirements that are hard to balance" when designing a new challenge. The puzzle must be balanced and give participants a good starting point and prompt to prevent a knowledge blockade or feel overwhelming, but it still must be challenging and teach a specific skill set. Jonathan stated that when designing a challenge to target specific knowledge, a common trap is that it can easily start feeling like a trivia game rather than something fun, and "then you just have a quiz rather than a CTF." Well-designed challenges are the make-or-break linchpin for the successful implementation of CTFs in technical training.

Effectiveness

After introducing CTFs into his training plan, Jonathan noted that he witnessed a significant improvement in the analysts' ability to recall and apply the new knowledge. Being able to use the skills practically in an engaging and rewarding context seemed to give the participants a deeper understanding of the concepts and how to employ them when problem-solving.

I was able to interview an individual who had taken both types of training methods, and they noted that "CTF challenges were far more enjoyable and memorable" when compared to their original training. In terms of retaining and applying learning objectives, they found "CTF challenges to be significantly more effective." They were able to remember bits and pieces better from the CTF than from classroom training, which allowed them to have a starting place to research when solving situations in their work.

Jonathan comments that debating why the traditional classroom training failed is a discussion unto itself and has merit in researching it further. However, he did ultimately find that CTFs provided a workable alternative that helped fix the retention issue he was facing.

Integrating Capture the Flag challenges into internal training can give tangible improvements to participants' ability to retain and apply the knowledge being covered in training sessions. Combining CTFs with traditional training methods can help cover the drawbacks of either methodology at the cost of more preparation time.

--
* This article was written with the assistance of AI tools, including ChatGPT.
* Permission has been given by the interviewed sources to use their names and answers in this article. Full names have been redacted for privacy.

[1] https://www.sans.edu/cyber-security-programs/bachelors-degree/
-----------
Guy Bruneau IPSS Inc.
My Handler Page
Twitter: GuyBruneau
gbruneau at isc dot sans dot edu