During my college years, I had a few opportunities to work on a blue team for collegiate competitions, particularity the Collegiate Cyber Defense Competition (CCDC). This meant working alongside fellow students to defend poorly configured networks against a professional red team. Over the few years, through trial and error, I learned a few things about what to expect from these types of competitions. Now that I have graduated, and sometimes even dabble in the red team opposition now, I feel I should write out what tips I would give to other students who are interested in these competitions. These events are pretty cool, and can help you learn a lot, so here we go.
Tips
Tip 1: Be prepared to fail
Yes, this is not a very encouraging tip, but I learned this the hard way. Be prepared to get rolled over by the red team. Be prepared to get a little heated under the collar (be it civil though!) and frustrated at every turn. Many of these defensive competitions are designed specifically to give the red team a leg up on you, so just do your best. It’s the best you can do. Use the opportunity to learn as much as you can. Ask the red team, ask the white team, ask other professionals that are at the event. It’s the reason these events exist, to help you learn and give you experience.
Tip 2: Know your OS
Know as much as you can, time allowing, about your OSes. Usually having certain people focus on a particular OS is the most effective, as every OS requires its own sea of knowledge. Know what systems might be at the competition (Linux and Windows are always safe bets), and whatever you are chosen to focus on, know the OS’s commands and configurations like back of your hand. Know the intricacies and ways the OS can be manipulated. For example, know about dot files for Linux and how they can be used to hide stuff. Know about cron jobs and how they can be used. Also, know the commands needed for you to get a handle on the system when you first access it. Focus more on quality rather than quantity. Better to be good at a few things, than not very good at a lot. Focus on commands and services that might be at the event, and learn tools you think will be useful. Knowing what to focus on may take some time, so keep an eye out when you’re at the events for what you might be missing.
Tip 3: Know your firewalls
It seems to me that good inbound and outbound firewall rules are key to success. Removing all the extra junk being sent at you is amazing. I think I’ve had rules that have blocked thousands of connections at events! You should have a handle on how to make firewall rules for servers, hosts and network devices. Know what’s required for a stateful firewall setup on each of these devices and be familiar with firewall best practices. If there’s time, setting up firewall logs is also a great source of information.
Tip 4: Know your monitoring tools
Apart from configurations, the most important thing for a blue team is information. What is going on on your network? What connections are going on? Who’s logged on? As the blue team, you need to know these things as it will help you kick the red team out as soon as you see them touch your box. I usually opened a screen (tmux can also be used) terminal and had multiple windows open to different monitoring tools. Here is very short and VERY incomplete list of common tools I remember using:
- w - This tool shows you who’s logged on. Combining this with the watch command turns this into a handy status on who’s logged on.<
- tail -f - Probably one of my favorite “in a pinch” tools. tail, with the
-fflag, tells tail to print out any updates to the target file. I usually ran this against Apache logs and other text-based logs to get an idea of what was going on. For example, when I used this with Apache logs during CCDC, it led me to previously unknown PHP based backdoors. Very useful. - netstat - with the flag combination
-tunap(tuna with a p), you’ll get a list of connections and listeners. Very, very handy. Usewatchwith this too.
Other tools I’ve seen mentioned are:
auditdlastlogsysdig
Dig around for other monitoring tools. You’ll want something easy to use and setup, you don’t have much time for these competitions before the red team will start fires you’ll have to put out. Information is your best friend, but do be careful. Red team can be a tricky bunch!
Tip 5: Know how to code
I know that some NCS majors may not like programming, but knowing your away around code and scripts can be priceless. Perhaps there’s a website that needs to be patched, or a mysterious script. Knowing your way around code will help you do this. Be familiar with programming essentials and common programming languages. Use the Internet, if you have access during the competition, to fill in any gaps. Know how to identify vulnerable code for different applications (see things like OWASP) and know what to do with them. I’d like to see competitors patch stuff, not just clean up after the damage is done. It’s better in the long run to plug a hole in a pipe than keep mopping up the floor!
Tip 6: Check your environment
If things aren’t working the way they should, check out what’s on the system. the file command is very handy for checking if your precious tool has been converted into a evil script. Also, package managers usually have ways to verify the checksums of package contents and binaries. Use these to improve your confidence in your environment. Otherwise, you may be doing something wrong. In any situation, be prepared with a backup plan, or a list of questions for later.
Tip 7: Take notes
Whether prepping for or in the competition, it wouldn’t be bad to keep some notes. If you do better than me in keeping track of them, you’ll probably have the grounds for further investigation, research, and questions later.
Tip 8: Keep learning
Very vague and broad, yes, but always be learning. It’s kind of a given for our field, but it’s worth reminding ourselves, me included, to keep the mind working.
In Conclusion
Hopefully, you can take something away from this list of tips. Just keep doing your best and striving to learn, and you should do fine. You may not win first place every time, but you’ll have valuable experience that you can take to the workplace. Defensive cybersecurity events are important to giving us an idea of what defenders have to do on a daily basis, and help us develop our skills in more realistic situations. Go out there and defend!