Tuesday, October 11, 2016

What not to do when taking a GIAC exam

I’ll discuss these in more detail, but here are a few items worth considering avoiding. I’m writing this the day after I passed my GIAC GXPN with my lowest score ever on a GIAC exam (90%). I’m accustomed to scoring 95% or better, and I feel like I had subpar performance on this exam. So, I’ll discuss the things I didn’t so, so you won’t make the same mistakes. My GIAC certs: GSEC (SEC401), GCIA (SEC503), GCIH (SEC504), GCFA (FOR508), GMOB (SEC575), GASF (FOR585), GREM (FOR610), GXPN (SEC660).

In case you don’t know me and have found this blog post via the magic of search, I’m a Principal SANS Instructor, and consultant. Yes, I still take exams. I really care what I get for a score on my exams.

1. Don’t procrastinate
2. Don’t skip making an index
3. Don’t skip taking the practice exams
4. Don’t squander your time during the exam
5. Don’t beat yourself up

1. Don’t procrastinate
The biggest problem for this exam was that I took the class in April at SANS Orlando. I consistently advise students to (HTFU) and create the index within a week (two maximum) of taking the class, take a practice exam within two or three weeks, take a second practice exam if you got below an 80%, and take the actual exam within a week or two of the practice exam.
This exam, I simply didn’t do that. Why? Because I let my schedule dictate my priorities and failed to allocate and/or follow through on the index creation and practice exam. In retrospect, six months later, I didn’t spend any additional time over the last few weeks that I couldn’t have spent 5 or 6 months ago. I spent approximately 9 hours studying for the exam. Most of those 9 hours was not purely focused and I had interruptions like messages and twitter during that time. It was only as I was rushing out of my house, jetlagged, printing my just completed index on my just connected printer (I just moved ;) that I really dedicated my time.

2. Don’t skip making an index
My index methodology is something I’ve shared with a number of people. Check out the details and Perl scrip here: http://bit.ly/crowley-index-script . I’ve had people come up to me to introduce themselves, thanking me for helping them to pass exams based on this script. The method I use has been translated into Japanese: contact me via twitter ( CCrowMontance ) if you want the Japanese version.
Short story for my index method is that I spend about 1-3 hours per book reviewing the content, and creating raw data to input to the Perl script. The raw data looks something like this:

14;GIAC, exam;exam, GIAC;certification, exam, GIAC;certification, exam, passing;exam, GIAC, pass

The point being that I include the topics on each page, in some cases referencing the same information multiple different way. The reason for the duplication is that I don’t know how I’ll need to seek the data when I attempt to retrieve it. My memory is excellent, but my recall is terrible.
The index helps me quick find detailed information in the books to confirm my thought, or differentiate a nuanced detail that I can’t recall.

3. Don’t skip taking the practice exams
This is where I deviated from my methodology substantially. My standard practice is to take a practice exam with my completed index then use the practice to update the index. I was simply too busy in the last month to complete this. I had already extended the exam once, and I really didn’t want to extend it a second time. So I skipped the practice exam. That hurt my score, I’m sure of it.
I did take a beta version of the new practical questions. But, that was just after I took the class. That was a cool experience, but in some ways skewed my perception about what the practical questions would be. In my beta exam, I used techniques covered in the class for developing exploits. In my actual exam, I had an environment I needed to use pen test techniques covered in the class to exploit an environment.

4. Don’t squander your time during the exam
I look up questions in the book to verify that I’m right if I’m not 90% sure that I know the answer. I mark an answer after reading the question, then go to the book for validation. Infrequently, I have to change the answer. But, my stance is that I have the time to do this.
During this exam, I completely ran out of time, and just answered the last 5 (practical) questions without having any idea what the answers were. I started the practical section with only about 15 minutes left for the exam.

Additionally, in the practical section, I crashed a service that I wanted to interact with. To restart the service, I restarted the virtual machine environment. This took almost 4 minutes to complete. So, that consumed about 30% of my time to work on the practical questions.

5. Don’t beat yourself up
My score percentage went down over the course of the exam. My recollection of my check point scores is below.
Checkpoint 1: 100% (15/15)
Checkpoint 2: 93% (28/30)
Checkpoint 3: 91% (41/45)
Checkpoint 4: 90% (50/55)

That being said, I didn’t lose my cool during the exam. At the first checkpoint, I was surprised at the 100% mark. There were two questions in the first 15 that I wasn’t sure if I was falling for a trap, or if I was over thinking the question too much. One danger for me is going way down an esoteric thought process to answer the question, rather than simply answering the actual question. After 30 questions, I saw I had missed some. No worries. Move along.

When I took the GCFA, I answered two questions incorrectly. This was back when you saw if you answered the question correctly immediately after answering. My first incorrect answer was on a legal question related to German law. I was really upset that I got the answer wrong, because I spent about 10 minutes considering the information I had looked up in the book. I was so bothered by this, I got the next question wrong, too. I would have entered into a failure spiral if I hadn’t taken a few minutes right then to simply stop answering questions, and allow the frustration and ire to dissipate. During that GCFA exam, I actually talked myself out of the frustration. If you’re feeling frustrated, counsel yourself that the frustration is detrimental. Pause as long as you need to, so you don’t make another mistake.

Here I am, unhappy with my performance. But, I got a 90%. I’ve done a root cause failure analysis, and will not do so poorly when I take the GIAC GMON here in the next few months.
Good luck on your cert exam, if you are embarking on it. If you have questions about how to use my Perl script ( http://bit.ly/crowley-index-script ) feel free to contact me on twitter – CCrowMontance.

Sunday, October 2, 2016

Risk Management, Community Interaction, Planning for Failure, and Exercises to get better - AFF Level 1

AFF Level 1

Organizational risk management is much of information assurance (cyber, if you must) is about. We can spend money to help diminish the likelihood that something bad happens. But, we can’t assure that the bad thing won’t happen. We spend time thinking about what might go wrong, practicing for things going poorly, and dealing with things actually going awry. I’m probably not telling you anything you don’t know. But, bear with me because I want to share a story about my recent experience with personal risk management in the form of skydiving.

Years ago I thought it would be exciting to try sky diving. I’ve heard of the risk associated with it. But, I want to try. The main reason is the prospect of eventually getting to fly in a squirrel suit. I’m definitely interested in speed and thrills. There’s about 1,999 more jumps between me and the opportunity to don a squirrel suit. Not sure that I’ll get there. But, that’s not the point. That was the objective initiating this drive.

From a practical standpoint, my poise and awareness during emergency situations is a self-rated moderate. I’ve dealt with medical emergencies, both of a group member and myself in isolated (by myself mountain biking, for example) situations. I’ve dealt with about 1,000 computer security incidents. That’s a round number because I don’t really know the number. In retrospect I wish I had an incident case log. I would be more effective today with exactly the same level of response action if I had been tracking my response actions. (TODO: personal system for logging and tracking response activity). I have recorded this data all over the place. Most of those tracking systems I no longer have access to.

But I digress. My poise and awareness during stressful situations are moderate on a scale of low, moderate, high. I have a good deal of experience, but I would rate better emergency room doctors, people with substantial combat experience, practiced airplane pilots, race car drivers, professional athletes as high on that scale. Most normal people I’d put in the low category. Unknown and stressful situations cause them to perform worse that they would otherwise. So moderate, is performance about equal to normal capability within stressful situations, but some experiences could still dislodge that person from poise. High level performance then is a person who has poise and grace in all situations: even unknown and unexpected situations well outside of their normal zone of comfort and practice. People with high degree of poise within their area of expertise not only meet level of performance, but exceed the expected level of performance.

Given this self-assessment level of moderate, I should be able to operate within a stressful situation without substantial prior knowledge of the tasks to perform, given adequate training.

The training. Accelerated Free Fall (AFF) is the program for becoming certified to sky dive. Level one ( http://www.affschool.com/8-levels/#1 ) included about 4 hours of classroom and physical practice, culminating in a practice jump with two instructors holding on to you while freefalling. The student learns to: orient his body to the relative wind; hold the appropriate position throughout the freefall; monitor the altitude; understand the altitudes at which specific actions must be performed; use non-verbal communication signals to coordinate with and receive direction from instructors; how to check to assure the chute is safely landable; how to deploy backup chute by cutting the inadequate chute loose and deploying the backup; how to navigate the landing path; and how to alight on the earth again after your freefall.

Those items are crammed into roughly 4 hours of instruction and practice, then you get in a plane and jump out of it. I went through the AFF Level 1 with a single other student and one instructor for the classroom portion. During the actual jump I had two instructors each with both hands in firm contact with my chute harnesses.

There was so much information. It was repeated multiple times, and there were multiple quizzes throughout the instruction. But during course of the jump, I had difficulty retaining it all and keeping it straight. Fortunately, I retained enough of it to get back without any major damage.

I had an hour delay between the instruction and the jump. I sat with my classmate, we talked about the sequence. We watched the other divers landing.

Finally, it came to be my time for the jump. I got suited up, got my chute, and went through one cycle of the exit from the aircraft with my backup (non-release side) instructor. It was more important to do this with him because he would be hanging onto the outside of the plane while I was doing my sequence (up-down-step out) within the plane.

We flew up to altitude. A couple who were clearly experienced jumped first. I got up, took my position at the door. “Check In!” Brian gave me the go signal. “Check out!” Craig gave me the go ahead. Up. Down. Step out.

I was falling out of the plane. I didn’t think about the relative wind, but I did try to keep my arms and legs back. I felt my body turning toward the direction of the fall, and I arched my back further.

Altitude 12, 000 feet. Circle of Awareness. Check and report. Look left - Craig gave me signals to adjust my position. Two fingers – legs out more. I stretched my legs, pointed my toes. Report right. Lazy W signal. My arms needed to go back more.

Release check. Left arm out in front of me. Reach back, put my hand on the hackey sack to be able to release my chute. Return to lazy W. Again. Left hand out straight. Right hand back to the hackey sack. Again. Left hand out. Right hand back.

Circle of Awareness. Check altitude. Report. Craig has me adjust my position. Lazy W. Fix my arms. Report. Fix my legs.

10,000 feet. Adjust position through hand signals.

9,000 feet. More bad position. Legs extended. Arms in a better W.

8,000 feet. Lazy W. Better arm position

7,000 feet. Extend legs.

6,000 feet. Lock on.

5,500 feet. I wave off. Single finger from Brian. I reach back for my hackey sack. It’s gone. Brian pulled it.

Then, I have my first moment of “Ok. What now?” I am paused. I don’t really know what to do for a moment. I’ve decelerated substantially. The chute seems to be working. I look up. I check the shape. It’s a rectangle. I check stability. I’m not really sure what I’m looking for, but I don’t see any substantial luffing or flapping of the chute. So, ok, I guess.

Steerability. I reach my hands up into the yellow steering handles. I’m supposed to pull them down a bit to release the brake, then locate the holding area (where I’m going to wait until I reach 1,000 ft.) I’m supposed to orient to the holding area with the steering handles then do a steerability check. Instead, I go right into the steerability check. Left turn? I looked down over my left shoulder to be sure I won’t collide with anyone by the maneuver, and pull the left handle all the way down. I start to turn left. I let the handle go back up. Right turn. I look to my right and down, then pull the right handle all the way down. I can make a right hand turn. Flare. I’m supposed to pull the handles all the way down, to be sure I can flare. I pull them down. I think that it seems I can slow down, so I think I’m good to go. I look around, and locate the trees I’m supposed to head toward. They’re behind me and slightly to the right, so I head that direction by turning about 220 degrees to the right. I check altitude. I can’t remember exactly where I was at beginning this maneuver to the right. About 4,500 feet, I think. I’m a bit concerned that I can’t really get to the holding area. I navigate with the handles to adjust my direction. I’m relieved that the steering mechanism seems pretty easy. The steering and landing were the areas of greatest concern. In retrospect, I should have practiced a flare and brake in this traverse toward the holding area. But, I didn’t.

3,500 feet. I’m approaching the holding area. Tracking the location of the other chutes in the sky. There were a bunch of tandem divers who were much higher. Several of them were doing interesting maneuvers. Some other time, I thought. I just want to get to the holding area.

3,000 feet. Still working my way toward the holding area.

2,500 feet. Not quite to holding area, but getting pretty close. I am a little concerned about getting there. Three or four other divers are beneath me. Presumably these are my two instructors and the couple who jumped first.

2,000 feet. The backup radio comes on. I can’t really understand anything Craig is saying. He tells me something, I maneuver a bit, because I’m actually heading the wrong direction (still traveling toward the holding area). I presume he is concerned that I am not oriented for the landing pattern properly. I adjust my position by making a 270 degree turn, so I’m generally heading back toward the landing path.

1.500 feet. I’m still in the holding area, but starting to leave it. I’m too high to leave it, but heading into the pattern. I turn a bit to the right and back to the right to try to stay in that area but slow down my exit from the holding area.

1,200 feet. I’m leaving the holding area, too high.

1,000 feet. Out of the holding area. Following the stream bed above the trees.

600 feet. I’m at the taxiway, where I should make a left turn. Instead of making a hard left, I make more of a 45 degree turn with the intention of travelling some more out of my way to extend my path a bit longer to try to lose more altitude.

300 feet (estimated). As I get to the center of the taxiway I make a 90 degree left turn to head down the taxi way. I tried to check my altitude at this height, but couldn’t really read it, so decided to focus on going straight.

There were several other people in the center of the field. I was too high. I knew that. Not terribly, though. The wind was stronger here since I was heading into it, and it noticeably required more steering. There were people in the center of the field, in line with where I was heading. I steered slightly to the left, making a bit of a lane change. I adjusted back to the right and continued straight. Craig was on the radio talking to me, but I really didn’t understand much of what he was saying. I think he said I was too high. But, I didn’t think there was much I could do about it at this point, except go straight and land.

25 feet (estimated) I was preparing to land. Well short of the trees at the end of the landing area. Which was a relief to me.

15 feet (estimated) I was supposed to flare at 10 feet. I estimate that somewhere between 20-15 feet is where I actually executed the flare.

Touchdown. I held the flare like I was supposed to, but I was too high. The training covered PFL – Parachute Fall Landing. Or something like that. The training had us jump from incrementally higher steps. We kept our feet together, pogo’d like a pogo stick, bent like a banana to one side, rolled onto our leg, hip, side. We kept our arms tucked in and let our body absorb the fall through transfer of momentum.

I didn’t do any of that. I had my legs apart. I didn’t transfer the momentum via a roll. I absorbed it like I was doing a squat, and fell backwards, like I was rolling out of a fall from bouldering. I boulder a lot and fall with some frequency during bouldering. I do a lot of squats and deadlifts. So, I’m not at all surprised that’s how my body reacted. It did the maneuver it is trained to do. It’s just that this maneuver wasn’t the appropriate maneuver in this case. I’m definitely sore as a result of that landing. A lingering ankle injury aches more today than normal. My right hamstring is sore. My left hip is sore. My gate walking feels a bit abnormal, like the position of my hips and legs is a little off from where each part expects the other to be. I don’t feel like I can hustle, and I don’t feel as spry as I normally do. Very fast walking through the airport during a transfer to make today’s flight wasn’t a welcome circumstance.

Will I go for level 2? I don’t know. I have 30 days to jump before I have to retake Level 1. My difficult schedule will probably prevent me from completing the level 2 within 30 days. Or maybe next Saturday I’ll do it, I have a time window of about 5 hours, which would be enough time to do it on the North Shore!

I hope you take something away from this. If you do, please let me know what it is. Let me share my take away lessons.

First, with regard to training. I think that this reinforces my commitment to training, simulation, and exercises more so than ever before. There are a few things that I like about training. One is the trainer assuring me that he is thoroughly competent in the area. When I am literally putting my life, safety, and well-being in the trainier’s hands I want to have the sense that the program he’s providing is solid. While I got that, I also got the sense that I was going to be on my own. Which, I was. There were several things that could have gone poorly which didn’t. I think these were the direct result of the training. I suspect thousands of people go through this training program on an annual basis across the USA. I didn’t research these numbers to write this article. But, it would be interesting to know what those numbers are, as well as the number of pass/ fails as well as the frequency of incidents with jumpers related to AFF level 1.

This leads me to the correlating questions for your information assurance program. How many of the tasks that you expect for your analysts can be broken down into a clear, repeatable, articulated sequence that can be drilled over, and over, and over and over? Where there’s no ambiguity for the actions to be taken?

** Question number one. Is there a plan?

If you can’t provide a clear sequence of actions to perform, can you provide a decision making matrix? Where a proscriptive plan cannot be created, can you provide unambiguous decision making criteria? In this experience the criteria for assessment is SSS: Shape; Stability; Steerability. The sky diver necessitated a framework for analysis to determine if the current state was adequate to safely land the parachute, or if a replacement parachute was in order.

This critical period (5,500 feet until 2,500 feet) had a defined entry, a clear period of assessment, and criteria for escalation. If at 2,500 feet there wasn’t a parachute that met the SSS criteria, there was a defined procedure to engage. For skydiving, this is the one escalation procedure. Cut away the main chute, and engage the reserve chute. You probably won’t be able to manually engage the reserve chute because the automated system to engage the reserve chute will be activated. We drilled this action no less than 10 times. This included decision making associated with the physical performance of the motions associated with cutting away and engaging the reserve chute.

** Question number two: What to do when the plan failed?

Throughout the training, there was only one other person who was a student. I sincerely couldn’t imagine going through a class of 20-30 other people who were attempting AFF level 1. As with most other training courses, there was a sense of comradery established. I’m a fairly solitary person. But, when I was finished with my jump, I waited a while until I confirmed that my classmate had successfully completed his jump. He probably jumped another time that day. I probably could have completed another jump, but my schedule and my plan precluded it. I suspect that another day I will jump again. I know that Jordan will remember that first jump and our class. I also knew that while we were both trying to develop an understanding of what was required of us we had a sense of mutual support and a drive to assure that each of us understood what needed to be done. As you guide people in the enterprise to complete a task, do they think that you are looking for a reason to fire them? Or are they sure that you’re there to help them complete all the details and achieve excellence?

** Question number three: Who’s there to help me if I need it?

This is my “lessons learned” report for my first sky dive. I’m sure that I could have performed better. I’m glad that I didn’t get hurt. I’m glad that I followed through on completing a challenging and ambitious plan. 

I will say that on the climb up from the airport, we discussed, double checked, reviewed, and reviewed again the steps for what we were going to do. The next time that someone tells you that we don’t need training for incident response, network security monitoring, or forensic analysis, ask them if they would be willing to jump out of an airplane without having gone through training.

This experience suggests to me that incident response is more complicated than skydiving. I’m not good at skydiving yet. But, from the sequence of sky diving that was taught to me I have a very specific sequence of actions that must be performed and a single clear objective. That’s substantially easier to perform and practice than security operations.


One item that I won’t belabor, but would ask for feedback from anyone who ever has an opportunity to listen to me speak. Please tell me whatever phrase I use to the degree that it becomes cloying. That thing that I fall back on to express a sentiment of importance when I become lazy and don’t use a more interesting word. I used to use the word “actually” a lot. Now I use the word “generally” too frequently. Help me to thwart my linguistic laziness. Thanks for following along with me on my first solo jump!


TODO: personal system for logging and tracking response event and incident actions