Par 7-11 Page 45 TABLE 7-3. SELECTED SCENARIO EVENT SET INDEX WITH PHASES OF FLIGHT
AND CREW RESOURCE MANAGEMENT BEHAVIORS
SCENARIO
EVENT SET NUMBER
SITUATIONAL
AWARENESS
WORKLOAD
MANAGEMENT
PLANNING DECISION- MAKING Event Set One- Predeparture Pilot Flying (PF) planned deice for winter operations standard operating procedure (SOP) PF briefed rising terrain PF analyzed departure weather radar (WX) and requests takeoff alternate Event Set Two-Taxi Crew discussed route and holdover times Taxi – Low Visibility Deicing pad Surface Movement Guidance and Control System (SMGCS) plan Pad coordination Event Set Three-Takeoff Crew discussed icing issue before it could become a problem Crew set clear priorities for tasks and their order
Event Set Four-Climb PF requested higher altitude
Event Set Five-Cruise PF directed pilot monitoring (PM) to deal with engine problem PM performed needed checklists and announced compliance Crew assessed one engine landing with WX at diversion field PF calculated time and distance to Mahlon Sweet Field Airport (EUG) PF stated that they cannot go back to Seattle-Tacoma International Airport (SEA) Event Set Six-Descent PF prioritized tasks and got ready for approach PF reviewed single engine approach procedures and aircraft evacuation
Event Set Seven- Approach and Landing PF properly prioritized PM provides backup for PF on all his tasks PF briefed cabin crew PF planed and briefed Safety Enhancement (SE) instrument landing system (ILS)
Event Set Eight-Taxi In
NOTE: The four CRM challenges are spread across the eight event sets, such that only two of the eight event sets presents more than two CRM challenges. One of the strengths of the event set methodology is that it allows the facilitator to focus on a limited set of CRM issues at a time.
Par 7-11 Page 46 TABLE 7-4. THE EVENT SET MATRIX
EVENT SET PHASE OF FLIGHT
TECHNICAL
REQUIREMENTS
KEY EVENTS CRM BEHAVIORS
EVENT SET #1– Predeparture the crew must consider winter operations.
Predeparture Pushback Deicing procedures must be followed.
Takeoff alternate is required.
Departure, en route, and arrival in winter conditions.
Destination weather is at CAT IIIa minimums.
During engine start there is no N1 indication on Engine #1.
OR The #2 engine has a hung start, but starts on the second attempt.
OR When turning the engine anti-ice on, one valve fails to open.
COMMUNICATION:
Open, interactive crew climate established, crew asks questions and seeks answers on operational issues they are concerned about.
DECISIONMAKING:
Captain asks and receives input, but makes decisive final decisions affecting mission. Crew continually assesses changing conditions to improve operations.
WORKLOAD MANAGEMENT:
Efficient workload distribution so no one is overtaxed.
EVENT SET #2–Taxi Taxi Takeoff from short runway in winter conditions with takeoff gross weight near runway limit.
Flaps 5/15 takeoff required Engine run-up required in takeoff position Cycle gear after takeoff Taxi via slippery and congested ramps and taxiways in low visibility.
The takeoff runway limited, low visibility and icing conditions near runway limit. There is rapidly rising terrain to the south of the departure runway.
Complex departure in icing conditions.
COMMUNICATION:
Air traffic control (ATC) interaction, problem definition about deicing and rising terrain.
WORKLOAD MANAGEMENT:
Prioritize tasks for deicing and departure.
DECISIONMAKING:
Captain decisive about rising terrain issues, with crew input.
Par 7-11 Page 47 TABLE 7-5. THE EVENT SET ASSESSMENT/GRADE SHEET Rate crew 1-5 Not Obs CRUISE Event Set # 5 Pilot Flying: ❒Left Seat ❒Right Seat
a. Proficient in use of FMS and AFDS. (4.1.2.2)(4.1.2.3)
b. All normal/non-normal procedures accomplished in accordance with standard operating procedure (SOP). (2.1.1)(2.1.2)(2.1.3)(2.1.4)
c. Evaluates options and determines a suitable airport for landing. (4.1.1.7)(4.1.3.6)(5.1.2)
d. Briefs F/As using TEST procedures in preparation for landing. (10.1.3.6)(BC 1.6)
e. Crew manages automated systems to increase SA and avoid work overload. (AT 6.6)
f. Roles, tasks, and responsibilities clearly assigned. Guidelines established. (LT 2.3)
g. Communications with cabin crew/ATC/company clear and timely. (SA 1.6)(SA 1.7)(LT 2.3)
h. Determine plan, and discuss aircraft configurations, airport-specific procedures, and performance. (SA 3.1)
i. Establish and brief “bottom lines” and “backup” plans. (DM4.3)
Pilot #1 Technical CRM Pilot #2 Technical CRM
COMMENTS: REQUIRED FOR ALL ITEMS GRADED 1, 2 OR NOT OBS.
Rate crew 1-5 Not Obs DESCENT Event Set # 6 Pilot Flying: ❒Left Seat ❒Right Seat
a. Proficient in use of FMS and AFDS. (5.1.1.2)(5.1.1.3)
b. Accomplishes descent/arrival, approach briefing, and normal checklists procedures in accordance with SOP. (5.1.1)
c. Ensures all non-normal checklists are completed prior to commencing approach. (10.1.3)
d. Ensures that F/As and passengers are prepared for landing and possible evacuation.
(10.1.3.6)
e. Crew verbalizes a plan, to include bottom lines and a backup plan. (DM 4.4)
f. Crew is assertive when voicing concerns, deviations from the original plan, and when nearing/reaching bottom lines. (LT 2.7)
g. Crew plans and briefs automation modes and configurations. (AT 6.1)
h. Informs appropriate personnel of emergency situation; keeps cabin crew and passengers informed and updated. (BC 1.6)
Pilot #1 Technical CRM Pilot #2 Technical CRM
COMMENTS: REQUIRED FOR ALL ITEMS GRADED 1, 2 OR NOT OBS.
NOTE: This example has two separate job task analyses: one for technical tasks and one for CRM tasks. Technical tasks are identified in parentheses with no letters in front of the numbers. CRM tasks are identified in parentheses with letters in front of the numbers. In this example, DM stands
Par 7-11 Page 48 for decisionmaking, LT for leadership and teamwork, SA for situational awareness, BC for briefings and communications, and AT for automation and technology. Event set 5 has nine events, three drawn from the list of technical tasks, five from the list of CRM tasks, and one event (d) drawn from both technical and CRM tasks. Thus, Event set 5 integrates technical and CRM tasks into a single graded event set.
7-12. SCRIPTING AND VALIDATING THE LOS SCENARIO.
a. Scripting. Experience has shown that the effectiveness of LOS relies on script detail and following the programmed script during the LOS session. To accomplish this, the LOS scenario should be carefully scripted, including ensuring that all ATC communications use correct terminology, timing, and routing. Attention to detail in communications scripting will enhance the realism of the LOS FSTD session. Below are some guidelines for developing effective scripts:
(1) Realism. Scripting should include events and conditions that will take place during the flight. The script should concentrate on realistic inputs that support the flight profile as well as variations determined during scenario development.
(2) Crew Autonomy. Scripting must allow for crew decisions other than the expected response. The crew must be given the flexibility to play out their hand to a logical conclusion.
These diversions can normally be identified as a result of a key event or within an event set
(e.g., scripting possible diversions and changes in routing). The script should include event set number, phase of flight, communications (including frequency and radio call), key events, and expected actions. With these categories, weather conditions and general environmental conditions should be presented as they would occur in an actual situation.
(3) Defined Criteria. If the script is being developed as an LOE, detailed success criteria must be established. Technical performance criteria are documented in applicable regulations or in company documentation, and only an overview is presented in the event set documentation. CRM performance criteria are presented for each event set and are divided by CRM behavior areas that have been integrated and validated by the design team. The desired behaviors are presented with a brief statement of what constitutes unsatisfactory behavior.
Examples of criteria for the sample LOS shown in Tables 7-2, 7-3, and 7-4 include:
(a) Technical Skills. The crew will be proficient in the knowledge and execution of all required takeoff data, analysis of terrain issues, winter operations, systems procedures, and performance limitations of the aircraft.
(b) Communication. The crew will accomplish a predeparture briefing, to include the entire crew (cabin and flightcrew). The briefing will establish the crew climate by emphasizing the importance of interactive decisionmaking and participation of the entire crew.
The crew is encouraged to voice concerns they may have. Crewmembers will ask questions and seek information from each other about operational issues and decisions. Crewmembers will advocate issues until an acceptable solution is achieved. All problems should be recognized and decisions for their solutions made.
Par 7-12 Page 49
(c) Decisionmaking. The captain asks for and considers crew inputs, but the captain makes the final decision for the aircraft configuration as dictated by weather, performance, and fuel requirements. The crew continually assesses the changing conditions to improve the operation of the flight.
(d) Workload Management. The crew will distribute the workload to ensure that each crewmember is used while no one is overtaxed. The crew will use available resources to analyze the required tasks for this complex departure.
(e) Unsatisfactory Performance. Unsatisfactory performance of this event set includes a crew that is completely unaware of winter operations and the ramification on performance operating considerations. Also judged unsatisfactory is a crew that is not prepared for the complex departure, including the issue of the rapidly rising terrain. Other issues the evaluator observes during this event set may be result in a judgment of unsatisfactory performance.
b. Evaluations. Using these success criteria for the LOE, the evaluation is based on the outcome of the event set much like the current evaluation of the outcome of a maneuver. Within the event set, specific objectives are assigned, any one of which could be involved in the unsuccessful outcome of the event set.
c. Validation. A systematic approach to validating scenarios in terms of their training objectives should be adopted. Formal and informal review panels, analysis of data on scenario attributes, and feedback from instructors, check pilots/check FEs, flightcrew members, and FAA inspectors provide the information needed to validate or modify the scenario.
d. Test Flights. After the LOS is represented by the script, the LOS should be flown by at least two different crews, and if possible, the POI or designated representative. If possible, crews flying the LOS should be recorded for viewing. These tapes will serve as a useful tool for training the instructors/evaluators prior to implementation. When the crews have finished the LOS, they should complete the same validations forms used to integrate the CRM. Quite often, the crews flying the event sets will have a different rating for the CRM behaviors as compared to the experts. From this final validation, the LOS will be modified prior to instructor training and implementation.
e. Scenario Validation. There should also be a scenario validation accomplished by a facilitator not conducting the LOS. The scenario will be evaluated for its value in meeting the training objectives and for determining the level of facilitator skill required to administer the LOS. The evaluation of the LOS will also provide an opportunity to note any errors that may exist in the facilitator’s guide or in the flight documentation.
f. FAA Approval. The FAA must approve LOE scenarios prior to their use. However, if LOE event sets are designed to allow for their recombination in a mix-and-match configuration to create different LOEs, then event sets may be individually approved by the FAA for that purpose. When preapproved event sets are employed in new combinations, the resulting new LOE scenarios do not require additional FAA approval.
7-13. SCENARIO FLEXIBILITY.
Par 7-12 Page 50
a. Real-Time Flexibility. Students should not know exactly what to expect when they enter a scenario-based training or evaluation session. Therefore, it is desirable to provide the scenario with some flexibility, allowing the facilitator to switch events around, often in real time.
One option is to switch out airports. For example, an operator may use scenarios that involve departing from one of three airports to a hub, and then departing to, again, one of three airports.
Other than the hub, the flightcrew members do not know which airport they will be departing or arriving, as the facilitator can change the airport in real time.
b. Branching Options. The most common approach to building flexibility into scenarios involves branching options. The flightcrew members may know that there will be one of three electrical emergencies in cruise phase, but they do not know in advance which electrical emergency will occur. When using this approach, it is important that the difficulty rating of each emergency is similar. The event set matrix in Table 7-4 shows three alternate possible key events in event set 1. This is an example of the branching strategy built into an event set.
c. Interchangeable Events. One of the most creative uses of scenario flexibility requires careful planning, but it provides maximum flexibility and minimal cognitive load on the facilitator. This approach involves designing two separate scenarios in such a way that the aircraft configuration at the beginning and end of each matching event set is identical. So, for example, the aircraft configuration at the beginning and end of event set two is identical across both scenarios. This allows the event sets within those two scenarios to be switched out at any point. So if there are two scenarios, A and B, one crew receives AAAAAA, the next BBBBBB, the next ABABAB, the next AABBAA, and so on. The facilitators must master only the contents of two full scenarios, yet from these elements they can generate a large number of somewhat difference scenarios.
7-14. GRADING EVENT SETS/GRADING PHASES OF FLIGHT. As originally conceived by the Airlines for America (A4A) LOS Design Focus Group, event sets were to be independent of the phases of flight. A hung start or a compressor stall would be an event set, and would be graded as such. However, operators have found that approach awkward and have switched to assigning grades to each phase of flight instead. Using the scenario in Table 7-2, the original idea was that the severe compressor stall that occurs in phase five and carries through phase eight would be a single event set. Instead, each phase of flight is now graded according to how the event set activities within each phase of flight are handled, even if the event set carries across multiple phases of flight. So although the event set is the linchpin of the design and development of the LOS, flightcrew member grades are assigned according to phase of flight. What this means is that instead of the crew receiving a single grade for the compressor stall response across four phases of flight, each of the last four phases of flight receives a grade, and that grade is based on how the crew manages the compressor stall within that phase of flight. What that means is that while scenarios are developed from event sets, they may be graded either as event sets or as phases of flight.