Par 1-3 Page 2
1-3. AQP CHARACTERISTICS.
a. Participation is voluntary.
b. Employs innovative training and qualification concepts with the regulatory flexibility to tailor training to individual company circumstances.
c. Uses performance data to drive curriculum changes.
d. Qualification is based on individual and team performance, using progressive evaluations of proficiency objectives, and the structure and maintenance of all elements (curriculum, facilities, training equipment, instructors, evaluators, courseware, and quality assurance) of the program.
e. Data collection and analysis to empirically validate individual and team proficiency, and the AQP itself.
f. Training is systematically developed with an audit trail for all training and data requirements.
1-4. EQUIVALENT LEVEL OF SAFETY AND AQP REQUIREMENTS.
NOTE: AQP is a voluntary program that entails a strong commitment from the air carrier to exceed minimum training standards in the greater interest of safety. To determine an equivalent level of safety of an airline’s AQP, the entire training system must be examined as a comprehensive whole rather than considering any one component in isolation. To ensure that the AQP does establish an initial justification and a continuing process to show an equivalent level of safety, mandatory requirements have been established.
Under an AQP, the FAA monitors the process as well as the product. Instead of basing curriculums on prescribed generic maneuvers, procedures and knowledge items, AQP curriculums are based on a detailed analysis of the specific job tasks, knowledge, and skill requirements of each duty position for the individual airline. Compared to traditional training programs, the AQP process provides a systematic basis for establishing an audit trail between training requirements and training methodologies.
AQPs are systematically developed, continuously maintained, and empirically validated, proficiency-based training systems. They allow for the systematic analysis, design, development, implementation, progressive evaluation, and maintenance of self-correcting training programs that include integrated Crew Resource Management (CRM), improved instructor/evaluator standardization, scenario-based evaluation, and a comprehensive data-driven quality assurance system.
Par 1-4 Page 3
a. Comply with all aspects of the approved AQP.
b. The processes used for development, implementation and maintenance of program operations will be continued throughout the life of the program.
c. Accommodate make, model, and series aircraft (or variant).
d. May build upon an existing training program or be completely new.
e. Provide three basic types of curriculums for every duty position; indoctrination (for new hires, new instructors, and new evaluators); qualification; and continuing qualification (see Figure 3-6). Specialty curriculums (transition, conversion, upgrade, requalification, refresher, etc.) will be derivatives from these basic types.
f. Duty positions to be covered must include all flightcrew member positions, instructors, and evaluators, and may include other positions, such as flight attendants (F/A), aircraft dispatchers, and other operations personnel.
g. Provide satisfactory justification to the FAA that the proficiency-based qualification of personnel under AQP meets or exceeds existing 14 CFR part 121 and/or part 135 standards.
h. Each applicable requirement of the Airman Certification Standards (ACS) or practical test standards (PTS) and 14 CFR part 61, 63, 65, 121, or 135 that is not specifically addressed in the AQP continues to apply to the certificate holder.
i. Document the requirements of the ACS or PTS and part 61, 63, 65, 121, or 135, as applicable, which would be replaced by an AQP curriculum.
j. The certificate holder must establish an initial justification and a continuing process to show how the AQP curriculum provides an equivalent level of safety for each requirement to be replaced.
k. Any training or evaluation that is satisfactorily completed in the calendar-month before or the calendar-month after the calendar-month in which it is due is considered to have been completed in the calendar-month it was due.
l. Curriculums must be based on an instructional systems development methodology. This methodology must incorporate a thorough analysis of the certificate holder’s operations, aircraft, line environment and job functions.
m. A list of, and text describing, the knowledge requirements, subject materials, job skills, and qualification standards of each task to be trained and evaluated.
n. A list of, and text describing, supervised operating experience, evaluation/remediation strategies, provisions for special tracking, and how recency of experience requirements will be accomplished.
Par 1-4 Page 4
o. A ll curriculums will include planned (not programmed) hours for ground training, flight training, evaluation, and operating experience.
p. Qualification and continuing qualification curriculums must integrate the training and evaluation of CRM/dispatch resource management (DRM) and technical skills and knowledge.
q. Include line operational training (e.g., full crew LOS).
r. Use LOE or an equivalent evaluation under an AQP acceptable to the FAA for proficiency evaluations.
NOTE: Regardless of number of events sets, a crash (i.e., unsafe individual or crew performance that would result in significant damage, hull loss, or loss of life) during an LOE constitutes a failure of the LOE.
s. Integrate appropriate advanced flight training equipment. Flight-training devices and simulators will be used to support scenario-based training as appropriate.
t. Develop data collection and analysis processes in order to obtain performance information on crewmembers, dispatchers, instructors, evaluators, and other operations personnel that will enable the certificate holder and the FAA to determine whether the form and content of training and evaluation activities are satisfactorily accomplishing the overall objectives of the curriculum.
NOTE: The FAA uses de-identified data for program monitoring not to monitor individual crewmember, dispatcher or other operations personnel.
u. Provide a Master AQP Transition Schedule (MATS) that provides a plan to transition from a traditional program to an AQP. In addition, the MATS should include a plan on how the applicant would return to a traditional program if that becomes necessary or desirable at some later date. This is not to imply that the traditional program must be kept current once AQP is entered, but only how the applicant would update the program and the timeframe needed for implementation.
1-5. DEVELOPMENTAL ASSISTANCE AND SUPPORT. Applicants may avail themselves of a wide range of support organizations, documents and services in the development of their AQPs. There is considerable support available from government and non-government sources.
The total inventory of such support is constantly increasing, and the latest versions of all such support resources are always available by contacting the appropriate Safety Standards office. The following is available directly from the FAA:
a. Flight Standards (FS). The appropriate Safety Standards office is the primary support available to all AQP applicants and participants. This office includes a manager, aviation safety inspectors (ASI), instructional systems development specialists, and data management specialists dedicated to the support of AQP programs. This office works in conjunction with the Principal Operations Inspector (POI) and other FAA personnel to transition applicants into AQP and to maintain the quality of those programs following full implementation.
Par 1-5 Page 5
(1) Role of the Appropriate Safety Standards Office Manager. The FAA manager is the person responsible for ensuring that the regulatory requirements for AQP are met and approved AQP standards are maintained. In this capacity, the appropriate Safety Standards office manager provides oversight of the AQP process and documentation through all phases. The manager will also chair the AQP Extended Review Team (ERT). The manager will retain a quality assurance role and coordinate with the POI any modifications and subsequent approvals throughout the AQP’s lifetime.
(2) Role of the POI. The POI is responsible for ensuring that AQP regulatory requirements are met and, that through surveillance, the AQP provides a satisfactory level of safety and airman competence. The POI will provide oversight of the Instructor/Evaluator program and adherence to approved documentation for all AQP curricula. The POI will coordinate with the manager of the appropriate Safety Standards office any modifications and subsequent approvals throughout the AQP’s lifetime.
(3) ERT. The review and surveillance of an AQP prior to final approval is equivalent to a recertification of the applicants’ entire training program. To ensure maintaining an effective and standardized process, the appropriate Safety Standards office will lead a team to conduct the review and certification of the AQP. The ERT will also include (but not necessarily be limited to) the following:
(a) The POI (or a designated representative).
(b) Training Center Program Manager (TCPM) (if applicable).
(c) The assigned aircrew program manager (APM) (if any).
(d) An appropriate Safety Standards office ASI staff specialist.
(e) A cabin safety inspector (if required).
(f) A dispatcher (if required).
(g) An instructional system design specialist.
(h) A data management specialist.
NOTE: Additional technical personnel can be added to the ERT as required in specific areas of expertise. These would include the focal point, a member of the Flight Standardization Board (FSB) or Flight Operations Evaluation Board (FOEB), as appropriate.
b. Technology Support. The appropriate Safety Standards office includes a technology staff assigned to enhance the effectiveness of the program through the development of websites, databases, collaboration techniques, software training and installation, and other support.
c. Other Resources. The following list (Figure 1-1) provides samples of the types of resources available:
Par 1-5 Page 6 FIGURE 1-1.
OTHER RESOURCES AQP Websites:
http://www.faa.gov/training_testing/training/aqp/ (public access) FAA Resources:
• Title 14 CFR Part 121 Subpart Y, Advanced Qualification Program FAA Advisory Circulars:
• AC 120-35, Flightcrew Member Line Operational Simulations: Line-Oriented Flight Training, Special Purpose Operational Training, Line Operational Evaluation (as amended)
• AC 120-51, Crew Resource Management Training (as amended)
• AC 121-42, Leadership and Command Training for Pilots in Command ATA AQP Working Group: Focus Group Reports:
• Line Operational Simulations: LOFT Scenario Design, Conduct and Validation
• Advanced Qualification Program Instructor/Evaluator Task Analysis
• Applied ISD in AQP Development: Front End Analysis
• Advanced Qualification Program Data Management Guide, Second Release AQP Software Tools:
• RRLOE: Rapidly Reconfigurable Even Set-Based Line Operational Evaluation Software [LOS Scenario-building software toolkit]
• Proteus Database [AQP curriculum & documentation-building software toolkit: AQP in a box]
• Rater Calibration Software Tools [Measures and assesses reliability across evaluators]
• Inter Rater Reliability Assessment Toolkit (workbook & diskette)
• Instructor Evaluator Training & Calibration (IETC) Toolkit AQP Research Products:
• Developing Advanced Crew Resource Management (ACRM) Training: A Training manual
• Developing Operating Documents: A Manual of Guidelines (on CD)
• Facilitating LOS Debriefings: A Training Manual The Appropriate Safety Standards Office Products:
• Advanced Qualification Program Overview, Powerpoint briefing (as amended)
• Advanced Qualification Program (AQP) Inspector Training (Student Guide, Leader Guide, Reference Book, Lab Book, Slides)
• Overview of the Advanced Qualification Program (by T. Longridge)
• Data Analysis and Reporting Tool (DART)
• Data Cleaner (used in conjunction with DART) Generic Documents:
• Generic AQP Application
• Generic Single Visit Training Program Application
• Generic Single Visit Training Plan
• Generic Pilot Job Task List
• Generic Pilot Job Task Analysis
• Generic F/A Job Task List
• Generic Curriculum Development Methodology Document
Par 1-5 Page 7 (and 8) FIGURE 1-1.
OTHER RESOURCES (CONTINUED) Research Papers:
• Scientific Evaluation of Aircrew Performance
• Application of Psychometrics to the Calibration of Air Carrier Evaluators [with UNM]
• Data Warehousing for AQP Databases
• Prototype AQP Data Analysis Process
• Training for Inter-Rater Reliability: Baselines & Benchmarks
• Line Operational Evaluation (LOE) Air Carrier Scenario Based Evaluation
• Performance Measurement Issues and the AQP Database Design
• Skills Development and Assessment in the AQP Environment
• The Importance of Quality Data In Evaluating Aircrew Performance
• Questions Your Database Should Address
• A Guide to the Evaluation Aspects of Entering AQP
• Predicting Behavioral Grades from Physical Flight Data for Rejected Takeoff
• Inferential Statistics and Sample Size: Applications to Air Carrier Training and Assessment
• Initiatives Towards More Affordable Flight Simulators for U.S. Commuter Airline Training
• Simulator Fidelity Requirements: The Case of Platform Motion
• Simulator Platform Motion – The Need Revisited
• Simulator Fidelity: The Effect of Platform Motion
• Relationship Between Objective Measures of Pilot Performance/Behavior and Instructor Grades
• The Effect of Simulator Motion on Pilot Training and Evaluation
• Realistic Radio communications in Pilot Simulator Training
• Air Traffic Control in Airline Pilot Simulator
• Rapidly Reconfigurable Event Set-Based Line Operational Evaluation Scenarios
• Do Three Easy Tasks Make One Difficult One? Studying the Perceived Difficulty of Simulator Scenarios
• Differences in General and Commercial Aviation Automation
• An Information-Processing Model Approach to Identifying Training Issues for Advanced Aviation Automation
• Do Pilots Know How They Manually Control the Vertical Flight Path?
• CRM in the Model AQP: A Preview
• The Use of ASRS Incident Reports in AQP CRM Training
• The Model AQP Database Process: Challenges in Process and Design
• AQPing CRM
• Implementing the Model AQP Database: Lessons Learned
• Data Collection and Analysis: The Next AQP Frontier
• Reconceptualizing Leadership and Followership for Event-Based Training Research Integrations:
• Training Approaches and Considerations For Automated Aircraft: A Summary of Training Development Experiences
Par 2-1 Page 9 CHAPTER 2. AQP DEVELOPMENT PHASES SECTION 1. INTRODUCTION
2-1. PHASES. The development of an Advanced Qualification Program (AQP) is a five-phase process. This process provides a structured building-block approach to program development.
This approach results in a program with fully documented curriculums, supporting rationale, and development methodology. The use of this standardized AQP development process, along with its documentation requirements, allows an AQP applicant to develop training and evaluation programs based on analyzed and justifiable requirements. The documentation from this five-phase process will then serve as the regulatory basis for promulgating the AQP-developed training and evaluation programs. Each phase is approved sequentially. Each phase consists of specific activities, including the documentation of those activities. These documents must also be FAA-approved. An applicant cannot exercise any training and evaluation provisions permitted in Phases III, IV, or V, respectively, until each preceding phase is completed and approved by the FAA Extended Review Team (ERT) (see Figure 2-1).
2-2. INSTRUCTIONAL SYSTEMS DEVELOPMENT (ISD). The FAA has incorporated the use of the systematic methodology known as ISD into the AQP process. Applicants may employ any of a wide range of current ISD models or customize their own approaches to curriculum development. This chapter provides one acceptable methodology, but applicants are encouraged to be creative in tailoring their own ISD approach to their requirements, subject to FAA approval. Innovation and practical application may result in equally acceptable AQPs. Because some ISD models are far more complex than others, the FAA has found it useful to define its minimal ISD requirements. These are:
a. Develop a job task listing.
b. Analyze that listing to determine essential skill and knowledge requirements (either directly or by reference).
c. Determine which skill and knowledge requirements must be trained/tested.
d. Develop proficiency objectives that capture all training requirements.
e. Develop qualification standards that define acceptable operational performance levels.
f. Develop tests that measure proficiency in skill and knowledge areas.
g. Provide instructional programs that teach and test training requirements.
h. Establish and maintain an audit trail of explicit links between task requirements, training requirements, training and evaluation activities, and evaluation results.
i. Measure student performance against proficiency objectives and qualification standards for all curriculums.
j. Revise the training program based on student performance levels on an ongoing basis.
This de-identified data (stored in the Performance/Proficiency Database) will be collected and reported to the FAA on a regular basis.
2-3. TRAINING SYSTEMS. While all applicants must meet the minimum requirements of the ISD approach, the amount of effort each applicant should put into each step of their training analysis and development depends on a number of factors. Applicants may use the ISD process to build complete training systems from the ground up, to build a proficiency-based quality control shell around an existing training system, or to make minor modifications to existing proficiency-based curriculums. Building a training system from the ground up for a new aircraft will require far more depth of analysis and development than merely modifying a current curriculum for an existing aircraft.
FIGURE 2-1.
AQP PHASE DEVELOPMENT PROCESS Joint Airline & FAA AQP Meeting Phase I Develop/Update AQP Application AQP Development Apply For Single Visit Exemption Single Visit Exemption (optional) Develop Single Visit Training Plan (SVTP) Reviews & Approves FAA Job Task Analysis Phase II Qualification Standards AQP Document Development & ISD Methodology & Curriculums Implement the SVTP Implementation & Operations Plan Reviews & Approves FAA Train Instructor & Evaluators Small Group Tryout Phase III • Revise Training Program & AQP Documents Initiate Performance/Proficiency Data Process Reviews & Approves FAA Conduct Initial Training Program Evaluation Phase IV Submit Monthly Performance Data Initial Operations Begin Revise Training & AQP Documents Submit Annual Report Reviews & Approves FAA Conduct Initial Training Program Evaluation Phase V Submit Monthly Performance Data Continuing Operations Revise Training & AQP Documents Submit Annual Report SECTION 2. PHASE I: APPLICATION
2-4. JOINT MEETING. AQP begins with a letter from the applicant, to the Principal Operations Inspector (POI), a duplicate copy of which is forwarded to the FAA manager of the Par 2-2 Page 10
Par 2-4 Page 11 appropriate Safety Standards office stating their intent to develop an AQP. The appropriate Safety Standards office manager will discuss the applicant’s plans with the POI. The appropriate Safety Standards office manager will respond to the certificate holder, through the POI, with a Letter of Acknowledgment requesting a meeting with representatives of the certificate holder, the appropriate Safety Standards office manager, and the responsible Flight Standards office inspectors responsible for oversight. The inspectors should include the POI or the inspector assigned to be the point of contact for AQP development. If the applicant is a training center, a Training Center Program Manager (TCPM) would represent the responsible Flight Standards office. The focus of this meeting is to ensure that all parties understand AQP, specifically the responsibilities and the commitment needed to complete the project. Discussions will include the anticipated benefits of AQP, AQP entry and exit strategies, the development process requirements, possible problem areas, and available tools. Notification of entry into AQP Phase I, Application Development, will be by letter from the appropriate Safety Standards office manager to the applicant through the POI.
2-5. THE AQP APPLICATION. The purpose of the application is to provide an overview of the applicant’s plan for developing an AQP for all of its fleets, instructors and evaluators, and for non-fleet specific curriculum (i.e., indoctrination). The application is submitted once and is updated as information in the application warrants changing (e.g., a change in the schedule, adding new aircraft, initiating flight attendant (F/A) or dispatcher AQP programs, etc.). To establish the applicant’s intent and approach for developing an AQP, the application should address thoroughly the following topics:
a. Letter of Application (See Appendix 2). The letter should clearly state the certificate holder’s intent to develop, implement, and operate an AQP.
b. Data Collection, Submission and Analysis Reporting. The AQP applicant must acknowledge their understanding and acceptance of the AQP Performance Data requirements as defined in Chapter 8 of this advisory circular (AC).
c. Operating Environment Description. Applicants should describe their operating environment, including the general physical environmental factors expected to be encountered in operations. Environmental factors are critical to development of Line-Operational Simulation (LOS) scenarios and meaningful proficiency objectives. Environmental factors include:
(1) Weather norms and extremes.
(2) Normal, abnormal, and emergency equipment operation.
(3) Geographic areas of operations.
d. Training Equipment Description and Location. The application should describe the training equipment, its location, and the organization responsible for its security and maintenance. If the type of training equipment for the AQP is not known at the time the application is submitted, the application will be updated when the training equipment requirements are identified. The FAA identification number assigned by the National Simulator Program manager and the level of qualification identifies flight simulators and/or flight training
Par 2-5 Page 12 devices. Other training devices will be listed and if qualification is required, the applicant should state when it intends to submit a test guide and a request for equipment qualification.
e. Facilities Description. Each AQP submission should describe the facilities the applicant intends to use. The description should include the location, type of facility, classrooms, training aids, courseware, and other features that contribute to creating and maintaining a positive learning environment.
f. Trainee Demographics. The application should provide a summary of demographic data on the experience level of the pilots, flight engineers (FE), F/As, and dispatchers that will be trained under AQP. This data includes the current ground and flight instructors and evaluators that are expected to continue these functions under AQP. For each fleet, this information would identify the:
(1) Entry requirements for ground and flight instructors and evaluators.
(2) Entry requirements for new hires.
(3) Students should be identified as a group in terms of previous experience.
(4) Current and anticipated need for replacement crewmembers by duty position.
g. AQP Organization. This section explains the organizational resources that the air carrier plans to employ for AQP development. Generally, an AQP applicant’s staff should support the following functions:
(1) AQP Coordination. The company focal point for the AQP development effort.
Responsible for AQP leadership and is the primary contact with the FAA and any other external organizations that may impact the applicant’s AQP. This position should be filled with an individual who understands job task analysis and the training requirements of Title 14 of the Code of Federal Regulations (14 CFR).
(2) Subject Matter Expertise. Crewmembers, dispatchers and other operations personnel who are current and qualified in company operations and represent the population of professionals the AQP will address. The individuals may be called upon to act as liaison with operational support personnel.
(3) Document and Curriculum Development. Interfaces with the AQP Coordinator and the subject matter experts (SME) to develop the requisite AQP process, curriculum, and instructor/evaluator documents.
(4) Document Management. Ensures AQP document control and congruence with FAA requirements.
(5) Computer Support/Database Management. Oversight of the development and management of the performance/proficiency data acquisition and analysis system. In addition, this individual could be used for other computer-related issues related to the facilitation of an AQP, such as electronic document review.
Par 2-5 Page 13
h. Master AQP Transition Schedule (MATS). The MATS will depict the projected transition/development schedule for all AQP curriculums. Since these schedules usually change, this section of the application must be updated accordingly. Transition from one type training program to another (traditional to Single Visit Training Program (SVTP) to AQP, or traditional to AQP) may include a period of overlap as one program is phased in and the other phased out.
The following guidelines for transition are applicable:
(1) Currently qualified personnel may transition between traditional recurrent training curriculums and continuing qualification curriculums.
(2) Personnel who have completed a traditional training program initial, transition or upgrade curriculums may enter a continuing qualification curriculum.
(3) Partial MATS (incomplete) are not acceptable.
(4) The MATS may provide for incremental implementation of indoctrination, qualification, and continuing qualification curriculums for each AQP.
(5) The MATS must provide the time frame necessary to withdraw from AQP if it becomes necessary to revert to the traditional 14 CFR part 121 or 135 training program.
2-6. APPROVAL. Approval of the application marks the end of Phase I and the beginning of Phase II.
SECTION 3. PHASE II: CURRICULUM DEVELOPMENT
2-7. OVERVIEW. Phase II is the development of the training curricula that apply to any of the duty positions addressed in the Phase I application (e.g., pilots, F/As, dispatchers, instructor, evaluators). There are five general stages in this developmental process, each represented by the required document listed below:
• Job Task Analysis (JTA).
• Qualification standards.
• Instructional Systems Development (ISD) methodology.
• Curriculum outlines.
• Implementation and Operations Plan (I&O Plan).
The task analysis is the basic document that supports the development and analysis of the qualification standards. The ISD methodology defines how the task analysis and qualifications standards will be used to support the development of the curriculums. The last document is the I&O Plan, which explains how the AQP will be put into practice. The applicant must establish, demonstrate, and maintain a clear linkage between each of these steps. This linkage is provided by a systematic approach to the development of a complete instructional system. This section recommends a systematic approach and a methodology that is acceptable to the FAA, but innovation and practical application may result in equally acceptable variations.
Par 2-7 Page 14
a. Rationale. Under traditional training programs, the FAA evaluates finished training curriculums by comparing their contents, as described in a curriculum outline, to guidance in the regulations and FAA Order 8900.1. Under an AQP, the ERT evaluates these curriculums by monitoring their development and by approving a series of documents that are preliminary to the curriculum outline. These documents include the task analysis, the qualification standards, and the ISD Methodology. This continuing process allows the FAA greater insight into the rationale used by the applicant to develop each component of each curriculum.
b. Analysis-Based Approach. The analysis-based approach allows each applicant the opportunity to develop operator specific training programs. Consequently, AQP curriculums will be more sensitive to differences in aircraft, operating conditions, emergency and abnormal contingencies, student skill levels, and other operational variables, than are traditional programs.
Under an AQP, the FAA monitors the process as well as the product. Instead of basing curriculums on prescribed generic maneuvers, procedures and knowledge items, AQP curriculums are based on a detailed analysis of the requirements of each duty position in each organization. To obtain FAA approval, these new AQP curriculums must be judged to be equal to, or provide better training than, a traditional training program. AQP curricula do not need to duplicate traditional curricula in order to satisfy this criterion, provided that they otherwise incorporate an alternative approach that addresses the required skills and knowledge to perform the job tasks.
2-8. JTA. To understand task analysis as it applies to AQP, it is necessary to build upon the following definitions:
• Job: A job is the summation of functions, identified as tasks and subtasks, performed by an individual at work.
• Function: One of the major subdivisions of work activities performed by one individual. One or more functions constitute a job. Examples of functions applicable to AQP would be: phases of flight, such as takeoff, climb, cruise, etc.
• Task: A task is a unit of work within a function having an identifiable beginning and ending point, which results in a measurable product. An example of a task applicable to AQP would be: perform a normal takeoff.
• Subtask: Specific separate step or activity required in the accomplishment of a task. An example of a sub-task applicable to AQP would be: perform rotation and liftoff.
• Element: A component of training analysis or design. In the case of task analysis, the element may be used as a level of analysis: phase of flight, task, subtask, element, sub-element (a further component of training analysis necessary in the accomplishment of a subtask (e.g., rotate aircraft at VR to 12 degrees of pitch)). In the case of curriculum design, the element may be used as a level of curriculum organization: curriculum, segment, module, lesson, lesson element, etc.
JTA is the method or procedure used to reduce a unit of work to its base components. The JTA document consists of a detailed, sequential listing of tasks, subtasks, and elements (if required) with the knowledge and skills (technical and Crew Resource Management (CRM)) that clearly define and completely describe the job. The following is a description of the above components: