Par 2-14 Page 33
d. Performance/Proficiency Data Submission. The applicant will submit to the appropriate Safety Standards office manager, on a monthly basis, the required performance and proficiency data. This data will be in electronic, digitized format. Any changes to format or procedures will be discussed with the appropriate Safety Standards office manager prior to submission. The monthly submission of this performance/proficiency data will continue for the life of the curriculum’s lifetime (i.e., through Phases III, IV, and V). See Chapter 8 for more information.
e. Program Revisions. Lessons learned as a result of this phase will be incorporated as changes to the approved AQP documents from Phase I and II and in curriculum maintenance reports submitted to the FAA on an annual basis. Improvements will be made to the courseware, hardware, software, personnel, organization, and data collection and reporting system of the AQP, based on system performance and the examination of performance and proficiency data.
These enhancements will be implemented using the processes and the procedures described in the applicant’s approved AQP maintenance strategy. This plan is part of the I&O Plan.
2-15. APPROVAL. Approval of the initial operations marks the end of Phase III and the beginning of Phase IV.
SECTION 5. PHASE IV: INITIAL OPERATIONS
2-16. OVERVIEW. In this phase, the applicant implements the indoctrination, qualification, or continuing qualification curriculum in accordance with the approved AQP documents. This phase for the continuing qualification curriculum calls for initial operation of the curriculum for a minimum of 2 years to complete the AQP training cycle. The 2-year Phase IV period may not necessarily be applicable to the indoctrination and qualification curriculums. Unlike the Continuing Qualification curriculums, these courses of training are not cycle-dependent.
Depending on the frequency that each curriculum is applied (new hires) and the successful results of each curriculum, Phase IV may be longer or shorter than 24 months.
2-17. PHASE IV ACTIVITIES. During Phase IV, the applicant will implement and complete a full evaluation, including collecting program audit data and individual performance/proficiency data, analyses, and reports. Collected data will be used by:
a. The applicant for its internal quality control program to maintain curriculum and courseware concurrence, suitability, and adequacy.
b. The applicant to analyze and validate crewmembers, dispatchers, and other operations personnel performance.
c. The applicant and the FAA to analyze and validate instructor/evaluators performance.
d. The applicant and the FAA to support analysis for special subjects, such as CRM performance factors.
Par 2-17 Page 34
e. The FAA to analyze and validate curriculum performance.
f. The FAA to analyze and validate program development, implementation, and maintenance procedures.
2-18. REVISIONS. The focus of this phase is the validation of the AQP curriculum by acquiring performance/proficiency data and lessons learned in the course of conducting the curriculum. Applicants will summarize the lessons learned and adjustments made to the curriculums in an annual report. In addition, adjustments made to the AQP will be reflected in revisions to the approved AQP documents. The successful completion of this phase, and approval of the commensurate revisions to the approved AQP documents and reports, will qualify an applicant for entry into the final phase of the AQP process: Phase V, Continuing Operations.
2-19. APPROVAL. Approval of the continuing operations marks the end of Phase IV and the entrance into Phase V.
SECTION 6. PHASE V: CONTINUING OPERATIONS
2-20. OVERVIEW. In this phase, the applicant continues operation of the AQP unless the FAA withdraws approval or unless the applicant withdraws or modifies the AQP. This phase requires the maintenance of the AQP-approved documentation and updated documentation of the data requirements for all curriculums.
2-21. DATA’S IMPACT ON CONTINUING OPERATIONS. Data will continue to be collected and analyzed by the applicant and the FAA as it was during Phase IV activities.
2-22. QUALITY ASSURANCE. Applicants must pay particular attention to overall program quality. The FAA expects an AQP quality assurance program to identify needed changes in curriculum, courseware, and equipment, and to make these changes before an unwanted trend in reduced proficiency manifests itself. Continued validation of performance/proficiency data as it pertains to individual and crew proficiency, as achieved and maintained by all personnel, is particularly important.
SECTION 7. DOCUMENT AND REPORTING REQUIREMENTS
2-23. OVERVIEW. The approved AQP documentation establishes the applicant’s regulatory requirements for the particular training program. This fact makes it imperative that the applicant develops a document structure that is organized and standardized. The structure must ensure that the information contained within the AQP documents will be used and translated down to the lowest level training guidance. It must allow the applicant’s entire organization to easily access and use the contents. It is important to draw a distinction between the AQP process and the required FAA documentation each applicant must provide to ensure regulatory compliance. The FAA has established a minimum listing of AQP documentation. The applicant will want to develop other more specific materials that ensure the approved AQP information distills to all training and evaluation guides.
Par 2-23 Page 35
a. FAA-Required Documentation. Each document contains unique information integral to the AQP that will be used when developing subsequent curriculums. The documents, once developed, will require periodic updates and, therefore, are subject to the revision control process discussed in Chapter 4. These documents should be submitted to the FAA in electronic format:
(1) The application.
(2) The task analysis.
(3) The qualification standards.
(4) The instructional systems development methodology.
(5) The curriculum outlines.
(6) The I&O Plan.
b. Document Structures. Figure 2-8 provides an example of an AQP document structure.
It is not required that an applicant follow this structure. However, the document structure that is adopted must easily identify the location of the AQP documents and more specifically, the required AQP information. If the AQP documentation is part of a subset of other manuals, a method should be developed to identify the manual and the specific AQP document that it contains.
c. Annual AQP Report. AQP requires that each AQP certificate holder prepare an annual report for the FAA (see Appendix 4 for specific requirements). This report is based on the certificate holder’s analysis of the data that is collected during training and at strategic points (validation/evaluation gates) in each curriculum and maintained in the PPDB. AQP requires data collection and analysis in order to establish and maintain quality control of curriculums for crewmembers, instructors, and evaluators. The annual AQP report should summarize the lessons learned and adjustments made to the curriculum(s) during the reporting period. The report should also include projected or proposed changes to the curriculum(s) based on the certificate holder’s current analysis. The actual adjustments made to the AQP are reflected in revisions to the approved AQP documents. The report should be submitted to the FAA no later than 60 days past the end of the report period. The reporting period is usually based on the approval date for a particular curriculum in either Phase IV or V. During AQP development, particularly for multiple fleet operators, with different approval dates for multiple curriculums, the reporting period may be modified (as agreed upon by the FAA and the certificate holder). Once the certificate holder has all its fleets and curriculums into Phase V, the reporting period can be fixed into a particular cycle. Copies of the report should be distributed to the POI and the appropriate Safety Standards office at least 2 weeks prior to the annual AQP review meeting.
d. Annual AQP Review—Phases IV and V. An annual AQP review meeting between the FAA and the certificate holder should be held to coincide with the submission of the annual AQP report in Phases IV and V. The purpose of the meeting is to formally discuss the results of the certificate holder’s data analysis, program revisions, future revisions and the analysis of the data that is submitted to the FAA. Below is a listing of suggested agenda items for a Phase IV and V review:
Par 2-23 Page 36
(1) Data Management.
(a) Collection problems and fixes.
(b) Analysis—data reliability/validity/sensitivity.
(c) Data usefulness.
(d) Problem areas investigated.
(2) Data Collection Method/Tools.
(3) Data Analysis Methods/Tools.
(a) Review of annual report.
1. Qualification.
2. Continuing qualification.
3. Line check.
(b) Identified trends (positive and negative).
(c) Corrective measures.
(4) Program Critique Summaries.
(5) Recordkeeping. Do records consistently demonstrate qualification of crews and instructor/evaluators?
(6) I&O Plan Adherence.
(7) Modifications to the Program.
(a) Due to PPDB input.
(b) Due to other input.
1. Demographics.
2. Operational.
(8) Validity and Usefulness of Qualification Standards.
(9) AQP Maintenance Strategy—Is the Described Process Working?
(a) Any changes to the maintenance strategy.
(b) Currency of PPDB.
Par 2-23 Page 37 (and 38)
(10) Instructor Evaluator Programs.
(a) Inter/Referent rater reliability data.
(b) Question and answer observations summarized.
(c) Methods to maintain I/E Standardization.
(11) FAA Surveillance Findings.
(12) AQP Challenges and Difficulties.
(a) Progress towards Phase III, IV, and V in other fleets.
(b) Special tracking.
(c) Seat substitution.
(d) Line-Oriented Flight Training (LOFT)/LOS scenarios for crewmembers.
(13) Use of Information from Related Programs, If Any (e.g., Flight Operational Quality Assurance (FOQA), Aviation Safety Action Program (ASAP), etc).
FIGURE 2-8.
DOCUMENT STRUCTURES CHART Title List of Effective Pages Section I – All Organizational Standardized Areas Application Instructional Systems Development Methodology Implementation and Operations Plan Section II – Indoctrination Training Curriculums Curriculum Outline Section III – Qualification Training Curriculums Task Analysis Qualification Standards Curriculum Outline Data Acquisition/Grading Forms Section IV – Continuing Qualification Training Curriculums Curriculum Outline Data Acquisition/Grading Forms
Par 3-1 Page 39 CHAPTER 3. AQP CURRICULUMS AND CERTIFICATION SECTION 1. CURRICULUMS
3-1. BACKGROUND. The Advanced Qualification Program (AQP) requires three primary curriculums for each make, model, and series of aircraft (or variant), and each duty position.
These curriculums are indoctrination, qualification, and continuing qualification. Figure 3-5 illustrates how traditional curriculums correlate to the AQP primary curriculums. In addition to primary curriculums, operational necessities may require secondary curriculums to fulfill particular needs. Secondary curriculums are transition, conversion, upgrade, and requalification (see Figure 3-6).
3-2. INDOCTRINATION CURRICULUM. An indoctrination curriculum consists of all training elements that will be learned and evaluated before an individual may begin a qualification curriculum. Indoctrination curriculum segments typically consist of ground training and evaluation. Two distinct areas of indoctrination ground training are:
a. Certificate Holder-Specific Training. This training acquaints crewmembers, dispatchers, instructors, evaluators, and other operations personnel with company policies, practices, and general operational knowledge. The subject matter of indoctrination includes elements that pertain to the certificate holder’s methods of compliance with regulations and safe operating practices.
b. Duty Position-Specific Training. This training provides the basic aeronautical knowledge needed to enter the subsequent qualification curriculum. Weather, the regulations, security, some emergency, and hazardous material training (usually part of indoctrination) is partially certificate holder-specific and partially duty position-specific.
3-3. QUALIFICATION CURRICULUM. AQP requires a qualification curriculum for each duty position in each make, model, and series aircraft (or variant). Each Qualification Curriculum will include training, validation, and evaluation. The training activities include ground and flight training, operational experience, and may include special qualification.
Figure 3-1 illustrates the relationship between training activities and validation points. If the training is to result in airman certification or the addition of category, class, instrument, or type ratings, curriculum segments must explicitly identify the training and evaluation strategy to be used in place of the prescribed practical test requirements of Title 14 of the Code of Federal Regulations (14 CFR) part 61, 63, or 65. The applicant must show, to the FAA’s satisfaction, that the proposed AQP training and evaluation strategy will ensure individual competency that equals or exceeds the applicable Airman Certification Standards (ACS) or practical test standards (PTS) and that each person certificated through an AQP has demonstrated satisfactory proficiency in the integration of technical and Crew Resource Management (CRM) skills.
a. Training Activities.
(1) Ground Qualification Training Activities. To be qualified for a particular duty position, a person will receive job-specific ground training. This training typically includes general operational subjects, technical systems, system and procedures integration, and emergency-type training. Ground qualification culminates in a systems knowledge validation
Par 3-3 Page 40 session that may take the form of a traditional oral examination or may incorporate another means of systems knowledge validation (e.g., computer-based, written, etc.), as approved by the FAA.
(2) Simulation/Flight Qualification Training Activities. Each AQP includes curriculum segments for training, validation, or evaluations, as appropriate in flight training devices (FTD) and simulators. The use of FTDs for training and evaluation is encouraged.
However, the maneuvers validation must be accomplished in a full flight simulator that has been approved for its intended use. Training and evaluation in an aircraft is discouraged, but may be approved by the FAA on a case-by-case basis.
(3) Special Qualification Training. Curriculum segments may include special purpose training. These are portions of ground or flight training that has specific application to crewmembers who are in international operations or for introduction of new flight operations, such as Category III (CAT III) approaches. Special purpose training and validation may initially be a separate curriculum segment that is later integrated into ground and flight training segments.
b. Validation/Evaluation/Remediation. In AQP, validation is a determination that the training produces the required results as identified in the qualification standards and that the individual has met the performance objectives of the training module. Except for the Qualification Curriculum Maneuvers Validation session, additional training may occur during a validation session to ensure achieving the training-to-proficiency objectives. An evaluation is an appraisal of an individual to ascertain whether the standards required for a specified level of proficiency has been successfully demonstrated. Interrupting the evaluation session for training is not permitted. Both validation and evaluation are assessments that the proficiency objectives of the training module have been met and the individual can proceed to the next level of training or line operations. Figure 3-2 contains a table that summarizes the validation/evaluation gates and the associated remediation.
(1) Systems Knowledge Validation. This is an assessment of an individual’s technical systems knowledge. The intent of the systems knowledge validation session is to ensure an individual’s systems knowledge is at an appropriate level before progressing into the next training phase. Systems knowledge validation may be accomplished by a written, electronic or oral exam. If a written or electronic test system is used, an overall score of 80 percent or better, corrected to 100 percent, would be an acceptable validation. An overall score of less than 80 percent will require retraining and another complete test. A failure of an individual test module or subsection, with an 80 percent or better overall score, only requires retraining and retesting of the specific module. Consideration should be given to establishing a maximum number of modules or subsections that if failed constitute an overall failure of the validation.
(2) Procedures Validation. This is an assessment of an individual’s systems integration knowledge and skill. This validation addresses the individual’s ability to assimilate system and procedural knowledge into the appropriate execution of procedures. This validation session should take place in an FTD or a simulator. The intent of the procedures validation session is to ensure an individual’s systems and procedural knowledge and skill is at an appropriate level before progressing into the flight simulator training phase. There is no established requirement that the systems and procedures validation be accomplished
Par 3-3 Page 41 sequentially. It is the prerogative of each certificate holder to develop an evaluation system that works for them. For example, the systems validation could take place in two parts. The first part may be a written exam administered at the conclusion of ground school. The second stage could be an oral conducted at the conclusion of procedures training, before the maneuvers validation or before the Line Operational Evaluation (LOE). Validation is accomplished when it is verified that the individual is trained to proficiency.
(3) Maneuvers Validation. This validation addresses the individual’s proficiency in the execution of maneuvers. It must take place in a simulator. For a qualification curriculum, crewmembers are expected to have reached a satisfactory level of proficiency in the maneuvers prior to the validation event. Qualification maneuvers validation should not allow more than two repeats of any one maneuver or one repeat of any two maneuvers. A debriefing of why the maneuver(s) was unsatisfactory is allowed, but the repeats must occur with no training, practice, or coaching. If the crewmember fails to demonstrate proficiency in the time constraints of the simulator session, an additional training session is required. After additional training, the individual need only repeat the maneuvers that were unsatisfactory. Failure of the maneuvers validation or failure to complete the maneuvers validation during the scheduled simulator period does not result in the issuance of FAA Form 8060-5, Notice of Disapproval of Application.
(4) LOE. This evaluation addresses the individual’s ability to demonstrate technical and CRM skills appropriate to fulfilling job requirements in a full mission scenario environment.
The intent of an LOE is to evaluate and verify that an individual’s job knowledge, technical skills, and CRM skills are commensurate with AQP qualification standards and that the individual is qualified to begin the Operating Experience (OE) portion of the qualification curriculum. LOE is considered a jeopardy event and a failure is reported to the FAA. If the LOE is to result in certification, a failure will result in the issuance of FAA Form 8060-5. Grading of an LOE is at the event set level. LOE success criteria is listed below:
(a) An LOE with more than 25 percent of the event sets graded unsuccessful would constitute a failure of the LOE and will require remedial training and another LOE.
(b) For LOEs with 25 percent of the event sets or fewer graded unsuccessful, repeats may be permitted at the end of the LOE session, if time permits and if it is possible to recreate the conditions similar to the original event set for the repeat. For example, if an LOE has five, six, or seven event sets, only one set may be repeated. If an LOE has between 8 and 11 event sets, two may be repeated. No event set can be repeated more than once. A debriefing of why the event set(s) is unsatisfactory is allowed, but the repeat must occur with no training, practice, or coaching. If any repeated event is unsatisfactory, remedial training and another complete LOE evaluation is required.
NOTE: Regardless of number of events sets, unsafe individual or crew performance that would result in significant damage, hull loss, or loss of life
(e.g., crash) during an LOE constitutes a failure of the LOE.
NOTE: An aircrew program designee (APD) or FAA inspector must administer all flightcrew member qualification curriculum LOEs. A check airman can administer continuing qualification LOEs.
Par 3-3 Page 42
(5) Operating Experience (OE). OE curriculum segments are integral to qualification curriculums. OE provides hands-on experience in performing the duties of a newly assigned position under the supervision of a current and qualified evaluator (check airman). OE is conducted during actual flight operations. Operating experience is complete when the individual is recommended for a line check.
(6) Line Check. Captains receiving this evaluation are assessed for their proficiency in the duty position. Successful completion of the line check verifies that the individual is adequately trained and is capable of performing the duties and responsibilities of pilot in command (PIC). If any task is unsatisfactory, the individual must receive remedial training on that task, additional operating experience if necessary, and possibly another line check. If a pilot receives an unsatisfactory overall performance rating on a line check, the pilot must be removed from continued line operations until the approved remediation has been successfully completed.
c. Planned Hours. All curriculums will include planned hours for ground training, flight training, evaluation, and operating experience. Planned hours represent the estimated amount of time (as specified in a curriculum outline) that it takes an average student to complete a segment of instruction (to include all instruction, demonstration, practice, and evaluation, as appropriate, to reach proficiency). Planned hours replace the programmed hours associated with traditional programs. Planned hours enable the FAA and the applicant to schedule their personnel resources more efficiently and provide a baseline for curriculum adjustments. The sum of planned hours is not used by the FAA as a basis for program approval, review, or compliance assessment. Planned hours can be shown on the course footprint that is part of the curriculum outline. Figure 3-3 provides an example of a course footprint with planned hours.
Par 3-3 Page 43 FIGURE 3-1.
FLIGHTCREW MEMBER AQP TRAINING, VALIDATION, AND EVALUATION Activity Device Purpose Can it be Interrupted? Training/ Validation or Evaluation Sequencing of Events Ground Training Classroom or Computer Based Instruction, and Training Devices Indoctrination and Systems Training Yes Training & Validation Syllabus Maneuvers and Procedures Training Flight Training Device & Simulator Aircraft Maneuvers and Operational Procedures Yes Training & Validation Isolated Maneuvers and Procedures.
Logical Specific Sequencing of Events Special Purpose Operational Training (SPOT) Flight Training Device & Simulator Focuses on CRM Skills, Differences Tng., Wind shear Tng., Special Qualification Yes Training Isolated Maneuvers and Procedures.
Logical Specific Sequencing of Events Line-Oriented Flight Training (LOFT) Flight Training Device & Simulator Crew-Oriented Training (CRM) in Preparation for LOE No Except To Begin Different Scenarios Training Logical Sequencing of Events Within Varying Length Scenarios Line Operational Evaluation (LOE) Flight Training Device & Simulator Evaluate the training and qualifications of the crewmembers May Be Segmented to Condense Distances as in International Scenarios Evaluation Specific Flight Scenario From Takeoff to Landing Operating Experience Aircraft Consolidation of knowledge & skills in a operational environment Yes Experience Routine Flight Operations Initial Line Check Aircraft To verify a crewmember’s ability to satisfactorily perform duties & responsibilities No Evaluation Routine Flight Operations CQ Line Check Aircraft To evaluate crew proficiency and their knowledge, skills, and ability to operate effectively as part of a crew.
No Evaluation Routine Flight Operations
Par 3-3 Page 44 FIGURE 3-2.
AQP VALIDATION/EVALUATION TABLE Gate Indoctrination Curriculum Qualification Curriculum Continuing Qualification Curriculum Evaluation Media System Knowledge Validation 80% or Greater corrected to 100%, < 80% retest 80% or Greater corrected to 100%, < 80% retest 80% or Greater corrected to 100%, < 80% retest Written, Oral, or Electronic Testing System (ETS) Procedures Validation Train to Proficiency As Approved Maneuvers Validation 2 Repeats of One Maneuver, or 1 Repeat Of Any 2 Maneuvers* Repeats Allowed Within Training Period Approved Simulation Device LOE Grading By Event Set 5, 6, or 7 Event Sets – Repeat 1 Event Set – 8 or more Event Sets Repeat - 2 Event Sets ** Grading By Event Set 5, 6, or 7 Event Sets – Repeat 1 Event Set – 8 or more Event Sets Repeat - 2 Event Sets ** Approved Simulation Device Operating Experience
Meets Approved Number of Supervised Cycles or Hours and Recommended for Initial Line Check Aircraft (Actual Line Operations) Line Check If Any Task Unsatisfactory, Remedial Training Needed, Additional Operating Experience if Needed, and Another Line Check as Recommended by the Evaluator If Any Task Unsatisfactory, Remedial Training Needed, Additional Operating Experience if Needed, and Another Line Check as Recommenced By the Evaluator Aircraft (Actual Line Operations) * For any repeat in a qualification maneuvers validation, qualification LOE or CQ LOE, no training, practice, or coaching is permitted. Training repeats are allowed in a continuing qualification maneuvers validation and are not counted as an evaluation repeat.
** Failure to meet the required level of proficiency during an LOE or line check will result in retraining, reevaluation, and will require special tracking. Regardless of number of events sets, unsafe individual or crew performance that would result in significant damage, hull loss or loss of life (e.g., crash) during an LOE constitutes a failure of the LOE.
Par 3-3 Page 45 FIGURE 3-3.
EXAMPLE OF AQP FLIGHTCREW QUALIFICATION CURRICULUM FOOTPRINT Day 1 Day 2 Day 3 Day 4 Day 5 Day Off Day Off Welcome :30 Intro :30 CBT 5:00 TT 6:00 CBT 5:00 Review 1.00 TT 6.00 CBT 5:00 Review 1.00 TT 6.00 CBT 5:00 Performance 1.00 TT 6.00 Evacuation 3:00 Ditching 3:00 TT 6:00 Day 6 Day 7 Day 8 Day 9 Day 10 Day Off Day Off CBT 5:00 Setup Lect. 1.00 TT 6.00 CBT 3:00 FTD Brief 1.00 FTD # 1 2.00 TT 6.00 CBT 5:00 Review 1.00 TT 6.00 CBT 3:00 FTD Brief 1.00 FTD # 2 2.00 TT 6.00
SYSTM VAL Testing 1:00 Flt Ops Brf. 4:00 Debrief :30 TT 6:30 Day 11 Day 12 Day 13 Day 14 Day 15 Day Off Day Off Sys Rev 1:00 FTD Brief 1.00 FTD # 4 4.00 TT 6.00 FTD Brief 1:30 FTD # 5 4.00 Debrief 1:00 TT 6.30 FTD Brief 1:30 FTD # 6 4.00 Debrief 1:00 TT 6.30 FTD Brief 1:30 FTD # 7 4.00 Debrief 1:00 TT 6.30
PROC VAL FTD Brief 1:30 FTD # 8 4.00 Debrief 1:00 TT 6.30 Day 16 Day 17 Day 18 Day 19 Day 20 Day Off Day Off Sim Brief 1:30 Sim # 1 4.00 Debrief 1:00 TT 6.30 Sim Brief 1:30 Sim # 2 4.00 Debrief 1:00 TT 6.30 Sim Brief 1:30 Sim # 3 4.00 Debrief 1:00 TT 6.30 Sim Brief 1:30 Sim # 4 4.00 Debrief 1:00 TT 6.30
MAN VAL Sim Brief 1:30 Sim # 5 4.00 Debrief 1:00 TT 6.30 Day 21 Day 22 Day 23 Day 24 Day 25 Day Off Day Off Loft Brief 1:30 Loft # 1/Spot 4.00 Debrief 1:00 TT 6.30 Loft Brief 1:30 Loft # 2/Spot 4.00 Debrief 1:00 TT 6.30 Loft Brief 1:30 Loft # 3/Spot 4.00 Debrief 1:00 TT 6.30 Loft Brief 1:30 Loft # 4/Spot 4.00 Debrief 1:00 TT 6.30 LOE Brief 1:30 LOE 4:00 Debrief 1:00 TT 6:30 Day 26 Day 27 Day 28 Day 29 Day 30
IOE IOE IOE IOE IOE Day 31 IOE Day 32 Line Check SYST VAL - Systems Validation PROC VAL - Procedures Validation MAN VAL - Maneuvers Validation LOE - Line Operational Evaluation Spot - Special Purpose Operations Training CBT – Computer-based training FTD - Flight training device Sim - Full flight simulator LOFT - Line-Oriented Flight Training IOE - Initial operations experience