Chinese Journal of Aeronautics
Volume 28, Issue 1, February 2015, Pages 152–163
Simulation-based training is a promising way to train a carrier flight deck crew because of the complex and dangerous working environment. Quantitative evaluation of simulation-based training quality is vital to make simulation-based training practical for aircraft carrier marshalling. This paper develops a personal computer-based aircraft carrier marshalling simulation system and a cave automatic virtual environment (CAVE)-based immersive environment. In order to compare the training effectiveness of simulation-based training and paper-based training, a learning cubic model is proposed and a contrast experiment is carried out as well. The experimental data is analyzed based on a simplified Kirkpatrick’s model. The results show that simulation-based training is better than paper-based training by 26.80% after three rounds of testing, which prove the effectiveness of simulation-based aircraft carrier marshalling training.
- Aircraft carrier marshalling;
- Kirkpatrick’s model;
- Learning cubic model;
- Simulation-based training;
- Training effectiveness
Aircraft carrier marshalling is visual signaling between ground personnel and pilots on an aircraft carrier. It is the main communication way between marshallers and pilots on the carrier deck. The traditional training of aircraft carrier marshalling normally consists of two parts: one is learning from operation manuals and the other is practical training on the carrier deck. However, the flight deck is a very complex and dangerous working environment,1 which is full of moving staff, devices, fighters, helicopters and other flight vehicles. So there is a high risk of injuries for marshallers who have not been well trained.
With the development of virtual simulation technology, simulation-based training (SBT) has been used in training situations where it would be too dangerous or logistically impossible to have users participate in an actual event.2 For example, Sastry et al.3 developed a prototype virtual environment for training flight deck officers (FDOs) with a view to study the types of interactions required in such an environment. In this virtual environment, trainee FDO signals the aircraft with arms to move and take off. Buche et al.4 proposed the integration of a generic and adaptable intelligent tutoring system into a virtual environment. The system is created using a multi-agent system, and aimed at instructing the learner and assisting the instructor. More applications were applied to other high risk work fields, such as Kizil’s system5 that helps prepare miners for dangerous situations that could not be addressed through traditional training methods. However, most of these papers are aimed at developing a virtual environment for training, instead of evaluating training quality. Eventually, both developers and users of SBT realize how important it is to scientifically evaluate SBT quality.
How to quantitatively evaluate the effectiveness of SBT is not intuitively obvious. According to a United States Government Accountability Office (GAO) report, Army and Marine Corps have increased the use of SBT and taken steps to collaborate on development efforts. They also cite benefits of SBT, but lack information to evaluate its impact on performance and cost.6
This paper develops a personal computer (PC)-based aircraft carrier marshalling simulation system and a cave automatic virtual environment (CAVE)7-based immersive environment. Considering that PCs are more widely used than CAVE, this paper focuses on evaluating PC-based training. In order to quantitatively evaluate the quality of SBT for aircraft carrier marshalling, a learning cubic model is proposed, as well as a training effectiveness evaluation model based on a traditional Kirkpatrick’s four level evaluation model. Then a contrast experiment is carried out with two groups of subjects and the results are analyzed. Finally, a description of the proposed future work is presented.
2. Two simulation systems for aircraft carrier marshalling
2.1. PC-based aircraft carrier marshalling simulation system
The PC-based aircraft carrier marshalling simulation system is developed for aircraft carrier marshallers. With virtual simulation technology, a virtual three-dimensional (3D) carrier is established as shown in Fig. 1. This system consists of 5 modules and 4 repositories. The 5 modules are terminal display control module, interactive operation module, operation process evaluation module, camera control module, operation process record and replay module. The 4 repositories are 3D module repository, motion repository, mapping repository, evaluation and record repository.
PC-based aircraft carrier marshalling simulation training system interface.
The marshalling data is from Naval Air Training and Operating Procedures Standardization (NATOPS)8 and this system contains 27 motion signals. Some typical motion signals — stop engines, check rudder, external starting air connected, affirmative, APU (Auxiliary Power Unit) disconnected, turn to the left, emergency stop, final preparation and lower wing flaps — are shown in Fig. 2.
Typical motion signals in PC-based aircraft carrier marshalling simulation training system.