The U.S. Air Force is adopting new technologies to train its members safely and effectively
Since its inception as a branch of the United States Army in 1907, the success of the United States Air Force has depended on the skill of the women and men who are the American Airmen. Training and supporting every member, from mechanics to pilots, has been critical to the Air Force’s success.
A few years ago, recognizing that they had finite training resources and limited capacity to rapidly train and deploy critical personnel, the Air Force recognized a need to broaden its training strategy. Their goal was to create skilled Airmen who excelled at their job and could seamlessly take on additional responsibilities. AFWERX, an organization founded to innovate within the Air Force, launched an initiative to incorporate advanced technology into training. They sought to increase the learning retention and engagement of new students during training—ultimately making their sessions more effective and everyday tasks safer.
Headquartered in Virginia Beach, Virginia, Vectrona is a PTC partner that provides technical expertise to solve complex problems and helps create the technology solutions needed to support and shape strategy. Innovation is at the core of their business, and they have extensive experience with augmented, virtual, and mixed reality technology, modeling and simulation systems development, requirements and architecture development, and more. Their expertise goes deeper than just technology—Joe Gelardi, the CEO of Vectrona, served more than 20 years as an active-duty Navy fighter pilot.
Vectrona saw the opportunity of augmented reality technology in training
Over the last couple of years, Vectrona has won three Small Business Innovation Research (SBIR) contracts, part of a program designed to encourage domestic small businesses to engage in Federal Research and Development. The project focused on augmented and mixed reality technologies and solutions for aircraft maintenance, weapons, and armament systems training. Vectrona began working with the Sheppard Air Force Base in Wichita Falls, Texas and the Kelly Air Force Base in San Antonio, Texas. The initiative centered around training Munitions and Armament Systems Specialists who worked on a wide range of tactical aircraft, including F-15s and F-16s.
“The Air Force was focused on training new Airmen coming right out of basic training, who sometimes have little to no background in maintenance. There’s a big culture change and a steep learning curve to overcome, particularly around learning the language of maintenance and the complex spatial nature of sophisticated equipment,” says Gelardi.
Vectrona envisioned an interactive augmented and mixed reality solution that would provide Air Force trainees with immersive, 3D training and task performance tools. To do this, they would give students the ability to train in interactive environments that simultaneously engaged them in all modes of learning—including visual instructions, aural cues, synthetic voice narratives, multi-media images, and realistic, immersive full-scale 3D digital representations of the aircraft. Training applications also needed to work in network connected environments, as well as offline in hangars or other remote areas
Augmented and mixed reality training can vastly improve the quality of academic classroom training and task rehearsals by:
- Reducing time spent in passive, PowerPoint based classroom training
- Allowing students to train even when physical equipment isn’t available
- Enabling students to do preliminary training rehearsals prior to live training on physical assets
- Preparing students to get the greatest possible value out of the precious hands-on training opportunities they need most
Vectrona knew that supporting students’ training with interactive scenarios would translate into significant benefits. At a high level, students would increase their knowledge of munitions and armament systems, achieving superior performance, efficiency, and motivation.
Furthermore, mixed reality training enhances the quality of instruction and standardization, while also empowering students with self-guided and self-paced learning. And finally, the Air Force would reduce both time to proficiency and training costs, all while improving safety and enabling remote learning.
“With augmented and mixed reality, students can learn faster and get more comfortable with the material. We wanted them to more quickly learn how complex systems work and become familiar with maintenance and inspection tasks on full scale models of the aircraft and its parts. By the time they get to the critically important, hands-on part of training, they’re confident and ready to dive in,” says Gelardi.
Vectrona partnered with PTC and Microsoft to build their augmented reality solution
Vectrona partnered with PTC and Microsoft to build their augmented reality solution into immersive, dynamic augmented reality experiences. “Vuforia Studio allowed us to move faster—we could build and deliver capabilities quickly. We were also able to scale faster than we anticipated, allowing us to more efficiently deliver content to the Air Force,” says Gelardi.
Vectrona wanted the augmented reality training applications they created to be easy to use and intuitive for students, which made Vuforia ideal for their needs. They used Creo Illustrate to build animated sequences for the CAD models in the training experiences which were created in Vuforia Studio. ThingWorx enabled Vectrona to capture student performance data and identify lagging or superior performance. It also allowed Vectrona to manage user data, user groups, credentialing, learning content, and data integrations that capture user performance data.
The strength of PTC’s partner ecosystem is built on complementary relationships and technologies. This is especially true of PTC, Microsoft, and Vectrona partnership—each brings unique expertise that drove the Air Force’s desired outcomes. Microsoft’s robust partnership with PTC meant that Vectrona could leverage a powerful combination of deep industry expertise and advanced technology solutions.
Together, PTC and Microsoft’s augmented reality technology helped Vectrona deliver the Air Force’s training solution. Vuforia Studio enables companies to distribute enterprise-wide augmented reality content on phones, tablets, and Microsoft’s HoloLens 2. Trainees can leverage the HoloLens 2 hand gestures, voice commands, and user interface to actively interact with 3D models in a highly immersive way that feels real.
“We knew we wanted to use the HoloLens 2,” says Gelardi. “We had a really great partnership with Microsoft from the very beginning, and the HoloLens is an incredibly effective tool.”
“It’s normal to see Airmen so engrossed in their training experience that they literally duck under virtual aircraft pylons when there’s nothing there,” adds Michael Gaar, the ACE-XR Solutions Manager at Vectrona.
Delivering immersive, 3D training experiences to students—wherever they are located
Vectrona built an augmented reality solution called the Augmented Cognitive Environment – Extended Reality (ACE-XR), enabled by immersive, 3D training experiences created with Vuforia Studio. The ACE-XR User Interface and User Experience (UI/UX) is designed to not only enhance training and operations, but to accelerate and reduce the cost of Extended Reality content development.
ACE-XR delivers step-by-step virtual product familiarization, component identification, and theory of operations experiences, leveraging animated sequences, exploded views, and advanced tracking capabilities. Students can learn how to perform maintenance and other tasks on full scale digital models of the aircraft and its parts, no matter where they are located. The application also overlays instructions and queuing over physical aircraft or weapons systems to enhance and accelerate live training
To date, Vectrona has built a learning library for the Air Force including more than 55 different ACEXR enabled applications. These include training experiences, system tutorials, task and inspection guides, quizzes, and practical application exercises. In these experiences, trainees can spatially orient themselves to the equipment and have the information presented in a tactile and realistic way.
“The experiences are designed to be engaging and gamified, encouraging interaction between the students and the 3D models of equipment,” says Gelardi. “It’s not just passive learning, it’s activated training that creates intrinsic student motivation to learn.
Students can move around and even inside the parts and equipment, viewing and digitally interacting with more intricate components of the asset. For example, the training experience for the F-16 gun system shows a virtual cutaway of the functioning system—something that would not be possible using a physical asset. Animations and x-ray vision capabilities in the augmented reality training experience allow trainees to look inside, visualize hidden components, and gain a better understanding of how the systems work. Because these trainings aren’t tied to real equipment, training can take place even when physical aircraft and equipment aren’t available, allowing training to continue when aircraft are needed for flight operations.
Furthermore, quizzes and tests allow trainees and instructors to track their progress. The solution leverages the 3D models from the augmented reality experiences and adds questions and selectable portions that challenge and spatially reinforce student understanding in ways that traditional “flat” 2D quizzes cannot. Students respond to questions by selecting the correct parts or answering specific questions. Feedback is provided instantly and ThingWorx captures their scores and puts comparative data together.
“Trainees can compare their current quiz to their previous quizzes or to class averages and get instant feedback on that as well. And the instructors can do that too,” says Gelardi.
The digital nature of augmented reality enables training content and experiences to be accessed and viewed wherever the students are located. This meant that the Air Force continued to offer training to students during the COVID-19 pandemic, which would not have been possible with traditional training methods.
“The results were substantially significant. Even with as little as five to ten hours of training with ACE-XR over a two-month period, study participants achieved an eight percent higher score on the assessments than those using only the legacy training program.”
Dr. Helen Crompton, Associate Professor of Educational Technology, Old Dominion University
AR makes training more effective with better learning retention, engagement, and outcomes
Currently, the solution is being used at Sheppard Air Force Base in Wichita Falls, Texas, and Kelly Air Force Base in San Antonio, Texas. The impact of the solution is clear: trainees have demonstrated superior performance, efficiency, and increased motivation using the repeatable and immersive training scenarios. “The feedback we got from students was that information presented in ACE XR was just easy to understand. As a result, learning the theory of operations was easier. Students also were able to get as much repetition as they needed, because there’s no limitation of equipment or resources,” says Gelardi.
Not simply content with anecdotal results, Vectrona brought in an academic research team to study the quantitative and qualitative impact of the augmented reality solution on training. Dr. Helen Crompton, an international expert in educational technology at Old Dominion University, and Dr. Adam Holland, a research scientist at The University of North Carolina at Chapel Hill, built a rigorous study. They had control groups of students who went through the legacy training programs, and they had an experimental group who used the devices.
“The results were substantially significant. Even with as little as five to ten hours of training with ACE-XR over a two-month period, study participants achieved an eight percent higher score on the assessments than those using only the legacy training program,” says Dr. Crompton.
Dr. Crompton also expects to see substantially higher results and benefits with increased integrated use of ACE-XR. She added that due to testing limitations during COVID-19, these numbers would likely increase further with students getting more experience through the solution. Benefits observed during research included the following:
Improved learning retention and greater knowledge of the munitions and armament systems—enabling trainees to progress more quickly to hands-on training
A key reason ACE-XR improves outcomes is because the system engages learners simultaneously in multiple cognitive domains. The solution creates more flexible, adaptive, and interactive learning environments to lower student cognitive workload, enhance insight development, accelerate learning, improve performance, and foster more durable knowledge retention.
Trainees using ACE-XR knew more about their systems than their counterparts using the existing training program alone and performed a full letter grade higher on standardized assessments. They reported the solution helped them understand exactly what was happening and that their information recall improved. One trainee said, “I just took a test on the gun system. And I think I only missed one question. ACE-XR helped me retain that information a lot better.”
Improved instruction through high quality visual learning
The ACE-XR system is a highly effective way to present complex information in a manner that promotes more efficient learning by participants. Participants cited improvements to the quality of instruction as a key reason for their higher levels of achievement, noting the importance of combining high quality visual information with accompanying verbal explanations.
“When students were interacting with a mixed reality experience of a full-size aircraft, it was a beautiful thing to watch them become fully immersed in the experience, fluidly move around the aircraft, and perform the tasks,” says Gelardi.
Uninterrupted and repeatable task training, regardless of system or aircraft availability
The ACE-XR system enables students to perform more repetitions at no additional cost. Furthermore, students don’t have to wait their turn on physical equipment. Together, these two factors resulted in less time spent by trainees waiting for their turn and increased time on task.
The solution also helps reduce equipment downtime by eliminating the risk of breaking equipment. With hands-on training, if a student breaks something, it needs to be repaired before anyone else can use it. But with ACE-XR, there isn’t a risk of students breaking any of the aircraft systems and equipment they’re being trained to use.
Improved confidence and motivation
It’s a daunting experience for new trainees to face a huge aircraft. Interacting with a digital version first was less intimidating and inspired more confidence. Students who trained with ACE-XR were much more confident and ready to dive into hands-on training without fear of embarrassing mistakes, getting injured, or damaging equipment and disrupting training for their fellow students.
Trainees can also go through the digital training sequences at their own pace, which helps build confidence. The tactile, step-by-step learning makes the information click. Furthermore, using the solution appeared to increase both the trainee’s sense of achievement and their motivation. ACE-XR drove a desire to learn, as well as an increased understanding of the learning.
Improved cross-training
The research demonstrated that the ACE-XR system successfully accelerates cross-training. “The other thing that was fascinating,” says Gelardi, “was the benefit to using these kinds of training tools for cross-training. People who might have experience in one area but are novices in another, are able to more rapidly absorb new information.”
Effective remote learning scenarios
And finally, all these benefits can also be realized remotely. With ACE-XR, students are able to practice wherever they are located—a feature which was critical to continuing training during the COVID-19 pandemic.
Vectrona is excited to scale the initiative and incorporate new capabilities
Going forward, Vectrona hopes to extend the effort with other organizations within the Air Force. One opportunity is around training courses where trainees must travel for three weeks of onsite training. Vectrona plans to build augmented and mixed reality learning content that supports those courses. Training devices will be shipped to the students’ home stations, so they can experience the academic courses remotely before they come to training. Then their performance will be tested against their peers.
With ACE-XR, students are able to practice wherever they are located—a feature which was critical to continuing training during the COVID-19 pandemic.
“The goal is for the Air Force to enable Airmen to
learn remotely at their home station before they
travel to the course itself, shortening the time they
spend away from home. This could have very
broad implications on how they train everywhere,”
says Gelardi.
Using Vuforia Studio and ThingWorx, Vectrona will continue to build out the capabilities of their solution and incorporate new technology. “We’re thrilled to have Vuforia Studio and ThingWorx as key components of the ACE-XR system. As we move forward, we’re building out enhanced features and capabilities in the augmented reality applications to enhance operational task performance support. And we’ll have even more integrated data and analytics capabilities,” says Gelardi. “We know the Air Force will need to make a high return on investment in training programs to meet the critical national security challenges of the future and we’re proud to support them with ACE-XR so they can do just that.”