Page 16 - 2017-Issue1

16 the pilOt’s peRspeCtiVe On CRM AssessMent

LAck oF reguLATory guIdAnce based on CRM performance alone although pilots, resulting in possible negative safety
While CRM has evolved over the past 30 CRM has not matured sufficiently enough for implications. Another undesirable result of
years, regulatory guidance has not kept up. evaluators to effectively evaluate a flight attempting evaluation of CRM would be the
The lack of standard terms, definitions and crewmember’s performance. Industry unwillingness of pilots to be themselves during
application methodology, and lack of experience has shown that it is difficult to train evaluation and training, and they would
regulatory guidance has led to lack of and calibrate instructors/evaluators to instead act the way they perceive the check
standardization across the industry. Two successfully identify markers that would lead airman wants them to act in order to achieve a
distinct CRM application methodologies have to an overall “grade” or “consistent grading.” passing grade. This would result in a
emerged since the inception of CRM: This is in part due to these markers not being misrepresentation of the crews CRM skills and
Awareness training and error management adequately defined and is therefore not most likely result in undiscovered deficiencies
strategies. For many years, the industry observable. in a crew’s performance because the evaluator
provided guidance material that centered on does not get a realistic representation of how
the benefits of flight crewmembers’ awareness unInTended conSequenceS oF the crew conducts CRM during normal line
of CRM, often called “soft skills”. The biggest evALuATIng crm operations, and thus cannot provide
benefits to teaching soft skills were changes in There has been no demonstrated safety case meaningful feedback.
attitudes, perceptions and teamwork. for improving safety by introducing jeopardy Introducing jeopardy assessment after 30
Currently, there is no governing regulatory assessment/checking of CRM. In fact some years of effective CRM training completely
documentation for error-management CRM experts within the aviation industry undermines the fundamental principles of fifth
techniques, although IFALPA strongly believe the unintended consequences of generation CRM. The success of an effective
supports training in this area. As a result, CRM evaluating CRM could actually reduce current fifth generation CRM program that focuses on
courses among airlines vary widely, some safety margins. IFALPA agrees in its published Threat and Error management requires the
teaching awareness training while others teach IFALPA Policy on CRM, which states in formal understanding that errors will occur
threat and error management. IFALPA Annex 6 Part I paragraph 9.4.4: and companies adopt a non-punitive approach
to error. Introducing assessment/checking of
SubjecTIve evALuATIon crITerIA “IFALPA believes that to introduce jeopardy CRM skills would introduce the possibility of
IFALPA stands firmly against any CRM assessment or checking of CRM at this point failure which can be perceived by many pilots
evaluations for flight crewmembers individu- would fundamentally change the facilitator / as punitive.
ally or as a crew in any jeopardy event, instructor and flight crew relationship and Since effective CRM must be embedded
especially utilizing only subjective criteria. potentially block or reverse the many benefits to within the safety culture of the airline, and
Little, if any, qualitative evaluation criteria exist be gained from CRM training, including the needs to be practiced and accepted at all levels
for CRM. There is no universally accepted possibility of having a negative impact on safety. of the organization to positively affect
methodology for identifying unsatisfactory Jeopardy assessment or checking CRM may operational safety, it is difficult to
pilot CRM performance. Regulators have result in crews producing acceptable CRM independently assess/check only one single
allowed operators with different corporate behaviour in the simulator but have little real element (in this case the pilots) of the entire
cultures much flexibility in introducing CRM impact on the safety culture of the airline.” system on company culture skills that involve
training, resulting in a wide spectrum of multiple people across the entire company
quality, quantity and effectiveness of CRM For CRM training to genuinely impact the culture. To evaluate only one aspect of the
training across the industry. safety culture in aviation, CRM must be company CRM system would do little to
Vague terms such as “Captaincy,” wholeheartedly embraced by the pilots without increase the safety of the entire system.
“Airmanship,” “Followership,” and “Synergy” the threat of any punitive action. To this end Further complicating the issue is that
lack any formal or recognized definition within IFALPA supports open feedback and evaluation would be based mostly on
the CRM concept. These worthwhile attributes discussion between facilitator / instructor and subjective evaluation criteria that has already
are presently beyond the ability of any expert flight crew on CRM topics. This feedback proven very difficult to train and calibrate
to evaluate objectively, much less a check should however be non-numerical (e.g., instructors/evaluators.
airman unskilled in the meaning of these “Enhanced – Standard – Detracted) and focus Just because crews can demonstrate
terms. Specifically, evaluation of the effective- on reinforcing good skills and discussing areas effective crew coordination while being
ness of non-technical training skills is very of improvement. IFALPA recognises that a assessed under jeopardy conditions does not
subjective and extremely variable. There is no high level of trust and openess must be present guarantee they will actually practice these
universally accepted definition of the CRM for such discussions to be effective. concepts during normal line operations.
concept or category of CRM terms within the Besides IFALPA, individual pilots are also Industry studies show that line audits, where
air carrier industry. IFALPA is concerned concerned about the sour implication of crews are observed under non-jeopardy
because flight crew CRM evaluators lack “evaluating” CRM skills. Evaluations can lead conditions, provide more useful data
adequate standards and guidance material. to a mistrust of the program, especially if the (Helmreich, Merritt, & Wilhelm, (1999)). Data
CRM evaluation exposes a crewmember’s evaluation of these skills is done in an arbitrary from such audits show that changes in pilot
certificate and career to unsubstantiated and capricious manner. If we evaluated CRM behaviour result from CRM training that
jeopardy when no objective industry today, it would be done in an “opinion-ori- includes LOFT and recurrent training
definitions or standards of CRM skills exist. In ented” fashion which could lead to evaluation (Helmreich & Foushee, 1993), which is
one case, an air carrier has terminated pilots controversies and mistrust of the system by the consistent with participant feedback.

Issue 1 | 2017 InterPilot | The safety and Technical Journal of IFALPA