DESIGNING A NAVAIDS FAULT DIAGNOSTIC EXPERT SYSTEM USING A FAULTS DISTRIBUTION FUNCTION

DESIGNING A NAVAIDS FAULT DIAGNOSTIC EXPERT SYSTEM USING A FAULTS DISTRIBUTION FUNCTION

1.0. INTRODUCTION

1.1.0.Background  

On 5th May 2007 Kenya Airways Flight KQ 5O7 was involved in an accident shortly after take-off from Douala, Cameroon. All 114 persons on board, including 9 Kenyans perished in the accident. In accordance with Annex 13 of the Convention on International Civil Aviation, Cameroon as the country of occurrence instituted a technical investigation into the cause of the accident. Kenya as the country of Aircraft Registry and Operator was accredited to the investigation (Government of Kenya, Press release, Daily Nation April 28, 2010).

Cameroon released the report of the inquiry into the accident. The report revealed that the accident occurred at night, approximately 4 nautical miles from Douala Airport in a swampy area, 2 minutes after take-off. The report stated that the aircraft had no technical defects and the flight crew were qualified and appropriately licensed (Government of Kenya, Press release, Daily Nation April 28, 2010).

According to the report the probable cause of the accident was “loss of control by the crew as result of spatial disorientation after a long slow roll during which no instrument scanning was done in the absence of external visual references in dark night."  (Kimunya, A. acting Minister for Transport, Daily Nation April 28, 2010). 

Accident statistics based on International Civil Aviation Authority (NTSB accidents and accident rates, 1997-2006) show that 51% of air accidents occur during final approach, landing and take-offs of an aircraft (Kebabjian, 2008). A statistical summary of commercial Jet (2000-2005) indicates that final approach, landing and take-offs are the periods when the flight maximizes usage of air navigational aids (Navaids).

Other statistics indicate that more than 70% of aviation accidents are related to human errors and 56% of worldwide hull lose accidents are caused by flight crew errors (McFadden, 1993; Boeing Commercial Airplane Group, 2005). It has also been claimed that all accidents have some forms of human error attached to their causes (Braithwaite et al., 1998). Estimation of the human error related risk in a given time interval that a particular airline would be expected to have, upon adjusting for the airline’s corresponding safety performance indicators, could help to identify situations in need of heightened level of surveillance by the safety inspectors (Shyur, 2007).

A study conducted by AIRSYS ATM (2002) concerning Navaids maintenance engineering in Kenya revealed that it takes between five and ten years of training and experience for Navaids maintenance engineers to gain expertise skills. According to Waweru (2008) the cost of maintaining and operating an aircraft is very high. The risks in aviation are enormous so are the insurance costs. There is no room for imperfection in maintenance, training or operation of aircraft. Quality training is expensive worldwide. One solution proposed for this problem is to develop a computer-based decision support tool (Navaids maintenance expert system) which can assist Navaids maintenance engineers to develop expertise in the shortest time possible and use the tool to solve future problems in Navaids engineering (Waweru, 2008).

These revelations have not just necessitated the use of expert systems in maintenance of Navaids but design of performance enhanced Navaids expert systems to capture, preserve and recover expertise skills that would otherwise be lost through death, resignation or retirement of expert engineers. It is not possible to keep these individuals in the organization for ever, yet their experience and expertise is required to save the company from huge costs and to ensure systems integrity without compromising the safety of those flying. Sawe (1997, pp2) in a case study of the Kenya civil service reform program stated that there should be a move from supply to demand driven training which ensures that it is not individual demands that are met but organizational demands.

According to Sisak (2000), Lirov (1992) and Gibson (2000) the current concepts of designing fault diagnostic expert systems are such that the rules and cases in the knowledge base depend on the knowledge and experience of the human experts who understand the functioning and design of the machine under diagnosis. In most cases the engineering equipment or machine is purchased together with its fault diagnostic expert system. However, during the life span of the machine, some faults occur due to working environment or wear-out. It is evident that these later faults were not captured in the knowledge base during design and therefore the current fault diagnostic expert systems, as they stand now, can not resolve these faults (Frank, 1990; Zang, 2002).

1.1.1. Overview of Proposed Study

There is need to develop a system and method in which problems realized in earlier methods could be minimized.  The proposed study shall use historical and case study research methods; to investigate faults distribution patterns in Navaids in Kenya and generate a faults distribution function. The distribution function will be applied in the design of Navaids faults diagnostic expert system. An experiment will be designed to evaluate performance of this method of design.

The proposed research study shall use both experimental and non-experimental design.

In the non-experimental design, descriptive statistics shall be used to determine central tendency and dispersion of faults distribution patterns. The central tendency measurements shall be the mean, mode and median. Dispersion shall be measured by determining the range, variance, standard deviation, skewness and kurtosis. Fault weight (W) is defined as a quotient of mean time to repair and least mean time to repair for the sample. A plot of fault frequency against fault weight (W) shall be drawn to create a fault distribution function. T-test statistics shall be used to determine if there is any significant difference between Navaids faults distribution function and a normal distribution function. The purpose of the fault distribution function is to determine the upper and lower cut-off points for faults that require being included in the knowledge base of the expert system. An optimized fault diagnostic expert system shall be designed around a Case-Based-Reasoning (Gibson et al, 1999) method that shall be founded on fault-remedy matching by use of visual programming.

In the experimental design the purpose shall be to compare performance of fault diagnostic expert system (FDES) and manual maintenance system (MMS) and control variability introduced by teams of technicians and fault weights.  To address all the three factors, the experiment shall apply a 2x6x3 complex factorial design. The two systems shall be treated as independent variables forming factor A. There shall be two control variables. The first control variable shall consist of six teams of technicians forming factor B. The second control variable shall consist of three levels of fault weights forming factor C. Analysis of variance (ANOVA) using F-test statistic shall be used to determine the level of interaction between factor A and factor B, factor A and factor C and compare performance between the two treatments, FDES and MMS. The research instruments to be used are performance tests, checklists, observations, fault logbooks and maintenance manuals. The research equipment shall be clocks, calculators, a randomizer and a computer.

1.2 Problem Statement

Shortage of Navaids expert technicians, high cost of their training and experience, requires that a system be developed to capture, preserve and recover expertise skills of Navaids technicians. It is proposed that such a system be based on Navaids faults distribution patterns. There is lack of knowledge about Navaids faults distribution patterns in Kenya. This knowledge is critical in improving performance of Navaids maintenance technicians and in designing performance enhanced Navaids faults diagnostic expert systems. This is to be realized by investigating Navaids faults distribution patterns and evaluating their performance in the design of Navaids faults diagnostic expert system.

1.3 Main Objective

To evaluate performance of faults distribution function in designing faults diagnostic expert system for Navaids in Kenya.

1.4 Specific Objectives

1.   Determine Navaids faults distribution function by plotting fault frequency against fault weight.

2.   Determine if there is any significant difference between Navaids faults distribution function and Normal Distribution Function.

3.   Using Navaids faults distribution function, design an optimized fault diagnostic expert system founded on Case-Based-Reasoning method.

4.   Compare performance of fault diagnostic expert system and maintenance manual system.

1.5 Research Questions

1.   What is the nature of faults distribution patterns in Navaids systems in Kenya?

2.   How is the faults distribution function for Navaids faults in Kenya similar to a normal distribution function?

3.   How can the faults distribution function be used in designing faults diagnostic expert system?

4.   To what extent does use of Navaids faults distribution function improve the fault diagnosis performance of Navaids technicians using faults diagnostic expert system

1.6 Assumptions

1.   To investigate faults distribution patterns, data extracted from Kenya Civil Aviation Authority maintenance log books for a period of 5 years from 2002 to 2006 is considered. The assumption made is that 5 years is long enough for the research data to be large enough for its statistical mean, variance and standard deviation to approach population parameters of the data. Again 5 years is short enough to contain enough data that could be analyzed within the time frame of this research. The 2002 to 2006 period is considered because during this period there were no major equipment replacements, upgrading or industrial strikes, the failure rate was constant and maintenance data is intact (KCAA engineering and maintenance operations manual 1998 Vol III). This assumption may lead to the risk of generating too little data for development of optimized Navaids faults diagnostic system; the knowledge base of such a system depends on its ability to have large numbers of reference cases and relationships (Lirov, 1997). However, a preliminary survey has shown that the data is large enough, repetitive and normally distributed. These characteristics make the data suitable in the creation of fault distribution function and the knowledge base.

2.   KCAA has 11 Navaids stations with 102 technicians (KCAA human resource manual, 2008). It is assumed that these stations have common characteristics and the technicians have uniform competencies. The risks arising from these two assumptions can be minimized by applying cluster sampling technique (Finney, 1960). Selection shall be done by cluster sampling method because the stations are scattered over a large geographical area. An intact group with representation of all members shall be randomly selected.

1.7 Significance

1.   The study is likely to change the way training and development of technicians in maintenance operations is done. For example, on one hand it will require technicians to sharpen their skills in computer operations, on the other hand it will not be mandatory for technicians to have in-depth knowledge of design, construction and operation of engineering equipment, but technicians shall be required to sharpen their skills in fault finding using the computer and historical maintenance data.

2.   As a result of adopting the use of faults distribution function in design of expert systems, maintenance data shall become an asset and a crucial tool in the organizations (Nutten et al, 2008). The knowledge in an expert system can be improved and refined if every bit of fault diagnosis data is inputted. Previously, maintenance data was used for justifying the work done and the cost involved and there after discarded or stored for future reference with no logical method of retrieval and application.

3.   Maintenance expert systems developed from faults distribution patterns can resolve problems that occur later during the life span of the machines due to environmental conditions and wear-out. An example is a case of equipment manufactured and tested in Europe (which is generally a cold environment) and sold to operate in Wajir Kenya (which is fairly a hot environment). For equipment that applies semiconductor devices, operating temperature and humidity are some of the major factors that determine failure rate (Ranjit, 2008). Thus equipment manufactured and tested in Europe may develop unforeseen problems during its initial operation in Kenya. However, these unforeseen problems can be recorded over a period of time to form part of the knowledge base of the Navaids maintenance expert system.

4.   As a result of adopting this expert system design method, occurrence of faults in an engineering system will be viewed positively because it contributes to the intelligence of the expert system. Every time the expert system encounters a new fault-remedy case it automatically updates its knowledge base, thus enhancing its performance (Sisak et al, 2000). The more the number of fault-remedy cases the better the performance of the fault diagnosis expert system.

1.8 Scope and Limitations

The magnitude of this proposed research depends on consideration of sample sizes, data volume and data collection procedures, length of study, size of the experiment and research instruments. With populations of 102 Navaids technicians, 11 KCAA stations and 32 units of Navaids equipment, 30% to 40% population samples may be considered for the experiment. Multi-stage sampling techniques shall be used to minimize bias in the selection of sample members from different populations (Finney, 1960). The optimized fault diagnostic expert system shall be developed specifically for experimental purposes to evaluate performance of faults distribution function in maintenance of Navaids in Kenya. There are some aspects that may negatively affect the study; they include non-uniformity of KCAA stations’ attributes, dissimilar competencies among Navaids technicians, illogically stored data scattered in KCAA stations and inaccessibility of some Navaids fault log books. Some of these sources of extraneous variability shall be addressed by multi-factor complex factorial experiment (Finney, 1960). The experiment is scaled to fit data volume that covers the period from 2002 to 2006. The duration to study and perform the experiment is 27 weeks at a cost of Ksh. 169,000.

1.9 Operational Definitions

Accuracy: Is a measure of performance that determines the ratio of the number of correct diagnoses to the total number of diagnoses. The higher the ratio the better the accuracy.

Artificial Intelligence Systems: Include the people, procedures, hardware, software, data and knowledge needed to develop computer systems and machines that demonstrate characteristics of intelligence (Menya, 2002).

Cluster sampling: Cluster sampling technique is where groups are scattered over a large geographical area. An intact group with representation of all members is randomly selected (Finney, 1960).

Cost: The unit cost of resources put into repair of a specified fault in engineering equipment

Evaluation: Measure by assessing value of a system using experiment, description, comparison or inference to facilitate decision making.

Expert System: Is a branch of artificial intelligence systems that consist of hardware and software that stores knowledge and makes inferences similar to a human expert in a particular field (Engelmore, 1993 and Menya, 2002).

Failure cause: the reason the failure took place; e.g. wear, vibration, contamination, voltage surge etc.

Failure effect: the result of failure for each failure mode; e.g. loss of communication, reduced control, injury to persons or equipment, loss of direction, crash etc.

Failure mode: specific way in which the item fails to carry out its intended mission; e.g. short circuit, open circuit, reduced output, symptoms, alarms etc. 

Fault Diagnosis: The logical and systematic process of identifying a fault in engineering systems.

Fault frequency (N): Is measured by determining how frequent given faults repeat over a given period of time. It is represented by a whole number (N).

Fault Weight: Is a value derived from the quotient of mean time to repair and least mean time to repair for the sample. The higher this value the more difficult the fault.

Faults Diagnostic Expert System: An interactive software tool that helps to perform fault diagnosis by logically and sequentially providing a solution to a distinct repair problem the way a human expert would do in engineering systems (Lirov et al, 1992). 

Faults Distribution Function: Is a graphical plot showing the relationship between fault frequency and fault weight.

Faults Distribution Pattern: Is a representation of types of faults and their frequency of occurrence.

Inference Engine: Is the reasoning part of the expert system that deals with searching through the knowledge base for a solution to a given problem.

Knowledge Base: Is part of the expert system that stores knowledge in form of rules, cases and relationships.

Kurtosis: Is the measure of the degree of peakedness of the distribution.

Maintenance: Activities carried out to prevent failure or correct a failure in engineering equipment.

Mean time to repair or replace (MTTR): Is the average time taken to repair equipment or replace components. Is measured by summing individual time intervals taken to repair equipment or replace components for given faults. This summation is divided by the total number of faults

Multistage sampling: Applying different sampling techniques at different stages of the study.

Navaids Engineering: Air navigational aids (Navaids) engineering is the planning, design and maintenance of avionics equipment that is used for communication, navigation and surveillance in air traffic control and management (KCAA ISO 2009 procedures for ANS engineering operations).

Navaids Technician: Personnel trained and deployed to perform maintenance engineering and repairs on Navaids equipment.

Performance: Time taken to accurately diagnose a fault of specified weight in an engineering system using specified resources within defined safety and environmental factors.

Purposive sampling: Is where the researcher chooses the sample based on the subject characteristics that would be appropriate for the purpose and study (Finney, 1960).

Range: Is a measure of dispersion that gives the difference between the greatest and the least value of the distribution.

Skewness: Characterization of the degree of asymmetry of a distribution around its mean.

Speed: Is a measure of performance that determines how fast a fault diagnostic system completes its fault finding operation for a given fault.

Stratified random sampling: Is a sampling method that achieves the desired representation from various subgroups in the population, by selecting subjects in such a way that the existing subgroups in the population are more or less reproduced in the sample (Finney, 1960).