Last Modified: 03/07/2023
Utah's Early Hearing Detection and Intervention Program Evaluation Model
This approach to conducting a statewide EHDI evaluation included collecting data from various stakeholders using data from questionnaires, analysis of archival information, and onsite evaluations. The goal of the evaluation was to collect information about how well hospitals and the statewide system were doing in achieving each of the following:
- All newborns will be screened for hearing loss before 1 month of age.
- All infants referred from screening will have diagnostic evaluations before 3 months of age.
- All infants identified with hearing loss will receive appropriate medical, audiologic, and educational intervention services before 6 months of age.
- All infants with hearing loss will have a medical home.
- Every state will have a complete EHDI Tracking and Surveillance System to minimize loss to followup.
- All families will receive culturally-competent family support.
- Every state will do regular systematic monitoring and evaluation to improve the effectiveness of the EHDI program.
The following data were collected:
Questionnaire to Hospitals.
- Each hospital in the state was asked to complete a questionnaire about their newborn hearing screening program. Questions were included about the hospital's activities with regard to screening, follow-up, referral for diagnosis, satisfaction with the program, obstacles which have been encountered, areas in which assistance is needed from the State Department of Health, and satisfaction with the previous assistance and support provided by the State Department of Health. Prior to collecting data, drafts of the questionnaire were reviewed by members of the State's Newborn Hearing Screening Advisory Committee for suggested revisions.
Analysis of HI*TRACK Data.
- Each hospital in this state uses the HI*TRACK data management system to submit data to the State Department of Health on a monthly basis about the number of babies born, number of babies screened, results of screening, follow-up activities, and information about referral and diagnosis. The external evaluation team analyzed this data to identify strengths and weaknesses of the program and areas where assistance was needed. The analysis of HI*TRACK data was useful in pinpointing areas where additional technical assistance and training were needed. For example, when inpatient refer rates are too high, it is frequently because too many people are screening babies or one or two screeners out of a group of seven or eight are having difficulty. Although this data is available to hospitals, they often don't appreciate its value for program improvement. Thus, it was helpful to have an external evaluation team review the statewide data to identify areas where assistance is needed.
Questionnaires to Parents.
- A random sample of parents whose babies were screened during the previous year were sent a questionnaire regarding their perceptions about the screening program, what they saw as the strengths and weaknesses, and any suggestions they had for improving the program. The sample of parents was stratified to include those who passed the inpatient screening, did not pass the inpatient screening but passed an outpatient screening, and did not pass either the inpatient or outpatient screening but passed the diagnostic evaluation. Questionnaires were returned directly to NCHAM, and respondents were assured of anonymity so they could be open about any concerns they had, as well as identifying aspects of the program which were working well. The results of these questionnaires were used to determine whether appropriate information and support were being provided to parents.
Physician Questionnaire.
- A random sample of physicians listed in HI*TRACK as the primary care physician for babies who are screened were sent a questionnaire assessing their attitude and knowledge about the universal newborn hearing screening program. In addition, they were asked to identify strengths and weaknesses and suggestions for program improvement. Questionnaires were returned directly to NCHAM, and respondents were assured of anonymity so they could be open about any concerns they had, as well as identifying aspects of the program which had worked well.
Onsite Visits to Hospitals.
- A small sample of hospitals were selected for onsite visits by a team of two external evaluators who were experienced with newborn hearing screening programs. The site visit included interviews with program staff (e.g., screeners, screening program manager, nursery coordinator, Director of Women's Services), observation of screening activities, review of records, and examination of materials for parents and program policies.
Data from all of the above sources were summarized by the evaluation team, analyzed according to the six goals described above, and written in a report. The report was discussed with staff from the Department of Health, the State's Newborn Hearing Screening Advisory Committee, and staff from interested hospitals.
