Ensuring End User Needs are Met through Design Validation – Part 1: Alpha Testing
May 11, 2019The Anzen Safety Scalpel was developed through a complex process built to achieve compliance with the US FDA and international regulations. Design validation is one of the many steps required in that process to ensure that the Anzen Safety Scalpel delivers what end-users really need in a safety scalpel.
Anzen is committed to meeting the needs of surgeons and surgical technicians in the operating room. Thus, we carefully considered the characteristics of the users and operating room environment. We engineered the reusable metal handle of the Anzen Safety Scalpel to match the weight and balance of a traditional scalpel, and combined it with safety features that prevent sharps injury. Through design validation, we put the Anzen Safety Scalpel to the test to ensure it meets surgeons’ needs.
The first stage of design validation for the Anzen Safety Scalpel began with alpha testing to ensure the features of the Anzen Safety Scalpel, such as the metal handle and disposable cartridge, were tailored to the needs of surgeons and surgical technicians.
Alpha Testing
Between July 2015 and December 2017, alpha testing was conducted at forty-six different facilities. Afterwards, end-users completed a questionnaire about their experience with the Anzen Safety Scalpel. User-specific questionnaires were developed to elicit feedback on all aspects of use.
The surgeons’ questionnaire covered topics related to the overall ease of use of the Anzen Safety Scalpel, blade sharpness, use of the device for incisions, and the feel of the scalpel when compared to a traditional scalpel.
Surgical technicians scored the Anzen Safety Scalpel on ease of use, ease of cartridge attachment and detachment, and passing the Anzen Safety Scalpel to the surgeon.
Simulated Use Testing
Alpha testing began with simulated use testing. The requirements set out in MediPurpose’s test protocol was to have at least fifty users and ten different facilities test the Anzen Safety Scalpel, and that each question would score an average of 4.0 or higher. Scores of 4.0 and higher were considered positive responses, 3 was considered neutral, and less than 3 was considered negative.
During the simulated use phase of alpha testing, surgeons used the Anzen Safety Scalpel on a synthetic surgical skin pad to get an idea of how it felt and worked during a surgical procedure. This testing was conducted among thirty-seven surgeons and forty-three surgical technicians at forty-six different facilities.
The simulated use testing met the requirements for number of users and facilities. Surgeons and surgical techs scored the Anzen Safety Scalpel positively overall, with an average score of 4.57. Areas which scored positively were ease of learning, safety, usage, touch and feel of the device, and overall impression. However, the category “sharpness” received an average neutral rating of 3.33.
Actual Use Testing
To validate its use in the operating room, actual use testing was conducted with actual patient procedures. The actual use testing requirements in MediPurpose’s test protocol were to have the scalpel used by at least thirty surgeons and surgical technicians at five facilities across at least thirty different procedures. The goal was to achieve a score of 4 or higher for each question.
Actual use testing was conducted with forty-five surgeons, twenty-five surgical technicians, and over fifty-one procedures, meeting the actual use testing requirements for surgeon use and number of procedures.
While the Anzen Safety Scalpel continued to score positively with surgical technicians, the rating dropped to neutral among surgeons in categories such as ease of learning, overall impression of the device, and touch and feel of the device. when compared to the simulated use test scores.
Based on the results analysis, opportunities for improvement were identified. To address surgeons’ concerns regarding blade sharpness, the research team plan to conduct blind blade studies from different blade suppliers and select new blades based on the feedback from the blind blade studies.
To increase surgeons’ comfort level using the Anzen Safety Scalpel, the evaluation process will be modified to ensure all users familiarize themselves with the device via simulated use testing first, before actual use testing. This would ensure that surgeons are not blindsided by the use of a new device that they had no prior experience with while in the operating room.
Once these improvements are implemented, MediPurpose will launch the next phase of the study, beta testing, to complete the design validation of the Anzen Safety Scalpel.
Click here to read about the blind blade studies conducted and how new, sharper blades were chosen for the Anzen Safety Scalpel.
Click here to read about Part 2 in our Design Validation Series – Beta Testing.