Let us start with the conclusion:
The Sponsor is ultimately responsible for User Acceptance Testing (UAT) of the final system and should do conduct an independent UAT.
There seems to be some confusion about what UAT is in the context of the configuration of an eCOA solution for a clinical trial. UAT is the process by which the Sponsor study team decides whether the final system provided by the technology partner (applicable not only to eCOA, but also EDC (Electronic Data Capture), eConsent, RTSM (Randomization and Trial Supply Management) etc.) meets their expectations and performs in a robust manner consistent with the requirements specification. A reason for the confusion is that some vendors will conduct their own UAT as part of their validation process. However, it is important that the Sponsor conducts their own and independent UAT.
C-Path´s eCOA consortium published in 2022 a detailed paper just on UAT. This paper was developed by some of the leading eCOA vendors to set a standard for the industry. In this detailed report the experts state that the planning (UAT Plan and scripts) is a Sponsor responsibility and “it is best practice that the sponsor or designee write the test scripts and not ask the eCOA provider to provision test scripts”.
Where there are inconsistencies or misunderstandings in system requirements, they will normally be detected during UAT. These can be avoided by the study team investing in a thorough review of the requirements specification prior to its approval, and by the vendor making aspects of the system available for inspection during the configuration process. For example, some vendors conduct interactive reviews and provide screenshots or simulations of the final solution during this process, before the Sponsor signs off on the final requirements. However, some requirements specifications are technical, have a complex structure and can be lengthy and difficult to understand, especially for unfamiliar study teams. In these cases, it may be beneficial to involve experts in UAT for support.
Ideally, UAT should not begin until the eCOA vendor has provided confirmation that their system validation work has completed and been accepted internally. eCOA systems often consist of a data acquisition module (the data entry system), an administration module (e.g., patient enrolment), a reporting module, and potentially will include integrations with other systems, such as randomization systems (RTSM), and EDCs (Electronic Data Capture) (Electronic Data Capture). Sometimes, to meet timeline commitments, vendors complete the configuration of these modules at separate times, and therefore the UAT may have to start before all systems are fully configured and tested by the eCOA vendor. In this instance, it is important that this will be communicated early in the planning stages and that UAT will be planned accordingly.
As with the system validation, UAT should be conducted using pre-defined test scenarios (test plan) and test cases, and the results of the UAT need to be documented and provided to the eCOA vendor. It is important to note that UAT by the Sponsor does not replace the testing and validation done by the eCOA vendor, and that sponsors should not perform a complete revalidation when conducting UAT – this is still the responsibility of the eCOA vendor. Rather, sponsors should perform a risk-based approach to ensure the system meets the key requirements.
Although UAT shall not be conducted by the vendor, vendor staff will have to support the activity. This may include providing access to backend databases to enable the UAT team to inspect data entries or resetting or clearing data from the system to enable new tests to be conducted. Some implementations also require vendors to support “time travel” to test date/time triggered events. Some vendors achieve these using solutions that enable changing the system time on the UAT server, other vendors use patients pre-loaded on the system to simulate time travel.
UAT should also include the associated patient management interfaces, reporting platform (often in form of a web-portal), data integrations, and the data export functionality.
Planning for UAT should be a part of the study implementation plan and should include identification of the group from the study team and/or external independent specialist organizations, who will develop the UAT test plan, test cases, and conduct (part) the testing. Following UAT, the testing team should hold a debrief meeting with the vendor to decide any follow up actions needed based on the outcome of the tests. In some cases, more system configuration or development work will be needed, and this may require revision to the requirements specification, further system validation, and a final user acceptance step.
It is important to allow sufficient time in the project plan for the UAT activities. Often there is a need for more than one round of UAT, and the vendor will require time to reconfigure the system between UAT rounds. While it is not the preferred method, various UAT steps can be conducted in parallel with vendor validation activities if these are clearly defined and communicated.
A partner with extensive experience and a library of script templates for various vendors will be able to efficiently support UAT and will minimize the burden on the clinical trial team.
References:
