FDA

Medical Device Software Validation and Verification for Regulatory Compliance

Delve into the complexities of validation and verification for cloud-based SaMD, including regulatory considerations, key validation strategies, and best practices for compliance.

Lee Chickering

•

September 30, 2024

Compliance

Validation

Medical Device Software Validation and Verification for Regulatory Compliance

In the realm of healthcare technology, Software as a Medical Device (SaMD) has emerged as a pivotal innovation, offering new avenues for diagnosis, treatment, and patient management. As SaMD increasingly moves to cloud platforms to leverage scalability, flexibility, and computational power, the process of validation and verification is paramount. This process ensures that SaMDs meet stringent regulatory standards, ensuring safety, reliability, and efficacy. Let’s delve into the complexities of validation and verification for cloud-based SaMD, covering regulatory considerations, key validation strategies, and best practices for compliance.

Defining Software as a Medical Device (SaMD)

First, it's essential to define what qualifies as SaMD. The International Medical Device Regulators Forum (IMDRF) defines Software as a Medical Device as “software intended to be used for one or more medical purposes that perform these purposes without being part of a hardware medical device.”

This category includes, but is not limited to, software that collects patient data to provide medical advice, analyze conditions, and use AI to detect abnormalities in medical images. The FDA requires that medical device manufacturers define the intended use of the SaMD, detailing its functions, specifications, and the benefits to the patient.

Understanding Medical Software Validation and Verification

What is Medical Software Validation?

Medical software validation confirms that a software product meets its intended use. This involves checking that the software fulfills user needs, mitigates risks, and complies with regulatory requirements. To validate SaMD, a comprehensive plan must be developed to test product requirements and specifications, ensuring safety and effectiveness. The process follows the V-model, a widely recognized framework seen in IEC 62304, the international standard for the lifecycle requirements for medical device software. The V-model emphasizes clear requirements and traceability between requirements and the implemented code. Validation and verification tests are conducted to confirm the software meets both technical specifications and its intended use.

Software V-model

Steps in Software Validation:

Create a Validation Plan: Define the entire software system, environment, assumptions, limitations, test criteria, and team responsibilities.
Develop Software Requirements Specification (SRS): Identify infrastructure and functional needs, including performance and security.
Develop a Validation Protocol: Outline expectations and testing methods through a test plan and test cases.
Conduct and Document Tests: Execute tests and record results.
Finalize Procedures and Report: Establish operational procedures and compile a final validation report.

What is Medical Software Verification?

Verification ensures that the software is built correctly and is free of defects. It involves activities such as code inspections, unit testing, and integration testing to confirm that the software functions as technically intended.

Verification Activities:

Code Inspections: Reviewing the code to identify and rectify defects early in the development cycle.
Unit Testing: Testing individual components of the software to ensure each part functions correctly.
Integration Testing: Ensuring that different components of the software work together seamlessly.

Medical Software Validation vs. Verification

Validation and verification, though often confused, serve distinct purposes. Validation focuses on ensuring the software fulfills its intended use and meets customer needs. On the other hand, verification involves testing at the technical level, ensuring the software meets internal requirements derived from customer needs.

In basic terms, validation ensures that the product built meets the intended purpose, while verification ensures that the product was constructed correctly according to the specifications.

Regulatory Requirements

In the US, the FDA mandates that all medical devices undergo software validation, emphasizing documented evidence of meeting specified standards. Similarly, the EU requires compliance with regulations like the EU MDR and standards such as IEC 62304 for software life cycle processes.

FDA Requirements

The FDA provides guidelines based on the software's risk class (Class I, II, or III). Higher-risk software (i.e., Class II or III devices) requires more rigorous testing and documentation. While the FDA does not suggest specific tests, manufacturers must ensure thorough testing to comply with standards and avoid potential issues.

Documentation: Comprehensive documentation of the validation process, including test results and protocols.
Risk Management: Identification and mitigation of risks associated with software use.
Traceability: Ensuring that all requirements are traceable throughout the development and validation process.

EU Requirements

EU MDR Compliance: Adherence to the European Union Medical Device Regulation for software as a medical device.
IEC 62304: Compliance with the international standard for the development and maintenance of medical device software.

Challenges in Medical Software Validation and Verification

Complexity

Medical software often involves complex algorithms and integrations, making validation and verification challenging. This complexity has increased significantly with the transition from traditional embedded software to advanced cloud-based solutions. Traditionally, medical software was embedded within devices; however, the rise of cloud software has introduced layers of complexity. Modern cloud software isn't just a single function running on a device— it's a network of interconnected servers, databases, and third-party services. This intricate architecture demands meticulous validation and management.

Regulatory Changes

Keeping up with evolving regulations requires continuous updates to validation and verification processes.

Resource Intensive

The process is resource-intensive, requiring significant time and expertise to ensure thorough validation and verification.

Dynamic Environment

The dynamic nature of software means it is constantly evolving, unlike hardware, which typically has a longer and more stable lifecycle. In the hardware domain, changes are rare and expensive, making thorough validation crucial before deployment. In contrast, software requires regular updates and maintenance to stay relevant and secure, adding another layer of complexity to its validation.

Customer Expectations

Today's customers expect timely updates and rapid responses to vulnerabilities in their software. This expectation stands in stark contrast to the static nature of hardware products, such as pacemakers, which are designed to remain unchanged over time. The ability to quickly deploy updates is essential to meet these modern demands.

Tracking Changes

A critical aspect of this approach is understanding and tracking changes between software releases. A robust system should enable developers to identify what tests need to be rerun, especially when addressing critical bugs that require immediate fixes. This is particularly challenging when different versions or variants of the software exist, each tailored to specific markets or regions.

Moving Beyond Document-Based Systems

Traditional document-based approaches often fall short in dynamic and fast-paced medical software development environments. Instead, an item-based or object-oriented approach, where individual software components are tracked and validated, proves more efficient. This method allows for precise tracking and validation of changes, ensuring compliance without the cumbersome process of managing extensive documentation.

How to Manage the Complexity of Medical Device Software Validation

As software evolves, its complexity increases, making validation an ongoing challenge. Effective strategies include maintaining detailed requirement architectures and design documents, ensuring that all changes are thoroughly tested. Automated tests play a crucial role but must be balanced with practical considerations to avoid overly lengthy testing cycles.

In the software domain, traditional waterfall development models, which involve lengthy phases of requirement gathering, building, and testing, are no longer viable. Agile development methodologies, which promote iterative and incremental development, are essential for keeping pace with the rapid evolution of software.

The Importance of a Risk-Based Approach in SaMD Validation

In the realm of cloud-based medical software, adopting a risk-based approach is crucial for ensuring patient safety and regulatory compliance. This method involves identifying and assessing potential risks in the software architecture, followed by implementing appropriate risk controls.

When a part of the software is identified as high-risk, a thorough risk assessment is conducted. This process involves determining hazardous situations and implementing risk controls to mitigate unacceptable risks. These controls could range from specific software requirements to functions that prevent certain failure conditions. Regular testing ensures these controls remain effective with each new software release.

With each software update or release, it is vital to retest all risk controls to ensure they haven't been compromised. This continuous validation process is a cornerstone of maintaining software safety and effectiveness. The FDA emphasizes this risk-based approach, which integrates risk considerations throughout the software development lifecycle.

Common Pitfalls for SaMD Validation

Some of the biggest challenges in the industry are the uncertainty and delays associated with ensuring all requirements and validations are in place before a release. This uncertainty can slow down the development process, impacting the timely release of potentially life-saving medical software.

Companies often resort to manual processes to track changes and validations, which can be time-consuming and error-prone.

Understanding Cloud Validation in the SaMD Ecosystem

Cloud validation for SaMD refers to the process of ensuring that both the software and its cloud-based infrastructure are designed, deployed, and maintained according to regulatory requirements and industry standards. This involves rigorous testing and documentation to verify that the SaMD performs as intended in the cloud environment, maintaining data integrity, security, and privacy.

Validating cloud-based medical software involves navigating numerous dependencies and ensuring every component functions correctly. Unlike the more static embedded systems, cloud software is dynamic, with frequent updates and changes driven by evolving dependencies and security vulnerabilities. This fluid nature necessitates a robust and continuous validation process.

Key Components of Cloud Validation for Medical Software

Cloud validation for SaMD encompasses several critical components, each addressing different aspects of software and infrastructure integrity:

Infrastructure Qualification

Manufacturers must ensure the cloud infrastructure (e.g., servers, storage, network) meets all requirements for reliability, security, and privacy. This involves validating physical and virtual security measures, data encryption, and backup systems.

Software Validation

Manufacturers must confirm that the SaMD functions correctly and safely within the cloud environment, meeting all specified requirements. This includes extensive testing of software features, performance under different conditions, and integration with other systems.

Data Integrity and Security

Verifying that data within the SaMD ecosystem is accurately and securely managed, with robust encryption, access controls, and audit trails is necessary. Compliance with standards such as HIPAA in the U.S. and GDPR in Europe is critical.

Operational Qualification

Manufacturers must demonstrate that the SaMD operates effectively within its intended environment, including the ability to withstand various loads and perform reliably over time.

Performance Qualification

Validation that the SaMD meets all user needs and regulatory requirements in real-world scenarios, focusing on user experience, data analysis accuracy, and overall system performance, is crucial.

Best Practices for Cloud Validation

Achieving compliance and ensuring the effectiveness of cloud-based SaMD requires adherence to best practices throughout the validation process:

Comprehensive Documentation: Maintain detailed records of all validation activities and results, providing a clear audit trail for regulatory review.
Risk Management: Implement a systematic approach to risk assessment and mitigation, addressing potential security vulnerabilities and performance issues in line with ISO 14971, the standard for application of risk management to medical devices.
Continuous Monitoring and Improvement: Regularly review and update cloud-based SaMD systems to address emerging threats, technological advancements, and regulatory changes.
Collaboration with Cloud Service Providers: Work closely with providers to ensure that their services and support align with regulatory requirements and validation objectives.

The Role of Ketryx in Medical Software Validation and Verification

Ketryx addresses validation and verification for SaMD challenges by allowing teams to continue using their preferred tools, such as GitHub, Jira, and AWS, while maintaining compliance in the background. Ketryx captures necessary data for regulatory bodies like the FDA, ensuring that every change and its validation are meticulously tracked. This approach combines the flexibility of modern development tools with the assurance of compliance.

Ensuring Robust Risk Controls

Ketryx excels in helping companies implement and maintain risk controls throughout the software development lifecycle. Every time a new release or change is made, Ketryx ensures that all associated risk controls are tested and intact. This continuous validation process is crucial for compliance with FDA regulations, which emphasize a risk-based approach to software validation.

Seamless Integration with Existing Tools

One of the key strengths of Ketryx is its ability to integrate seamlessly with tools that developers already use, such as GitHub, AWS, and Jira. This integration allows developers to continue their workflow without disruption while Ketryx works to maintain compliance and track necessary data. This ensures that all code changes, approvals, and associated documentation are captured accurately.

Comprehensive Traceability

Organizations are able to see links between items at a glance through an easy-to-digest traceability matrix. Ketryx surfaces any traceability gaps, ensuring that all requirements are traced to the appropriate specifications, risks, and tests.

Audit Trails

Ketryx provides detailed tracking of every change made in the system, including who made the change, who approved it, and the nature of the change. In this way, Ketryx automates the generation of Part 11-compliant audit trails. Ketryx allows users to go back and view the state of any item at any point in time, guaranteeing that all changes are documented and compliant with regulatory requirements.

Efficient Handling of Post-Market Complaints

Managing post-market complaints and anomalies is a critical aspect of medical software compliance. Ketryx simplifies this process by linking complaints to specific items in the software architecture. This traceability helps identify the root cause of issues and certifies that fixes are properly tracked and documented. It also ensures that all relevant information, such as the version affected and the resolution timeline, is easily accessible.

Moving Beyond Document-Based Systems

Traditional document-based approaches can be cumbersome and inefficient, particularly in dynamic and fast-paced development environments. Ketryx adopts an item-based approach, where each piece of information is treated as an individual item that can be tracked and managed more effectively. This approach allows for more precise validation and easier management of changes over time.

Enhanced Validation and Compliance

Ketryx supports the validation of external services in cloud environments by integrating continuous integration and continuous deployment (CI/CD) jobs, such as automated testing. These jobs run regular checks on external services, ensuring they perform as expected. This is critical for maintaining the reliability and performance of software in cloud environments, where external dependencies play a significant role.

SaMD Validation and Verification Interview

For the benefit of our readers, we have included and abridged transcript of the interview conducted on the topic of SaMD validation and verification with Ketryx’s subject matter experts: Jan Pöschko and Patrick Ecker.

Validating software as a medical device, or sometimes software in a medical device, is particularly challenging with newer cloud software. Traditionally, our focus as an industry was on embedded software, which, despite its complexities, was somewhat simpler to manage. While embedded software required great skill to create, it was simpler. However, the shift to cloud software has greatly increased these complexities. Over the last few years, the intricacies of cloud software have multiplied manyfold. Managing this growing complexity while still producing validated software is the challenge we are tackling.

There are many different aspects to consider. Underneath your software, there are numerous layers of infrastructure and a ton of dependencies, typical of a web project running in the cloud. Such projects may have thousands of dependencies, both direct and indirect. The architecture of the software is more complex than a simple function running on a device; it involves various servers interacting with each other, client interfaces, databases, potentially third-party services, or even a microservice architecture you might choose to implement. All of these elements add to the complexity.

Managing this complexity, thoroughly testing the software, and knowing precisely what you've shipped pose significant challenges. In the traditional medical device sector, developers often dealt with hardware systems and perhaps firmware. This scenario involved integrating a single component and ensuring it passed through a validation layer. However, as the industry has evolved, many systems now operate purely on software, which progresses much faster than hardware. In the case of hardware, the production and distribution could span nearly the entire product lifecycle. Software, by contrast, requires ongoing maintenance distinct from hardware, often due to major dependencies such as the operating system.

Much of the software built today isn't validated per se. Therefore, it's crucial to ensure the validation of the most critical parts that could fail or pose risks. Even if a software product is released with no immediate plans for updates, inevitable changes will occur—whether due to shifts in the underlying architecture or the need to address critical vulnerabilities.

In the hardware domain, the challenges related to software are fewer because the products are less dynamic. In software-only products, there is a need to respond promptly to vulnerabilities as they are discovered. Customers expect rapid updates, setting a different standard compared to hardware. For instance, no one expects a pacemaker to undergo frequent changes, whereas an app on a phone is expected to receive updates to address changes in technology standards, like Bluetooth, or to keep pace with modern technology. Similarly, browsers continuously update, necessitating adaptations.

Developers in this space must adopt a different mindset due to these customer expectations. The goal is to deliver updates more quickly, build incrementally in a more agile manner, and create tools to support these processes. The traditional waterfall model, which could extend over several years from writing requirements to testing, is ineffective and undesirable in this dynamic environment.

In the hardware space, the extensive manufacturing process requires that everything be double, triple, or even quadruple checked to ensure it's defined as planned. Once a product goes to the factory, any change becomes prohibitively expensive. Conversely, in the software realm, updates can be deployed with just a button click, and changes can be introduced quite easily. The distribution channels are also completely different, highlighting the distinct challenges and differences compared to traditional embedded device manufacturing.

The question then arises: What do you need to do to validate cloud software, particularly when it concerns Software as a Medical Device (SaMD)? First, it's crucial to define what constitutes SaMD. For example, software that collects patient data to provide health advice, analyze conditions, or an AI system analyzing X-ray data to identify potential anomalies for further medical review could qualify as SaMD. This classification generally includes software that poses a risk of harming a patient, causing injury, or leading to incorrect diagnoses.

When defining a SaMD, the FDA requires you to specify its intended use—essentially, what you would state on the package if you were selling the software to an end-user. Based on this intended use, you would develop a set of use cases and product requirements detailing what the product does, how it is supposed to be used, and the benefits it offers to clients. The FDA's primary interest is in ensuring that these product requirements are implemented effectively and safely, and that all associated risks have been thoroughly assessed. These elements must be tested and validated to meet regulatory standards.

The validation process is designed around a specific plan to test the product requirements and specifications, ensuring that the implementation is thoroughly tested in a manner that guarantees safety and effectiveness, while also fulfilling the intended use. A crucial concept here is the V model, which is emphasized by the FDA and particularly the standard IEC 62304. This model involves starting with clear requirements, which can include multiple layers—from marketing requirements down to system requirements. These are then distilled into detailed specifications and the actual code that implements these specifications.

As you move up the V-model, the focus shifts to verification and validation tests for all these specifications and requirements. It is essential to ensure complete traceability between these elements, confirming that there are tests for everything. This comprehensive approach to testing and traceability is what ultimately defines a validated product.

In terms of how tests are implemented, the FDA does not provide concrete guidelines; it simply mandates that the product must be tested. In the software domain, this typically involves a combination of automated tests and, in some cases, additional manual tests conducted by humans. The terms "verification test" and "validation test" are frequently used, though their interpretation can vary among companies with different Quality Management Systems (QMS).

Broadly speaking, I define a verification test as one that occurs at the lower layers of the V-model, where we start by implementing some code and then verify it through unit tests, integration tests, or perhaps system tests that are more technical in nature. Validation tests, on the other hand, focus more on validating the requirements or the intended use of the product. These are high-level tests that concentrate on the product's functionality and how it meets user needs. Verification, therefore, concerns the technical aspects detailed in the specifications, which may not be directly relevant to the user but are crucial for engineers or R&D teams.

It's interesting to note that not everyone uses the terms "validation" and "verification" in the same way, but fundamentally, there is a clear distinction. Validation is focused on demonstrating that the system meets the needs of the customer and effectively fulfills those needs. Verification, on the other hand, is more internally focused and aims to show that you meet the requirements you have derived from customer needs. Thus, while the FDA doesn't provide very detailed guidelines, these principles guide the testing and quality assurance processes.

Is it up to the manufacturers and the companies to decide what tests they need to perform in order to be compliant with the verification?

Yes, it is largely up to manufacturers and companies to decide which tests are necessary to comply with verification requirements. The guidance provided through standard IEC 62304 is that the extent of required testing depends primarily on the risk class of the device. For instance, if the device is classified as Class I—considered the lowest risk—only the most basic level of testing is required. In the documentation submitted to the FDA, they primarily expect to see system-level testing at the highest level. However, this isn't an invitation to cut corners. While the FDA doesn't require companies to show all details of testing, it is still prudent to rigorously follow thorough testing processes. Formally, the required level of documentation and testing depends on whether your device is classified as Class I, II, or III. The FDA has also recently updated its regulations to distinguish between basic and enhanced documentation levels in their guidance.

Discussing the risk class of a software item is indeed a complex and intriguing topic. Software can quickly evolve from a small product into a large, multifaceted one with numerous moving parts that require validation. This growth presents a specific challenge: maintaining oversight of the entire requirements architecture, the Software Requirements Specification (SRS), and the design architecture that represents your software.

When introducing changes to your software, it's crucial to ensure that the most critical parts are validated. If you alter one component, you must have proof and evidence that this specific part has been retested. This leads to the challenge of how to trace a change throughout your system, including your requirements architecture and software items, and ensuring that all relevant tests have been executed.

While you could adopt a rigorous approach and rerun all tests for every iteration, this becomes impractical as the software's complexity increases. For example, executing one iteration of your upcoming version might take many months, or running a suite of automated tests might consume an unreasonable amount of time. Therefore, it becomes necessary to strategically divide and manage the testing process.

We are committed to a risk-based approach, where we identify particularly risky parts of our architecture and conduct a thorough risk analysis on them. This involves creating a risk assessment to determine how risky it is and identifying potential hazardous situations that could arise. Any risks deemed unacceptable are then mitigated through risk control measures, which could involve implementing specific requirements or software functions to ensure certain conditions or failures do not occur. Additionally, tests are designed for these controls.

At Ketryx, we ensure that every new release or change to our software includes verification that all risk controls are tested, in place, and have not been compromised by software changes. It's crucial to maintain a comprehensive understanding of the risks within your system and to sufficiently address them. Moreover, testing these risks—or the controls put in place for them—is a critical step with each release to prevent inadvertently compromising any risk controls. This rigorous attention to risk management is also a key focus of the FDA, emphasizing the importance of considering risks throughout the software development lifecycle.

It's crucial to have a solid understanding of what changes between releases and to establish a baseline so that when modifications are made, you can efficiently rerun necessary tests. This requires a robust system that allows you to track changes and determine which tests need to be repeated. For instance, you might have a specific version of a product that requires a unique variant, perhaps tailored for a different market or region. If this version encounters a critical bug that needs immediate resolution, you cannot afford to delay the fix until the next scheduled release, which could be six months away. Instead, a hotfix must be created based on the version already in use.

This introduces the challenge of understanding the requirements that were in place for that particular version at that specific time, how it was validated, and ensuring that all risks were adequately managed. This process delves into the post-market aspect of product management, highlighting the importance of thorough documentation and risk management throughout the product's lifecycle to swiftly and effectively address such urgent issues.

The document-based approach often breaks down, particularly in environments where changes are dynamic and granular. While this approach might suffice in a slow, traditional waterfall model—where extensive requirement documents are followed by comprehensive testing documents—it is less effective for managing frequent, detailed changes. Instead, we advocate for an item-level approach, which you might also consider object-oriented. This method involves working with different objects within your system and meticulously tracing each through the validation process. This is a significant departure from old-school, document-based systems.

Furthermore, the concept of maintaining accurate records and snapshots, effectively locking in configurations at the time of a release, is not common in non-regulated tools because they aren't required to operate this way. However, in the realm of medical devices, it's crucial to adhere to these stringent practices to ensure compliance and maintain the integrity of the product throughout its lifecycle.

I believe Ketryx's key value proposition lies in enabling people to continue using the tools they are accustomed to, like Jira, GitHub, AWS, and various others, in their day-to-day operations. What sets Ketryx apart is that while users work within these familiar environments, Ketryx seamlessly maintains compliance and manages the necessary data in the background. The real magic of Ketryx is that it allows users to operate in their usual tools while it provides the necessary guardrails and captures the critical evidence needed for regulatory compliance, such as with the FDA. This integration ensures that workflow is not disrupted, yet compliance is assured.

So if companies didn't have Ketryx, what would their process be if they needed to go back and make changes? What are the challenges would they face without having the Ketryx software?

Without Ketryx, companies would face significant challenges if they needed to go back and make changes to their software. Typically, a developer can easily access a historical commit in the codebase. However, the real challenge arises in connecting that code to the specific requirements, tests, and necessary re-evaluations relevant at that time. This often results in a substantial amount of manual work, as there is no reliable system to maintain this information effectively.

Many tools still rely heavily on a document-based approach, where modifications are made in numerous documents or spreadsheets. As projects become more complex, even robust tools like Excel can become cumbersome, struggling with excessive data or requiring complicated management of separate data chunks. These tools are also not easily searchable, making it difficult to understand how a system was configured for a particular released version.

Ketryx addresses these issues by tracking every change within the system comprehensively. This capability is something that other products often lack. With Ketryx, you can trace the history of any item, determine its state at any given time, identify who made changes, who authored them, and who approved them with an electronic signature. This historical tracking is akin to what software developers expect in their unregulated coding environments, like using Git for source code.

Moreover, Ketryx facilitates the management of hotfix releases based on previously released versions, allowing changes to be made in a manner that mirrors a software developer's approach to problem-solving. This similarity extends to how Ketryx handles items from systems like Jira, ensuring a seamless integration and providing greater flexibility in how developers work. By adopting an item-based approach, Ketryx enables the inspection of every item and its relationships within a vast requirement and item graph, showcasing the interconnections at any selected point in time. This method aligns well with the ways software developers are accustomed to working, making regulatory compliance not only manageable but also more intuitive.

Common mistakes during the Verification and Validation (V&V) process often stem from industry-wide uncertainties about the extent of regulatory requirements. Many managers express concern over how long it takes to prepare a product for release with all requirements in place and approved. This has led to a slowdown in the industry; releases take a long time due to excessive caution, such as double-checking that all documentation is correct and ensuring that all trace connections are properly established. A typical oversight might be missing a validation test for a specific component or lacking a clear trace from a software item to a product requirement that supports a particular intended use. Questions like "Why are there product requirements that do not fulfill any intended use?" highlight gaps in understanding or process integration.

The risk for companies is significant—if they cannot adapt to changes quickly, competitors who can will outpace them. Speed is critical, not only for market competitiveness but also because quicker releases can lead to faster patient benefits, potentially saving lives or preventing injuries.

Additionally, the operational costs and logistical challenges of long development cycles are considerable. Companies may find themselves without a clear sense of progress, from R&D through quality management to final document approval. The process of reviewing and signing off on massive document sets can be daunting; some companies resort to printing these documents for manual signatures, which is impractical and inefficient for documents that can span tens of thousands of pages.

This challenge has led some companies to attempt building their own systems to manage these processes. However, these tools themselves need to be validated, which can paradoxically introduce new delays and complexities. The need for a reliable system to streamline and support the V&V process, ensuring compliance and efficiency, is therefore crucial.

Effective processes need to be in place to ensure that any code change is rigorously vetted before it becomes part of the main product line. For instance, peer reviews are essential, where a code change must be reviewed and approved by another developer. Additionally, integrating static code analysis into the continuous integration and continuous deployment (CI/CD) pipeline is a common practice. This involves using an integration server within the organization to run various checks on code changes, including code quality assertions like static code analysis, type error checks, or even assessing variable naming and function length to maintain readability and ease of review.

Such information is crucial and must be meticulously tracked as evidence. If a team claims to conduct code reviews, then all related documentation, including reviews, must be systematically collected and stored. Ketryx facilitates this by integrating directly with tools like GitHub. When you add your GitHub repository to a Ketryx project, Ketryx manages the tracking of all code changes, discussions around them, approvals, and more. This data is then compiled into a separate release document before proceeding with the release.

Moreover, Ketryx enables the implementation of engineering controls that can restrict a release or merge if certain criteria are not met, such as the approval of specific items in the system. This level of oversight ensures that all changes meet stringent quality and compliance standards before they are implemented.

In the post-market phase, when software is released and provisioned to specific servers or environments—sometimes uniquely configured for particular customers—managing different versions becomes crucial. When a complaint or anomaly is reported, it's imperative to determine the root cause and provide evidence that the complaint was taken seriously, as required by the FDA. Failing to track these complaints adequately can lead to significant compliance issues.

One of the major challenges is connecting a complaint to the specific elements of your software and requirement architecture. Identifying the cause of anomalous behavior and its root cause is often complicated by traditional document-based approaches, which lack the granularity needed to directly link complaints to specific software items. Typically, you might reference a general document without precise details.

With Ketryx, this process is streamlined. For instance, if an anomaly is reported, Ketryx allows you to trace it back to a specific software item via a traceability widget. You can see how the complaint or anomaly is linked to a particular function in the code, which might have been malformed, and track how it was rectified through subsequent code changes. These changes are documented comprehensively, including pull requests on GitHub, reviews, and other relevant information.

Ketryx also enables you to denote which software version the anomaly relates to and when it was resolved. This capability significantly simplifies compiling a complete history of the issue, including details like whether the anomaly was immediately fixed or marked as a residual anomaly if it was deemed less critical—a practice also encouraged by the FDA.

Overall, Ketryx provides an effective way to navigate through the software architecture and maintain meticulous records, ensuring that every step from anomaly detection to resolution is accurately documented and easy to access. This method addresses one of the biggest challenges in post-market software management: maintaining clear, actionable connections between complaints, their causes, and their resolutions.

In a document-based approach, the format for managing information is essentially about how data is organized and stored within the entire system. Conversely, Ketryx adopts a risk-based approach where everything is considered an item. In this context, each risk is treated as an individual item. While Ketryx still generates documents, these are created based on an item-based approach, with each item acting as a fundamental building block for the final documents.

Rather than making an either-or choice between document-based and item-based systems, Ketryx allows for a hybrid approach. Items can be rendered as documents whenever necessary. This flexibility also extends to creating documents at different milestones within the release process, enabling the capture of snapshots of specific item configurations. This method not only maintains detailed records but also enhances the ability to track changes and assess risks effectively throughout the development and release cycle.

In the cloud domain, a critical aspect of the validation process is ensuring that external services function as expected. Typically, this involves deploying the software in a production-like environment, often referred to as a staging environment. This setup mirrors the server configuration specified to the FDA, detailing necessary resources like server capacity and power. For instance, you might validate your software on an AWS EC2 medium-sized instance with specific RAM specifications.

During the validation phase, you execute validation tests based on a defined test plan. If using a tool like Jira, it's crucial to validate key aspects such as the availability of the service in the environment over a specific interval and the correctness of responses from REST endpoints.

This process is vital in a cloud environment where reliance on external services is high. It's essential to monitor and verify that these services perform as expected. Evidence of this testing must be meticulously documented. Additionally, in relation to CI/CD practices, you might run CI jobs at regular intervals, generating reports that are integrated into our system according to your configuration settings.

In the broader context of the validation phase, it's paramount to ensure all cloud services used are performing correctly, as failures can introduce significant risks depending on the product. This area, while complex, receives considerable attention to mitigate potential issues effectively.

Read the full article