Rothstein: Maybe starting with the scope of the document itself, one of the preliminary items that we raised is that this document needs to include PMA products at the outset. Right now it’s written to address only 510(k) products. Those are the most common types of devices that come to market, and a lot of AI and machine learning-based products that have come to market are in that 510(k) or de novo category. But PMA is an area where our members believe FDA needs to include them, as well.

The other scoping issue that we took with the document is that right now it’s limited to what’s called software as a medical device, or SaMD. We think, ultimately, looking at where products will be developed long term, SaMD is not enough. It needs to include what’s called SiMD, or software in a medical device, because those are the types of software that operate within a hardware-based medical device. And that’s where most of the software today exists in the medical device space. So, focusing only on SaMD right now seems a big shortsighted. We think that SiMD ultimately needs to be included in this concept. Another area we talk about in our response is around this concept of good machine learning practices, or GMLPs. It’s not a fairly new concept in its entirety, but it’s kind of a new area that the health-care community is working on together.

One of those groups that’s working on this is out of the Xavier University’s artificial intelligence initiative. That group has actually been working now for about three years. FDA has actually been a part of it, we have a lot of machine learning medical device companies, pharmaceutical companies, and vendors and providers to those technologies. It’s really been a positive initiative. And that group, last year, actually released a white paper that is very much related to this GMLP issue. So, what we recommend to FDA is that they continue to support initiatives like the Xavier University one; because FDA is already at the table, you have a very diverse set of stakeholders working together on these concepts. Overall, we were happy to see a lot of what FDA put in their discussion paper tracks a lot of what the Xavier University white paper included.

Another area that we raised was the use of real-world evidence for artificial intelligence. So, real-world evidence is this concept where the data around the use of the device can come into either the manufacturer’s possession, or FDA’s, in a number of different avenues and it’s outside of, say, the clinical regulatory or clinical trial context. FDA talks about using real-world evidence to support the ongoing assessment of these devices in the postmarket environment. What’s not clear though in the document is exactly how that should occur. Things like what sources of real-world evidence are appropriate? If you take, say, a wearable device, how much of that tracking data is really relevant to the performance of the product? And then ultimately how that type of information should be assessed and used to make those postmarket considerations. So, that’s another area we brought up.

The other one I’ll mention is on clinical evaluation. So, this is on the premarket side, and specifically, what is that clinical evaluation going to look like from an FDA review perspective for AI and machine learning software? FDA references a document issued by the IMDRF, or the International Medical Device Regulators Forum. That document was developed by one of their software initiatives. It’s a great starting point, but it’s not yet tailored to the US FDA [regulations] and laws that exist within in this country.

So, at a minimum we said in our comments FDA will need to at least turn that guidance from the IMDRF into something that’s more manageable on the US side, but more information about clinical evaluation will also be important. One other thing I will mention … and maybe I should also caveat this point by saying, overall, we’re really happy that FDA took this on. All of these are constructive-type comments that we gave back to the agency. We are, overall, happy with both where the FDA is thinking and how this document came out. But the other area that we think FDA needs to think about is the general premarket environment in which AI and machine learning will be reviewed. As I mentioned before, this discussion paper talks about a change protocol; so what happens to AI in the postmarket environment when it changes? I think what a lot of manufacturers today are looking for is how does FDA assess the product in that premarket phase in the first place. So we raised that as another kind of key scoping issue that we want FDA to be aware of.

I have no doubt FDA is thinking about this already and it’s something that they’ll take on. I think this was an area that they wanted to address maybe first, or it was just something they were able to release first. But that is this premarket review phase is just another area, I think, that we all need to start thinking about from a regulatory perspective.

Podcast: US FDA’s Artificial Intelligence Guidance May Need Congress And Is At Least A Year Away, Expert Says

More from Regulation

More from Policy & Regulation