PegaWorld iNspire 2023: CDRH Promotes Digital Transformation

The U.S. Food & Drug Administration’s (FDA) Center for Device & Radiological Health (CDRH) improved support for the medical device review process through an enterprise modernization initiative. The CDRH Progress Tracker (PT) portal provides a centralized, plain language, self-service touchpoint for medical device review processes – using a low code platform capability to accelerate application development and increase user productivity. Join this session to see how PT re-envisioned how CDRH interacts with medical device manufacturers worldwide, using a secure cloud-based solution and significantly improving the customer experience.

Transcript:

- Thank you for coming to this session. We saved the best for last and I'm not kidding.

- Yay.

- So my name is Cynthia Stuebner, and I am the industry principal, industry business lead for the US North American Government at Pega. And I am really honored and very excited to have you here from some representatives from the Food and Drug Administration in the US. They've been on a Pega journey and they're here to talk about it. We have Elizabeth McNamara, who's the program director for Digital Transformation at the FDA. She is an MHA and is a health IT executive. Liz is an associate director in the Office of Strategic Partnerships and Technology Innovation at FDA's Center for Devices and Radiology Health. I had to take a little bit of time with that. Liz joined the FDA in 2019 as the director of CDRH's Digital Transformation Initiative and is currently also serving as the acting associate deputy chief information officer at FDA for CDRH. Say that fast three times. Liz has over 30 years of experience in successful planning, development, and implementation of complex health information technology applications and solutions. Prior to joining FDA, her roles have included Director at Blue Cross Blue Shield Association, program manager at NIH, National Institutes of Health and the Principal of Healthcare Consulting at CSC and Health IT Solutions. LLC. Ms. McNamara earned a master's in Healthcare administration and a bachelor's degree in Decision and Information Systems from the University of Maryland. And has not revealed her baseball loyalties. Next to Liz is Ken Sullivan, a principal at Booz Allen Hamilton. Ken is an IT executive with 30 plus years of experience working with clients and companies to develop their strategic vision, create and execute a plan to achieve their goals and ultimately transform their operations. How excellent is that? At Booz Allen, Ken works with clients like FDA on digital transformation programs that take advantage of the Pega low-code no-code platform to streamline processes, lower operational costs, and create unified consistent user experiences that helps the FDA meet its public health mission. Please join me in welcoming Ken and Liz.

- Hi everyone. Thank you for joining us today. I know we're the last session and hopefully, it is the best one for today. I also we're sitting in between the chess match, between you and the chess match and the closing reception, so hopefully, this will be worth your time. So I have to say this as a disclaimer of, everything we say here is not intended to be representative advisor guidance from the Food and Drug Administration. Done. Check mark. So before I go into CDRH's mission statement, I thought I'd talk a little bit about what we are. So I don't know if you guys know, but the Food and Drug Administration actually regulates 20 cents on the dollar for every dollar American spend. We're really quite diverse in what we regulate. And Center for Devices and Radiological Health is actually the second largest center at FDA. And just a few statistics have to go to my notes for this, but we regulate over 23,000 different products, medical device products. Everything from how many you have an Apple watch on that is actually regulated by the FDA, especially the heart monitor aspect of it, to wheelchairs, to tongue depressors, to PPE and respirators. So you guys know we were hit hard with COVID just with those two last ones. Basically, anytime you visit a hospital, what you see and what you use are is probably regulated by the Center for Devices and Radiological Health. We manage about over 18,000 submissions a year for medical device approvals. And we regulate 27,000 manufacturers who manufacture these devices. So it's quite a diverse stakeholder group. Then of course, we have our most important group is our patients in the United States. So this slide talks about three of our major missions in digital transformation. So digital transformation started in 2019 and it was basically an effort to transform, not just modernize our IT systems. And what we mean by that is every transformation includes a business transformation as well as IT. We really needed to transform the way the business did their review process and how they shared information, especially their data. We also of course, have an IT systems modernization effort as part of this and a IT and program governance. So kind of the bumper stickers for these things is for business transformation. We're going from siloed processes per medical device, which as I just said, there's a lot of different types to a core workflow. From modernization, we're going from custom code to low code, hence, Pega. And from governance, we wanna go from policing to enablement. We wanna make sure we enable folks and all of their IT needs and not just police them. All right, so this we call our Frisbee diagram. So we broke those three things into five segments and just like a Frisbee, you can't fly a Frisbee if you're missing a major section. So we definitely pay attention to all five of these. So I'm gonna start at 12 o'clock. So IT and data governance, that's what I just spoke about. We wanna make sure we're working on the right things and we're not working on just one person, whoever's the loudest, you know. That we're working for something that will benefit the entire center and public health. Down at two o'clock is CDRH IT services. So we wanna be able to sustain what we're doing. Digital transformation is a program, but is it really? I mean, it's kind of, it's a new way of doing things. So we need to be able to enable our staff to sustain all of the work that our contractors are doing. Data modernization is really, to me, one of the most important features. Technology changes, data persists. CDRH is burdened with about 85% of our data is unstructured. Submissions come in in PDF files and it's very unstructured. So we are undergoing a large effort to structure that data and get it ready, get it integrated, data quality, data governance, data management in order to enable data-driven decision making so that we can see how effective medical devices are before we approve something similar. And also, we need to enable IT for advanced analytics, including AI and ML. Then over at like seven, is technology solutions. So this is the one that we're most complete with. Pega was our second largest acquisition. We were looking for a solution that had both a CRM capability and a business process and Pega fit that bill and we've been very happy ever since. And we've purchased a lot of other major platforms. So we're kind of done with that little section. Business process improvement, we wanna make sure that we get what our customers, our stakeholders, internal and external, what they need, not what they want. So it's really just in making sure we streamline harmonize processes so we don't automate bad processes. Previous state of affairs, okay, you know, we probably all have something similar to this. But it's not only a spaghetti diagram, which we call it, but you'll also see there aren't a lot of lines. And that just kinda shows that the data is not integrated. In fact, our staff had to log into nine different systems. Have to, 'cause we're in the middle of our digital transformation, have to log into nine different systems in order to review one medical device. So, you know, there's not a lot of integration. There's siloed data. Data was really what was captured in the database in our transaction systems and nothing much beyond that. So that's definitely something we were working on. I got through that fast. Am I talking too fast? No, okay. Okay, so one of the things, the first things we did is we realized we needed help. We couldn't embark on something like this on our own. So we put together an integrator which I think was a good decision. We needed one throat to choke. But we also needed, I mean, managing multiple contractors, it's adds an administrative burden that none of us were prepared for. So we partnered with Booz Allen Hamilton to get this work done. And so Ken's gonna go into our, how this all relates to Pega and why we're here.

- All right, thank you. So my name's Ken Sullivan from Booz Allen and yeah, I think, you know, the partnership between CDRH, Booz Allen and Pega has been very successful. It's been exciting. We use the word "journey" a lot and it feels really overused, but when you do sit back and look at it, we have been on this 3 1/2-year journey and I've listened to some others talk about their time with Pega and it really is a journey. Three and a half years feels like a long time, but you get stuff done and you just kinda keep moving forward. So it's been a great partnership so far. So what we're gonna talk about here then is sort of, you know, three of our early key elements, if you will, to the digital transformation. Some we are done with but they're worth making note of and then others are really where we are sort of forward thinking today. So this sort of idea of establishing the foundation. Really, really important. You know, working with our business to develop processes, workflows, and then we'll talk ultimately the long-term goal is really to make a great experience for our external constituents as well as our internal folks in terms of a customer collaboration. So establishing the foundation, a lot of you, if you are in software development, if you've got particular activities going on, you'll probably recognize, you know, we brought a very product driven mindset where we created agile teams, so very much agile principles with the idea of, you know, achieving that, which most folks that get into Agile are going after high quality, consistent delivery, more capabilities, more features, more products that really spoke highly or loudly I think to our collective team. And so that's really worked out well. We started off with three or four sprint teams, if you will. We've been as many as eight to 10. It kind of fluctuates depending on the capacity and what our priorities are. And then within that sort of agile approach, if you look at that little diagram at the bottom there, we talk a lot about discovery. Discovery moving into a minimally viable product and then, you know, operationalize that product. And so one of the great things about Pega is in that discovery process, that's really where we get an opportunity to prototype ideas. Really work more closely with the business, let them sort of get a better hands-on experience, if you will. It really, it cultivates ideas. It's a real strength I would say that we've leveraged well and I think we're looking to leverage it even more as we go forward. But it does allow us to generate ideas working with product and business owners and really amongst our own team. Once you get outta that sort of, you kind of come to an agreement, you've got a backlog, you're groomed, you're ready to go, then we move into that development deployment around that minimally viable product. And then you look for how do you improve it? You may find that there's a feature that you hadn't ever thought of, or you may be right in line with what you had in your backlog, but one way or another, you're constantly looking to move forward. But obviously, all of that business process, all being said, needs the technology support. And so on the right hand side, we sort talked about, you know, Liz had showed you sort of where CDRH was, you know, we've looked to improve upon that. You know, we wanna take advantage of the cloud. So really putting together a cloud-based architecture, modernizing our technology stack and really optimizing use of enterprise data. So generally speaking, those are some of the really important themes from a digital transformation perspective. When you then start to look at Pega specifically, so how does Pega fit into some of those things? So we, use Pega on the cloud, but as setting Pega up, and we'll talk a little bit more about this, but you know, just getting some things like governance in place. Liz made mention of the importance of IT governance, for example. But there are lots of ways that governance comes into play around user experience, around how you develop codes so that your DevSecOps pipelines work. And really, one of the big challenges that we've had is figuring out how to integrate enterprise data models into Pega. That's been a real challenge for us that we spend a lot of time working on. But ultimately at the end of the day, the thing that we care most about is sort of this idea of having workflows, having basic workflows. When you look at the business of CDRH and all the different offices that make up the center, there's a lot of commonality. And you know, it's sort of classic that they were off either doing manual processes, maybe they had processes that were wedded around the Microsoft solution stack, or they had legacy systems that were very expensive to maintain. So really, what we have spent a lot of time in our first few years is really discovering and really working on developing base workflows. As we now advance, we're now really focused more on the regulatory workflows that gets very complex. If you could imagine you would want some complexity around those people that are def deciding whether your heart valve is up to standard. So anyway, so that's sort of our two sides of the story so to speak, is getting ourselves ready from a business process and then the technology support that is needed. So when you look at this, it's interesting as I thought about that journey, we really, I would've liked to extend the right hand line out much further, but this sort of does summarize where we are in putting that foundational roadmap in place. Again, I made reference to, you know, sort of our reference architecture from an enterprise perspective. And Pega is right smack dab in the middle of that. Pega is very important to what we do. So we've built a lot of capabilities around it. It's a centerpiece. Within that Pega, you know, we've certainly, we adopted, I'm sure many of you're familiar with this idea of the Pega layer cake. And really, what it boils down to, I mean, you know, it feels a bit like marketing when you first see it and hear it, but it's true. It really is important that you adopt sort of this idea, the center out strategy that Pega likes to talk about. You really wanna get your core capabilities, your foundational things in that base layer so that you can extend out. You'll always have to specialize here and there for very specific application needs, but that idea of investing in the core is extraordinarily important. In a similar way, we took a lot of time in developing a UI design system. And so again, it gives us the ability to predict, you know, as we talk to business owners and product owners, it gives them an idea of what things will look like. You have less variability with your designers going off and picking, you know, is it gonna be red, green, blue or black or whatever. That's oversimplification, but the idea holds. And it's also great to help you out with things like 508 compliance, which is a big deal in the federal space. So that was a lot of time spent in our early days just kinda coming up with some of that foundational work, the architecture to support it, etc. As we start to move to the right, we started implementing things. So we implemented our CDRH portal, which is an external collaboration vehicle, and we implemented our internal decision management portal with a bunch of different applications that are on the bottom there that we'll talk a little bit about. But then you start to project where are we going as we go forward? So we talk about wanting to implement those more complex regulatory workflows. How can we start to take better advantage of our data using data science capabilities? Maybe within the platform, maybe outside of the platform, but irrespective you want to be able to integrate. And so when we think about enterprise search as a great example, we may not do all of our enterprise search within Pega, but we know that we're gonna want users to be able to call our enterprise search feature. And so that just that overall integration and having Pega sort of in the middle there, east to integrate with is just another key capability. So we take a little bit of a deeper dive. So we start to think about our decision management portal. So one of the big things that was really important to the staff and to CDRH in general was this idea that they wanted a one-stop shop for their staff, for their 2,000 folks who were reviewing and logging onto lots of different systems, so on and so forth. So one of our early initiatives was really to bring the decision management portal up and use it as the one-stop shop for not only the work that you're responsible for, but eventually, we look to fold in all sorts of things that maybe your administrative tasks that you need to do, your learning, your HR. But the intent here is that again, you can integrate. So even though... The airplane passover. Even though we may not have sourced or wrote everything in Pega, and our goal is to bring more and more into Pega, we've created a pathway so that not everything has to be in Pega right out of the gate, but because we were able to do that integration, we were able to facilitate, right? As you want to bring in more applications, you can bring them in early and then pick and choose which ones you wanna modernize. Another key element of this is, is that what you see on the decision management portal as a reviewer is oftentimes, the exact same thing that you'll see if you are an external user. So if an external user has a question, the reviewer's gonna see the very same thing that the questioner has on the submission side. So again, just taking advantage of sort of the way that you can sort of centralize your data, you can use Pega to create these very common user experiences. It's a very powerful part of the platform. So then when we think about our, you know, collaborating with customers and so, you know, it's the government. We have rules that we have to follow, there are certain things that you have to do, but we wanted to make this as great of an experience as we could. So lots of challenges with that. As, you know, external stakeholders, as all external beings, they want to know what's going on. Where is my thing? What's its status? Prior to the customer collaboration portal that was a phone call or perhaps an email. When they wanted to submit to CDRA. So Liz talked about the hundreds and thousands of devices that get regulated, right? There's a whole process. That process was all manual. It was dependent upon ultimately dropping a thumb drive in the US mail. As you can imagine, that's not-

- Can you believe it?

- That's not a very exciting user experience, right?

- [Liz] 2022.

- Yup, so that was a challenge for us. And then, you know, just the submission itself was very clunky to pull together. We haven't completely cracked that nut yet, but we're working on it. And then as you look on the right-hand side, you know, we created, we've got some things, we have some wins under our belt in production today that people can log on and see the progress of their submission. They can log on and actually submit electronically. We got some amazing praise that Liz will talk about a little bit later on about people. It's like, wow, that's just amazing. And then we are very ambitious as we go forward. We're looking at how can we increase that collaboration? How can we make a 360 degree view of the data, sort of along the lines of what I was just referencing. And then even further out structuring and validating that data, all that massive data that we bring in, and ultimately improving the efficiency of the review process, right? I mean, at the end, that's what entities are looking for. They're looking for efficiency. They're looking for what can you automate? Where can you take out inefficient, poorly defined processes? So then we just look at, again, the journey map itself. So you can see we, you know, hey, a portal. Portal 101, you gotta be log on. So we had to figure out how to get our security access controls, logins, all that kind of good stuff. Then we implemented the progress tracker, electronic submissions, more progress trackers. So that sort of idea around iterating, right? So we got our first set on, learned some lessons, figure out what do we need to do to get the next set on. And we'll continue to iterate over that throughout the course of our journey. And then, as I said, we're in the middle now of, you know, developing a backlog, starting to prototype around the customer relationship management feature with Pega. Really excited by that. And then further out, you know, we see opportunities to structure ingest and again, just to sort of promote a better user experience. Something that we'd all benefit from. So the progress tracker, the thing that was really interesting about the progress tracker is, is that we had a really good business owner. And so, you know, this idea of a plain language explanation was really, really important. So what we said, the person reading it could actually understand and sort of not have to translate into a lot of regulatory language. But we started to build business rules around that. We still had to put in, you know, if we wanted to show timelines or detailed information, there were still rules that we needed to come up with. And so that just speaks to Pega and its bread and butter. We had to pull data from external systems. We had to sort of interpret that data, build these rules, create a very consistent user experience. So you continue to hear that theme, right? User experience, user experience, user experience. And we also put in a nice ability to get customer feedback. So that's where we do get, you know, we get pluses, we get minuses, hey, way to go, Hey, we'd really like this to, you know, would be a great improvement. So it's a nice feature of the progress tracker. With our electronic submissions, I mean, I think it says it all, right? Just the ability to do a submission electronically was a huge win. We created a very simple drag and drop interface, just what you would expect. In the backend though, there was a lot of integration that we had to work through. And again, it just speaks to, you know, our ability to sort of take and integrate with Pega into our backend cloud, our, you know, the architecture that we put together, incredibly important that we not let files into the CDRH ecosystem, if you will, right? So you've got vulnerability scan, you have to make sure that formats are all right. Send out a confirmation email as you would expect. And very easily make it so that people can share the status of their submissions. We sort of put that in the control of the end user. So then as we think, again, around that whole customer collaboration portal. so as you might appreciate, the amount of conversation, the amount of questions, the amount of technical back and forth that can occur when you are talking about a very complicated medical device is extensive. We've only begun to just tap the very surface of that. But we have great ambition to really deepen that experience so that when a submitter does call and is working with a reviewer, they have information right there at hand about not only where their submission is, but the content of their submission. Where does it sit today? What's missing? What needs to be done? We'll take advantage of some of the features right in the CRM knowledge management. Really about the next best action advisor. We think that's gonna be a really an interesting feature. You know, this idea of co-browsing, so you wanna make sure as all I think all customer service, applications strive for, you want both the questioner as well as the person answering the question to see the same thing, right? That whole idea of co-browsing or having a common view of things. We're building a lot to sort of pull our communications, keeping it within the web, keeping it within that walled garden of our portal so that we're not so dependent upon emails, things get lost in people's email boxes. But we know that mailboxes we will continue to be a thing. So taking advantage of the mailbox automation capabilities of Pega is another feature that we're very much looking forward to. So again, I would just sort of emphasize right, this idea, we've been on a journey, we're about 3 1/2, give or take, years into our journey. We've got a long way to go, really excited about some of the new features that we've heard about here this week. But it's been a great collaboration, a great partnership. With that, I will give back to Liz and she can sing our praises.

- [Liz] Yeah, so I did wanna say on that last screen that the progress tracker, you wouldn't believe how hard it is to report on a clock. So basically the clock feature of how far along you are in the process and how far, 'cause we have FDA has SLAs, they have to review devices in a certain amount of time. So, but if you have an amendment, the clock stops, you know. Once they send in the response to the amendment, it starts up again. So that was probably the most complicated thing we had to do that in figuring out who has authority to view their progress, drive their submission, to view the status. You know, we need to make sure that we're not giving basically intellectual property information to the wrong folks. You know, everybody talks about HIPAA and all that other stuff. FDA's biggest thing is intellectual property. The intellectual property the FDA has is immense. So we have great concern, let me go next side. Yeah, so I had the opportunity to speak to industry, to the people who sent the manufacturers who submit medical devices last week in Chicago. And I thank them for their feedback. On the customer collaboration portal, the ability to see their status and to submit electronically upload has been a tremendous success. You know, I'm not gonna read all these, but we also heard, I've been working with FDA for 37 years. It's about time, you know. So you know, when I went to the conference, I asked, I said, how many of you have rushed to the mailbox in order to get your submission in before midnight? You know, half the people raise their hand and they were very thrilled. So we wanna take what we've done through our external stakeholders and the customer collaboration portal and start bringing it into our staff. I mean, that's one of our main goals is we need to reduce the burden of our staff. We need to be more efficient in how we review medical devices. So we're gonna kinda turn our focus now on that and that's gonna be more 'cause the processes are so complex. So that's gonna be difficult. But we are very excited about where we've been and couldn't have done it without all the partners with Pega and Booz Allen and our industry. That's it.

- [Cindy] Okay, so we have some time for questions and if anybody has a question, just please let us know who you are and where, where you work and then we'll get your question asked.

- [Tony] Yeah, Pega, Tony with Pega, just a few minutes ago, I was curious, did you guys start on-prem and transition to PCFG, or you started on PCFG?

- Actually, where are we? We're SaaS.

- [Tony] So okay, so you're on the SaaS. So you started on the cloud then?

- We did.

- Versus on-prem. Okay, thanks.

- That was one of our directives is that we had to get off-prem to move to the cloud. So that was definitely. And we were the first center at FDA to actually have a real cloud presence. So we had to work through all of that as well.

- Congratulations.

- It's a big deal.

- It's been generally good. I mean, I will say that Pega's SaaS support, it's not perfect. None of us are, but we get really good support. I mean our portal is available. We hardly ever have an outage, which is a reflection of both the code quality but also the service that Pega provides. So I would say that's been a real benefit. Something we were skeptical of coming in, but I think that's been something we don't think about much anymore.

- [Tony] Okay, thanks.

- [Jennifer] Jennifer Zwicke with USDA NRCS. So on some of the files that you said the folks submitted, PDFs, ZIP files, you know, the examples up there. Does your system open those up and process those or do you still have a human then opening those files?

- So, yes to both. So they do come in. One of the first things that we had to do, which I made reference to, is we have to open those files to just do the vulnerability scans, right? So before it can actually get into our manual process that happens today, much of what we kinda have on our roadmap around structuring the data is really the intent behind that is to get into the heart of automation and really start to take the thing, you know, manual intervention out of the process to the ready extent possible. So our submission process is sort of a big step forward, but is only a step, but that is our intent to reduce that manual touch because it's expensive. I'm sure you guys have a similar experience.

- [Audience Member] I have a question. Do you have a typical timeline for how long it takes a submission to go through the internal processes and get approved? Is there a common thread with that or is it very difficult to determine because things are so different? I mean.

- The common thread is faster.

- This differs by submission types. So all of those products can come in as in print submissions. There's a five, 10K, there's a whole bunch, but I think there's six submission types. Each one of those has different timeline SLAs, so to speak, but it's not documented as well as it could be. So we kinda had to figure it, unravel it all out.

- [Liz] aI see. And so that's kind of an agreement with industry that once they send it in and, you know.

- Actually, it's more an agreement. They actually pay us based on us meeting that they pay user fees. So yeah.

- [Audience Member] Okay, so there's an incentive there.

- Yes, there is.

- Great.

- [Cindy] Any other questions?

- [Pat] Hey, I was just curious. I'm pat with Pega. Over the last couple days, you've heard a lot about, you know, AI, gen, AI, etc. And you had next best action in there, but has this got you thinking about anything else that might be applicable?

- So I'll tee it up and then you can kinda answer it because. So I would say we're very interested, but the government is conservative and so our ability to really start to consider some of those more advanced techniques, if you will, large language models and LPE, AI, etc. It'd be a little bit constrained. So we will dip our toe in the water, so to speak, but we're limited as to just how much we can do until.

- Right. So FDA has a central IT. And they do most of the infrastructure, the cloud, the networking, and they have taken on approving large language models. So we're waiting for them and they, in turn, FDA is waiting for HHS to approve them. So it's kinda like, yeah, we're still kind of on hold, but stuff like NLPs and other advanced analytics we can do, it's just really the generative AI that we're being asked to wait on

- [Audience Member] , Booz Allen. Just curious to understand how the portal, the industry portal that you've built is handling the documents uploads. Like have you faced any challenges like the size limitations and how did you overcome those?

- That's not high, is it?

- Well, some of it. So we have hit some... Most of our limitations were theoretical. So the business is like, "Hey, we don't want any size limitations." Well that's probably not very realistic, right? But we did sort of size ourselves for what we knew to be the common, you know, what was typical. So as long as we stay within that typical range, we've had very few problems with file uploads. But we do have an upper limit and we expect that that upper limit, as technology advances, right, you see increases in that upper limit. And there are limits really even within Pega, how much Pega can take on. But those limits are increasing. I don't know if we will always stay caught up to where industry needs. So you can imagine that when a submission could be, we hear a lot of like, "Hey, there's gonna be a video included with a submission." We haven't seen that yet, but we expect that as the portal gets more and more advanced, as CDRH gets more and more advanced and be able to taking, being able to take that kind of information in, that we will see those challenges. But so far we've kind of, we've kind of been able to stay at a threshold that is very operational, if you will. But we do see that in the future as being a challenge. Cindy's gonna get her exercising today. More steps.

- [Patrick] Patrick with Pega. I'm curious on the shot clock or the clock that you had on there for timing, Have you experienced, does that have a business rule built into it while it'll start to alert or escalate as time is running out or as you hit the pause button come back on? How do you command?

- Our industry gives us the red alerts. But yes it does, right?

- Yeah, I mean I'd say it's more of a backlog item. So again, what Liz was referring to, just the regulatory workflows, we're not doing them justice. It's incredibly com complicated. Again, you would expect it to be, when you're talking about making a new heart valve, a new knee, right? Those are very complex interactions between, you know, incredibly bright people both within the FDA as well as industry. Getting those, they like to, you know, FDA likes the word harmonize. Getting those workflows harmonize is just an incredible challenge. So as we get further and further along, absolutely, we would expect that as we get more and more of those workflows sort of figure out, there are sort of common denominators push that into our base workflows. That kind of notification is very much built in. We do some very simple notifications within our base workflow today, but they don't necessarily map all the way back to what the industry has submitted just because we just aren't that far along yet in our complexity curve.

- Yeah, so that practice tracker's actually reporting on our legacy system transactions. So that's the work that we have to do for staff is to make that part of our workflow, Pega-based workflow. Right now we're just reporting on the legacy systems.

- [Cindy] Any other questions? Okay, well please join me in thanking Ken and Liz for their time today.

- Thanks, all.

- Great presentation.

- Thank you. Good luck in the test tournament.

- Yeah.

Tags