Antony Guerra 0:06
Good afternoon and welcome to strategies for providing application training and support to maintain user satisfaction and productivity, a Health Systems CIO media in production sponsored by 314e Corp. Just a little housekeeping before we get started. My name is Anthony Guerra. I'm the editor in chief of health systems CIO. And I will be your moderator today, we're looking forward to your participation. You can send in your questions or comments at any time using the q&a box, and we'll take them later in the programme. Nice way to view the screen, click on the top center, get it in side by side mode. Then you can adjust the divider to get the video boxes and the slides the size you want them. And it should say speaker view in the top right hand corner, just so you see how we're going to spend our time today we're going to go about 40 minutes with our main panel discussion featuring Jess Cornelia CIO at Sutter Health, Mike Restuccia SVP and CIO at Penn Medicine. Karen Marhefka, deputy CIO at R W J Barnabas health and Ryan Serat, Director of Training and Development with 314e. So lots to talk about. Let's jump right in. Jess, let's start with you. Can you give us an overview of your organization and your role? Absolutely. Thank you, Anthony. So I am Jess Cornelius. I am the CIO here at Sutter Health. We are in Northern California. We serve about 100 communities here, we have 24 hospitals. We're a $13 billion organization and we have about 5000 physical locations to support across the northern half of the state.
Antony Guerra 1:41
Very good. Jess thank you, Mike.
Michael Restuccia 1:46
Thanks, Anthony. Michael Restuccia, Senior Vice President, Chief Information Officer for Penn Medicine. Penn Medicine is the integration of our patient care facilities, six hospitals, hundreds of clinics along with the Perelman School of Medicine. So from a responsibility perspective, patient care, as well as research computing possibilities. Very good, Mike. Thank you, Karen.
Karen Marhefka 2:14
Thanks, Anthony. Hello, everyone. Karen Marhefka the deputy CIO, Vice President for R W. J. Barnabas and Rutgers Health System. We're the largest health system in the state of New Jersey, the largest private employer, we have 12, almost 13, hospital acute care hospitals, we have over 350 practices within the medical group, over 9000 physicians. And my role is primarily focused on the ambulatory side of the house. So all of those, the huge medical group, our new affiliation, our partnership with Rutgers University, and I'm excited to be here today. Thank you. Thanks, Karen, Ryan.
Ryan Seratt 2:55
My name is Ryan Seratt. I'm the Director of Training and Development for 314e. And… we provide training products and services for healthcare IT and from a consulting standpoint. Very good, Ryan. Thank you. Alright, let's jump in. Jess we're gonna start with you. Please discuss the challenges around providing users with software training and or support in the following scenarios. You have new employees that are coming into the organization, existing employees when they have to be trained on completely new software, and then sort of the trickier situation that we're really going to explore existing employees on changes to existing software. So just give me your thoughts around those. Yeah, absolutely. So the biggest challenge is meeting employees where they need to be met. And I will say this, this is probably everybody's case, with the pandemic, we've had a really high rate of turnover. We are constantly looking at how do we train travelers. Can we even take clinical staff out of clinic, to do training. So we're dealing with all of that, and each of these scenarios for new employees, we need to get them up to speed as quickly as possible, and provide them as many venues to do that. So whether that's webinar training, tip sheets, training on the job, we need to provide that and we do existing employees on new software. So we have probably about a 100 new things coming into the environment every month. Well, that's a real challenge. So we tackled that, through our service desk with an IS through trainers that we work with clinical operations and business operations on and try and get people in. And then on spot training, you know, what can we embed within that software to help with the training, and then existing employees on changes to existing software? We actually went through a period for five years where we didn't do anything new in any of our applications. So now we're dealing with that challenge. About 12 months ago, we started saying you know what we need
Jess Cornelia 5:00
To make some changes we need to standardize, we need to change process. So that has been a challenge for us. But again, we've partnered with operations to try and pull that together and make that effective for everybody.
Antony Guerra 5:12
So partnering with operations, talk about that a little bit more. Yeah, absolutely. So within the business units, and I say business, anything outside of clinical is what I'm referring to. And then in our clinical areas, we've actually set up optimization training teams. So they go out and round, they might have a certain color shirt on on a certain day to say, hey, we want to get you trained on this. Anytime we take anything alive. We do that for 30 days after, and then we have monthly rounding where we go out and meet people where they are and say, What are you struggling with? Or maybe we've looked at the data and said, Hey, we think we have some issues here. Can we show you how to do this in less clicks? Can we help you with what you're struggling with? So we are very proactive in that approach? And we found that to be really, really effective?
Antony Guerra 6:00
Very good. Mike, your thoughts?
Mike Restuccia 6:02
Yeah, of course, very similar. And we've kind of embraced the idea of every, all training being virtual at this point in time, and relatively low touch.
Mike Restuccia 6:16
So those two components have led us to, obviously, all online type of training to take place across the board, whether they're new or existing. And I think that has been a big lift for all of us from multiple perspectives. One is to make sure you have the right technology in place, not only on site, but your employees need to have the right technology in their home or in their location, in order to be able to connect, and I would say we had a few bumps along the way when we were forced into this situation.
Mike Restuccia 6:55
But my trainers have shared with me, they said, you know, luck favors the prepared. Mm hmm. And and I will say that, we were generally prepared just because like you, we had begun the efforts of saying, you know, what, why are people always having to come on site, and be in person for training, we had this belief that that was the most effective way and, quite honestly, was the only way we knew at that point in time. But after I think, some initial efforts in standardizing the technology working with our employee base, we are finding that virtual and remote training is much more efficient, it's equally as effective, and allows us to be quite nimble, in the manner that we can provide education and at this point in time, we can actually provide more training to our end users, because you know, it's simply
Mike Restuccia 7:52
clicking on a link and getting to the site or clicking into our knowledge management system and getting that education versus dragging people into a particular location, whether it's in one of our city locations, or suburban. So I think that's kind of the challenge we encountered like most folks, nothing is as easy as I make it sound at this point, I'm sure that I know those first. 30 to 60 days were bumpy and rough. But over time, we've really smoothed it out and virtual low touches is our approach.
Antony Guerra 8:29
Mike, what are your thoughts around? How you evaluate if someone's absorbed the training? So are there tests involved? So you know, and I just actually a little bit, uh, we were talking about before, we went live today about different documentaries. And I just saw one, I forgot to mention everyone on Boeing, and what went on a Boeing. And one of the things was, the training was insufficient, changes were made, obviously, this is completely there. So the changes were made, and the training was insufficient. The testing was in there. So does that, when do we institute testing to make sure that the training has been absorbed?
Mike Restuccia 9:10
So as part of our training classes, there's a testing component to.. to ensure that you meet a certain threshold of knowledge gain. But then we do three follow ups Anthony. The first is those that participate in the training actually rate the trainer for effectiveness. And that's done immediately at the end of the training session. So there's a piece of feedback gain there. Then the second thing we do is we survey the participants and say, Well, how do you think you did? Here's your score. But how did you think he did in the class and what could we have done better? And then the third piece, and I think that's this is actually the most important piece is several weeks after the training, usually 14 days, we go to the employees manager and say, you know, did did your employee grasp what they needed in the training in order to get their job done, or, you know, were they like Mike, and they needed lots of hand holding remedial assistance, and you know, needs to go back and take the class again. But it's those three pieces that come together, that allow us to continually improve our process for virtual training. That is pretty a comprehensive approach to making sure the material has been absorbed. Karen, your thoughts? So um, my response to this, my answers are going to be I have a little bit of a swag to them, since our organization is in the middle of a massive transition to… to different EHR. So we're going from at least 12 different systems or more to one. And we're not doing big bang, just given the sheer size of our organization. So we're doing it in what we call waves. And those waves are groupings of hospitals, as well as portions of our medical group. And there's got to be, there are six of them, for which we're almost ready to launch wave three. So from a training perspective, our entire organization is training, initially training for our new… our new EHR. So we're having and this is going to be a great question for me to ask my co panelists. As we get into that portion of the programme, today.
Karen Marhefka 11:34
We are struggling with how to keep our focus on our implementations, while at the same time, we've got half of our organization almost live on our new system, how do we support them, as we're continuing to do upgrades of our new system as we're training and bringing other hospitals and again, part of our Medical Group live. So we're in a conundrum, we're working it out as we go. We're…we're not incredibly good at it, we… we need to get better as every… as every wave progresses. But I will say this, one thing that… that we have learned is that for our folks who are onboarding into locations that are live on our new system, we've made their training part of what we call our embark programme, which is our orientation programme. So that all of that is all wrapped together with their, with them, obviously, with them coming on board and learning everything about our organization. They're also learning how to use our… how to use our EHR. And number two, we don't do anything without partnering with our operations, colleagues, nothing. We all are completely tied at the hip, I am to the CEO of the Medical Group and the COO of the Medical Group, my colleagues on the acute care side, the same for them for each CEO of their acute of their acute hospital that's going live, we don't make a single move without without moving together with that operational partner. So from a training perspective, we just stay in lockstep.
Antony Guerra 13:13
You know, Karen, it's funny. I mean, how long did it take us as an industry? To figure that out? I mean, right. 15-20 years, it used to be, oh, hey, we bought a new EHR. We're gonna roll it out tomorrow. And then you started to hear you need to have the clinicians on board, they need to be involved in the buying process. And that's what you're talking about. They need to be involved every step of the way. It can't be an IT project. Correct. Anthony, when you and I first met I was at UMass, outside.. outside of Boston. And I was an enigma at the time, given my… my IT role, my leadership role. I am… I think I'll use the phrase again, I'm the least technical CIO you'll ever meet. I came from the operation side, and they asked me to step into a role where I would be the face of the implementation rather than an IT known IT entity. Instead, they wanted an operations entity and I enjoyed it.
Karen Marhefka 14:13
Think are UMass was better for it? I think R W J Barnabas is also going to be better for it. But it's a it's a unique skill set that that I that I bring in a lot of people are bringing into the role to the CIO role. Very good. Ryan, your thoughts?
Ryan Seratt 14:32
So… so working with a lot of our clients, some of the… the trouble that people are having, is that… that used to we used to do live training for everything. And I think that in a post COVID world, is there really a post COVID world? I think it's a COVID world, right? It's never going to end that. You know, there's a lot of restrictions depending on what's going on in the environment.
Ryan Seratt 15:00
And, and how it's affecting our staffing, that remining develop new ways to reach people. And the biggest challenges that I see that is that sometimes our classes are half the size, sometimes we're doing virtual training, which doesn't allow for a lot of kind of over the shoulder looking at it. So how do you support people, as they're, as they're learning, and the way they needed using the different presentation modality that we have. Also, along with that, our training staff is having to learn new skills. So they're, and they're learning how to be, you know, how to present on the telephone, how to present in a recording. And so there's some challenges in that, along with the technology to deliver those methodologies, but presenting presenting, it's getting used to presenting in many different ways.
Antony Guerra 15:56
All right, very good. Mike, we're going to start with you on this one, how do you work with software vendors to understand the changes that will be involved in any upgrade? So you can take the necessary steps. How do you make sure you get notified in sufficient time to do what you'll need to do internally?
Mike Restuccia 16:12
Yeah, my comments here are more directed towards what I would call our primary vendors.
Mike Restuccia 16:19
Obviously, we all have hundreds of vendors that are within the space, and many of them, you know, their vendors, their staff actually perform the training and to… to staff, particularly in niche, departmental areas. So you know, that sort of takes care of itself. And in many instances, although through our change control processes, we're aware of any upgrades or enhancements that are taking place. A check mark is always, is there training taking place and who's training on it. And normally, it's the… the vendor in those instances. But, you know, I look at our primary EHR and lab and other clinical revenue cycle type systems.
Mike Restuccia 17:00
You know, the… the notification is pretty straightforward for partnership, right, you get scheduled, you know, when you're going to receive your upgrade, you know, the path that you're going to go on in the process. And then you build in your training component as part of that process. So I think with good relationships with your vendors, I don't think we've experienced any major surprises along the way. And making sure that the right materials are reviewed ahead of time with our clinical and revenue cycle committees. And then based upon what we're going to accept as part of any upgrade.
Mike Restuccia 17:38
Then we put together… the right training materials, and whether it's virtual, whether it's through microlearnings, whether it's through our knowledge management system, then it flows through, but I thought this was a pretty good topic that you brought to light. And again,
Mike Restuccia 17:57
I think the vendors have gotten much better at recognising your point earlier that if we have enough insight, we have enough time. And we have enough partnership with operations. And IS, this shouldn't be a big surprise to anybody from a training perspective. Karen, have you seen surprises come up where software was rolled out and the users call you up? And they say, what…? this used to be over here? Where is it? I can't find it? How do I do what I used to do? In my past life? Yes. Given the structure that we have in place at R W J, Barnabas, and especially since we're in implementation mode, one thing that's worked incredibly well to provide that insight and just all around understanding of what's happening with an upgrade, we have something that we call a tower Council. And it meets every week, it's three hours long. And the representation there is well the focus is obviously for successful implementation. But while we're doing this huge thing, we're having upgrades. Our vendor, our main vendor sits on that tower counsel with us every week for three hours, representation by all the applicant major applications, as well as are the support structures that weave their way through each of those applications, training, project management interface, you know, our interface group, our conversion group, everybody is at the table, but then a prominent seat at that table is our main vendor. And that has helped tremendously with providing clarity, transparency, up to the minute, this is what is happening, this is what to expect, this is the feedback, so forth and so on.
Karen Marhefka 19:45
Every week, it's worked very well.
Antony Guerra 19:47
Very good. Jess. So Karen, I really like to hear you say that because that is how we have been successful. So in every project that we have, I'm actually partnered with our CNO, our COO and
Jess Cornelia 20:00
Are CMO. And we meet every other week and we talk about changes, and we look at things from different perspectives. And our main EHR vendors are also included in those meetings. So there's no secrets we really try to treat. And we want to convey this in all instances that our vendors are not vendors, they are partners, and they're in this with us and our success, their success and vice versa.
Jess Cornelia 20:25
So I think from a primary vendor partner perspective, we do that really well. It's the little things that we find that get us just the little changes, we had a change that… that was made to iPads from one of our smaller software vendors completely took them off offline. And it was only 10 practices that called out of the 5000. We have. But that was a huge impact to them. So those are the little things that we're trying to figure out how to get ahead out right now. Just you go back to the vendor and say, Hey, you can't do this, what you did, you can't do this, we need to know about this stuff. So we can get ahead of it. Is that conversation happen? Oh, yes, absolutely. And I think it's, you know, sometimes in the technology world, it's easy for vendors to forget that we support patients, your family member could be here. So just that reminder that we are healthcare, we are not, you know, a hotel where a reservation system may go down, we may not be able to complete the treatment. So we need them to be mindful of that. So it's often good for us to give that reminder. We don't we don't want to figure it out on the fly. Right. Um, quick question. Um, Karen, and… and you both mentioned having an EHR vendor included in these meetings?
Antony Guerra 21:47
Can we get more specific on that is in? Is there a certain level of representation you want? You don't want them sending the intern? So what?
Antony Guerra 21:57
What are you looking for there?
Jess Cornelia 22:00
Karen, do you want me to go first?
Jess Cornelia 22:03
So we actually have, so our EHR vendor, we have second to the CEO of that company that sits on that council, that person reports directly to the CEO of the company. And they come to those meetings every other week. And I actually talked with that person, every week, and say, This is what we're looking at. This is our strategy. This is where we're going, this is how… can you contribute? What do you want to talk about regarding this? Do you have suggestions? So it's a very bi directional conversation, but it does have to be at the right level. And I have been in the position where I've had somebody attend and said, You know what, this is not the right person. I'm sorry.
Antony Guerra 22:43
Yeah, I feel like what do we do for you? Again? I don't remember.
Antony Guerra 22:49
Comment there? Yeah, same. I'm not 100% sure exactly of the hierarchy, you know, how top any top this person or the folks are, but
Karen Marhefka 23:02
very senior level folks from our vendor, who… there's two of them, who are sitting side by side to our executive project directors, there are two of those as well. And so it's more like a two in the box situation, if you come from the consulting world, you know what I mean by that, but this is with a vendor, which is a little bit different. We also have to have the box with a consulting partner to help manage … manage the huge amount of work that... that we're having to accomplish. But no, they're… they're pretty prominent, these two folks are very, very prominent in their vendor… in their vendor organization, and to the point and their infiltration with us, to the point where many folks often ask the same question. Are they with us? Are they you know, with the… with them or them? And when? And my answer to that is, this is great, because they shouldn't actually know the difference necessarily. All right, very good. Ryan, your thoughts?
Ryan Seratt 24:08
No, I just like to reflect what everyone on the panel was saying that, you know, the two things that are really key is really strong partnerships with the vendors, and then project what I like to call project discipline, that everything is planned out. And I think that we all do a good job of handling the changes, handling the launches, it's the…it's the interesting things like Jess was talking about, that were planned for that are really effective. So the more project discipline, the more teamwork the more the more partnership that you can have, the better those activities all go. All right, very good. Ryan, we're gonna stick with you, give your first shot at this one. Once you understand the changes that will be involved in an upgrade, had… and, this is what we're touching on. How do you work with operational and business leaders on training in terms of commission
Antony Guerra 25:00
indicating the impending changes and getting feedback on what type of training is needed? How do you work on scheduling, measuring effectiveness and collecting feedback on the training itself? And how do you determine this is an interesting one if the training should be voluntary or mandatory. So we talked about working with the vendor, understand what's coming down the pike. Let's say we're told what's coming down the pike. Now we have to go and prepare the users to be trained, and then execute on that training. So your thoughts around that; anything else you want to add? Yeah, the… the most successful model that… that I've seen that works really well is there's always going to be a lot of information, a lot of changes that are coming, software is getting more advanced as its capabilities grow and grow, that also makes training it a little bit more challenging. So I always recommend a three column approach. So first of all, what is discoverable? What are people going to find on their own, and actually, they very much the way our cell phones work today is we don't get a long list of training, every time I get an upgrade, that a lot of it, I will just kind of discover, as I go through, there's some new options on a drop down there, you know, something's in a little bit different order, but I can navigate through it based on the knowledge I already have. So there's a discoverable type of activity that we don't need to train for. There's also a… what I call awareness. So something that I want people to be aware of, it's a big enough change, where people can still use what they know, to work through that training. But they, we want them to be aware that there is going to be a change there, and so that they can take what they know and adapt to it. And usually those are very quick, I think I'm a huge proponent of video training for those, just a real quick, here's five things that are changing, please be aware of them. And then you kind of get into what I call skills training. And that's something very, very different, where there's a new skill or a new process, that's very different, that people are probably not going to be able to work through on their own. And that's when we're probably going to look at some sort of training delivery, so that they can interact with an instructor, or they might be an e-learning, followed by a q&a session, something… patient safety, revenue cycle, those types of activities. So Ryan, that's a really, really great piece of advice there. Who's going to make these decisions on where any change falls just mentioned, the change, they got rolled out. And it was something that was not I think discoverable as a tool used. She didn't want them to have to figure that out on their own. You're saying there are some things that people can and should figure out on their own.
Antony Guerra 27:56
So we talked about partnerships, you have IT, you have the vendor, you have the users, the vendor comes and is going to give you an explanation on what the change involves, they're going to have their opinion of where it falls in that strategy. You're going to look at that as it you say, I don't know, I'm IT, I gotta talk to the users. Right? Then the users have to get involved. And they have to then comment on, I'm guessing, is this how it works? They have to then weigh in on? Hey, no, that's not something we want people to figure out on their own. Is that kind of the dynamic? It is. And you know, when you talk about when you're getting people in the room, I think you really want to have their people have different viewpoints. So you want to have training involved with the… with the IT group, and along with operations. And if you don't agree on something, it's probably not at that level. Right? So if two people think it's discoverable, what one person thinks that needs to be communicated, that you probably need to communicate it to be safe, because that means that 1/3 of the people wouldn't have been able to discover, right? So you want to make sure you're covered in that. But usually, it's people do agree on kind of what's in that. So you have to really separate the rocks to find the diamonds. And with so much information, you really have to focus on diamonds during upgrades. Mike, are you initially getting the opinion from the vendor on the degree of training that's involved with any change? And do you find that they often downplay the degree to which the change is going to require training?
Mike Restuccia 29:39
Um, I don't think they mislead us. I think they.. they provide a fair level of.. of what training needs to be provided. I think our approaches if it's really a net new function, it's a new system.
Mike Restuccia 29:55
Maybe very much like what Karen and her team are going through then it's generally some type of delivered training. And in this instance, it's.. it's virtual for us. But there needs to be some one-on-one training that actually takes place. Generally, for upgrades, we follow the path that Ryan had indicated that we try to highlight what the major changes are, what the impact is on operations, and then provide either through micro videos or tip sheets, what those changes might be.
Mike Restuccia 30:26
And then, you know, we extend it a little bit. In that, I think, you know, we're all from relatively complex organizations that generally have some level of residents and fellows, who often tend to be really good at using systems and discovering all those little function features that might be hidden in the system. And, you know, really leveraging those folks to provide support, whether it's to fellow residents and to the nursing community. And then pre-COVID. And this is where I thought we really excelled is we do have an EHR transformation team that is constantly receiving feedback from our end user basis to… what's working well, maybe where we need some strengthening of training. And we would schedule that team, generally eight to 10 people to visit either our ambulatory sites or our particular hospitals. And we'd have a schedule and say, you know, the folks with the whatever coloured vest they were wearing that day, are going to be coming through, and there's two or three things you've identified, you'd like some updates on, and so we'd work one on one with people, but then they're just there for general awareness in general questions and answers. You know, God, I was thinking about this, and when I'm doing a recurring order, and I'm not quite sure when it has to extend past, you know, a certain deadline, how do you make that happen, and, you know, having that shoulder to shoulder support we found really elevated our scores, we participate in one of the class initiatives, the arch collaborative, where you actually measure your adoption and level of utilization. And wow, we were really able to make a big leap forward, based upon that shoulder to shoulder, obviously, at least for us, that's been really, really limited recently. But we're looking forward to reintroducing that for the betterment of our clinicians, particularly.
Anthony Guerra 32:34
Very good. Jes?
Jes Cornelius 32:36
So, Mike, I like what you said there, because I think I want to touch on this a little bit, you know, we've.. we've pushed towards, we have a three tier system where we look at safety and quality. So how do we train to ensure safety and quality? Then what do we do to look at skills, what do you need to do to your job, and then the personalization, we let people figure out, but having those people embedded at the sites for the transformation piece, we've actually pulled all that out, when… when COVID started and really went to the technology training, the tip sheets, everything was virtual. And now we find that people are wanting us to bring people back and I actually have a module implementation that will go live in April. And that specific service line has said, please don't abandon us. If something happens with COVID, we really need the people on site to make sure this is, this can be successful. So we're starting to see a turn back to that. But of course struggling with that as we look at you know, labour.. labour issues and staffing. userway. Yeah, yeah. Yeah.
Anthony Guerra 33:43
Karen?
Karen Marhefka 33:47
So first of all, in our organization, and a lot of the systemic from the fact that we're an implementation mode, but training is never voluntary, ever.
Karen Marhefka 33:55
So we it's, it's a mandate, and we have pretty stern guidelines around that. And then I guess one thing that I.. that I haven't heard mentioned yet was our prompting and training are our service desks or help desks.
Karen Marhefka 34:15
Because when we didn't do a good job of informing communicating training for these upgrades, or just just system implementation, that's the first number that gets called or an online ticket goes in. So we do a tremendous amount of supporting our support desk, if you will, our help desk making sure that they're that they're well.. well versed, well scripted, well prompted for any of those changes. Finally, I'll.. I'll add this.
Karen Marhefka 34:45
I think Jessie described it really well. I'm going to put what our WGBH label is on that where you have the folks that are embedded at the hospitals and within maybe within the practices. That's our informatics programme.
Karen Marhefka 35:00
And it works. It's very, very inbred and, and very active in our acute care cyst in our acute side of our house, our medical group is.. it's not a new medical group, but it's newly combined. So we as we've acquired hospitals, we've brought along those medical groups, but there's medical groups have acted as independent entities for.. for a while. Now they're all one. So it's a new.. it's a new entity. We're not quite there yet with our informatics programme for.. for that, but that's our intention is to take what we have in place for the acute care and bring it over.
Karen Marhefka 35:43
Finally, finally, I'll say, we're having trouble with our words. So transformation, optimization, in format in an informatics programme, versus just pure training, because there's something new to learn. So having a real good definition of each of those things, and what they stand for within your organisation. So you don't get mixed up with.. with.. what the.. you know, what the strategy is for each is extremely important. We are struggling with that right now. We will get there, but we're struggling with that. So words are important.
Anthony Guerra 36:21
All right, very good. Great stuff. Um, next question, Karen, let's stick with you.
Anthony Guerra 36:28
Let's talk about the training and a little bit more detail. These are all the different types that I was thinking of. And if you want to talk about how you decide to use what, when, and then there's the concept of which was mentioned on alluded to before, which is in the workflow available in the workflow.
Anthony Guerra 36:46
You know, you have all this live stuff. In person classroom sessions, which we said went away, but may be coming back at least, it sounds like they requested to come back people, some people want them back, where they're at the elbow, actually, in the hospital or in the practice, ..live educational classes.. live immediately available to chat with a person ..immediately available to chat with a bot. And then things that are recorded and available on demand. So how do you decide what you're going to offer is based on the change that's going on, and how that change is being rolled out. Or you try and throw everything at it.
Karen Marhefka 37:25
So, again, given the fact that we are in implementation mode, primarily, and at all, everything I'll agree with, with my congest, everything during the intense COVID periods, everything was was virtual, we have since moved back to a mandated in person training for our EHR for that initial, you're going live in three in two months, this is your training that's in person.
Karen Marhefka 37:55
Now, and with all the safety accouterments that come with that, but because we were in virtual mode for for such a long time, we do have really very robust recorded versions of the of a lot of those training sessions that are available as refreshers, they're available, we use them a lot for for new onboarding of resources that are going into already live sites. So all that entire list there, Anthony, we do, we all have it, all of it, we.. depending on the situation we employ, everything that you have there, we do have something, I'll use this phrase is called a learning home dashboard that sits within it's accessible within our.. our electronic health record system, where we add content to that constantly, we have to be careful about how we categorize it. So the lookups are intuitive to the person who's trying to find the help. But our articles are tips.. tips and tricks, sheets, all of that. That is a very, very robust part of our support and training programme here, but everything that you see there, we're doing it.
Anthony Guerra 39:10
Alright, very good. Ryan, your thoughts here?
Ryan Seratt 39:14
And I completely agree with Karen, that it's not one size fits all. And really you have to, you know, look at what is the and especially when you're talking about, you know, the different language, is it a communication? Is it a training? What.. what results from the learning interaction do you need from that particular exercise? And I think that kind of informs what the delivery is going to be and what you're measuring against, what does success look like? So, you know, definitely there are things that only can happen in person elearning and video just don't do a good job that there's other things where, you know, looking at the recorded knowledge that Karen was talking about. Using that..
Ryan Seratt 40:00
..then sending out those best practices might be enough to achieve the learning outcome. So there's just it's a.. it's a lot of figuring out what your learning outcome is, what's the best way to deliver it, and what's in your budget to to make that happen?
Anthony Guerra 40:16
Mike, anything you want to add?
Mike Restuccia 40:18
One of the thoughts I had here was, and I mentioned that up front, our approach to be able to provide all these different types of training has made us more nimble, more agile. And thank goodness, because I think I heard just say the same thing we've experienced, and perhaps Karen's experiencing the same thing, our clinical community is exhausted.
Mike Restuccia 40:41
And there's been a fair amount of turnover attrition within those ranks. And if we stuck to just the first bullet, we're going to do live in-person classroom sessions. We don't have enough classrooms, we don't have enough trainers, to train all the new personnel that.. that are coming through. So really, this virtual approach allows us to scale and train more than, you know, make believe there's 20 seats in a training room, you can put train 120 at one point in time, where you know, you couldn't back in the day, if you were just fixated on one approach. So really, as mentioned, you have to be able to provide all different types of training, in order to meet the ever changing flow that's taking place within our facilities. And you know, whether they're travelling nurses that are coming in, or travelling physicians, or just net new employee, it's a lot of transition taking place that requires more training than ever, through the month of January, we have trained as many people in the first seven months of our fiscal year, as we trained all of last year. So you could never do that, if you were just fixated on.. on one approach. So being nimble, I think, is really important.
Anthony Guerra 42:00
Jes?
Jes Cornelius 42:02
Just so the only thing I want to add to this, and this has been new for us, we provide all these different types of training. And I think providing all the mediums has really met our employees where they are. But we are finding that because people are so stressed out, we've had a couple people that have reached out and said, Hey, I took this online class, I'm still struggling, I need help. And so you know, they're like, Can I get some in-person attention, and it's really paying attention to you know, what their level of learning is, and everybody's different. So we've recently started doing that. And I actually started rounding on our clinical areas just to say, hey, we had this change, how did it go for you? What could we have done differently and collecting that, and we've been able to refine it that way. But that's a new approach for us. We can say we can provide 10 things and we think all of them will work. But sometimes it doesn't always work for everybody.
Anthony Guerra 43:00
All right, very good. We got.. my favourite.. Our Ask a co-panelist section. Karen, I think you mentioned having a question. So pose it and if you want to direct it to anyone specifically, go ahead.
Karen Marhefka 43:15
Okay. Um, well, directors anyone specifically, since I'm..
Karen Marhefka 43:18
I welcome all.. all comments on this.. all suggestions.
Karen Marhefka 43:23
So I'm flipping, I went, I had my question, but now I'm changing. I'm changing it up a little bit.
Karen Marhefka 43:31
Training within the workflow. Anthony, you mentioned that. To me.. to me, that is the number one request that we get from our, or I should say that our training team gets the feedback from our operational colleagues or the folks that are on the on the front lines,
Karen Marhefka 43:52
that I'll quote them, that is the best training experience I could ever have is train me on this within my workflow, how we work, we haven't been able to do that. How I, Mike and Jess, have you guys done that? Have you had any success with that? Are you thinking about it?
Anthony Guerra 44:12
Jes.. want to go ahead?
Jes Cornelius 44:15
And so we are thinking about it. This is something that we struggle with too. And I actually wanted to ask, so thank you for.. for posing the question. Um, we are working very closely with.. with, in this case, our EHR vendor to see what they can do to help us and more in there. We're also working with our ITSM to have a link to that to say this is a problem. Can I get a live chat bot to help me work through things. So those are two things that we're thinking about trying to address it, but it's an area we struggle into.
Anthony Guerra 44:50
Mike?
Mike Restuccia 44:52
And we are no different. I think we're all pretty much going to the same vendors and with the same architecture. So in the same shortcomings,
Mike Restuccia 45:00
So, we are no different. But I will say, I go back to two things. One is we are looking to reintroduce our shoulder-to-shoulder support that I think was so helpful. And although not in the workflow, certainly it's as they're working, and that.. that was most beneficial. And then the second thing I would say, and this a little bit of a deviation, no matter how much training we do, no matter, how much communication we do, you still have individuals who say, you know, I got a question. And the.. the service and service desk isn't the right person, my HR analyst isn't the right person.
Mike Restuccia 45:41
And so we created a separate group. That's for all of those one-off questions. And we call it the Information Services Advisory Committee. If you take some liberties, you add an extra A in there. It's known as Isaac. And so as Isaac has been the catch all, for either responding or navigating or coordinating responses to all those one-offs, like, Hey, do.. do we have a patient progression system? Do we.. you know, is there a safety related system that I should be more knowledgeable about?
Mike Restuccia 46:19
And remarkably, these folks handle, on average about 300 contacts a month,
Mike Restuccia 46:27
which doesn't sound like I mean, we have 40,000 employees doesn't sound like a tremendous amount. But what we figured was that we're 300 end-users that were dissatisfied, because they didn't know where to turn they didn't, and maybe went off and spun off. And God forbid, purchased a system that we already had two or three already or did something. So, you know, what I would share with this group is having that on- off system, they're sort of generalists that know how to navigate the complexity of our organizations. Isaac has been really a hidden gem for us that has really elevated and has become I mean, it's just simple. You know, it's a phone call, or it's an email address, and they get rapid response.
Anthony Guerra 47:14
Ryan, your thoughts on the workflow support?
Ryan Seratt 47:18
So I think I need to disclose, I'm a little bit biased on it, we actually have a product that does the workflow support…
Anthony Guerra
Well, what do you know?
Ryan Seratt
…check it out at 314e
Karen Marhefka 47:29
That's not a question..
Ryan Seratt 47:31
So no, I absolutely think it's.. it's a huge gap.
Ryan Seratt 47:36
And so what so once, and oftentimes training is an event. And the question is, how do you support people after that event, and it's really got to be in the flow of work, the closer you can get your information resources to someone, the more likely they're going to use it. Whether that's videos, the micro videos, Mike was talking about wonderful solution documents, but it's got to be easy for people to pull to themselves. And that would be my advice, get it as close as possible, embedded in the tools, make it where people can find what they're looking for. And you'll see a lot of success that way. And then on top of that, the live elbow to elbow support, as people are going through that live interaction. Those things will definitely improve the.. the performance of people adopting to the tools and their usage of those tools.
Anthony Guerra 48:39
Very good. Mike, do you have a question for your co-panelists?
Mike Restuccia 48:41
Do I have a question? So here at Penn Medicine, we have a mantra regarding training, which is
Mike Restuccia 48:53
no training, no access, no kidding. And we implemented that, you know, probably eight to 10 years ago, like many of you, we put t-shirts together with that. We posted it on our, you know, activation posters and countdown clocks and all that stuff. And generally, it's worked. I think more than generally, it has absolutely worked. And I was wondering, do you all have similar types of approaches? And how is it enforced?
Anthony Guerra 49:26
Karen, let's start with you. I believe you.. you said.. yeah.. absolutely mandatory. It has to happen and it's also in person. It's happening. So your thoughts.
Karen Marhefka 49:40
So we get a lot of pushback, and we expect it.
Karen Marhefka 49:41
Shocking.
Karen Marhefka 49:44
We don't have a cool slogan. I wrote down like, no drain.
Karen Marhefka 49:52
Um, and I'll probably steal shamelessly, but that sometimes I do + really appreciate.
Karen Marhefka 50:02
There's a little bit of humour there, there's a little bit of self -deprecating humour, you know, almost, if you will. But it's real, it's honest. And I think that's where we might have made a mistake, because we did apply very.. and they're still there, there'll be there for all time, most likely very serious mandates around the fact that you will not be able to access it if you're not fully trained. Oh, and that means you don't see patients.
Karen Marhefka 50:28
Yeah, you just don't see patients. And so we.. we have to get better at delivering that message upfront, communicating it in a way that makes the person stand up, maybe smile a little bit, take a pause, but understand that.. that that's there for a reason. So thank you for that. I'm taking that one with me. Yeah. You guys.
Anthony Guerra 50:54
Any thoughts? Jes?
Jes Cornelius 50: 57
Yeah. So.. So I think we we have that same struggle. So we've partnered pretty closely with operations and explaining the why.
Jes Cornelius 51:05
And we continually check on that. I just want to say that because it's very easy for somebody to have someone sitting next to them say, yeah, just do this, and you're good. And then you can get access, and you can do things. So we do have some of that follow up that we do to make sure that it's sticking. Yeah.
Anthony Guerra 51:26
Ryan, anything you want to add there?
Ryan Seratt 51:28
No, I think it comes back to testing and making sure that people have a level of competency.
Ryan Seratt 51:35
The only other kind of thought I had in that process is also making it easy for people where a lot of people have worked in other health systems with similar EMRs. And having a test out option is hugely beneficial and a great satisfier.
Anthony Guerra 51:54
All right, we're gonna do a lightning round of final thoughts.
Anthony Guerra 51:58
We covered a lot of interesting round today, talked about partnering. And if we heard anything today, we heard about partnering with Operation, partnering with your vendors, Project.. Project discipline. And I would imagine, you know, whether it's fair or not, to a large degree, IT departments, their.. their quality is reflective of how well people are using the software. So it's not.. you can't just install it and say, I'm done. Right. Nobody can use it, but I'm done. So it may not be fair, but IT is.. is I guess, to some degree responsible for making sure these things aren't just rolled out, but, but used well. So a final piece of advice for your colleagues. I think this is a subject that many are struggling with. Jes, let's start with you.
Jes Cornelius 52:47
Yeah, so I would say stay diligent, don't lose heart, don't treat people as your opponent, they are your team members, we are all working towards the same end goal and never treat training as just a technology problem.
Anthony Guerra 53:02
Perfect, Mike.
Mike Restuccia 53:04
Yeah, along those lines, training and I like what Ryan said training is not just an event, it's a journey. So embrace that.
Mike Restuccia 53:14
And then measure it and mentioned we use the class arch collaborative and improve, see where you've improved and then work to get even better.
Anthony Guerra 53:25
Karen it sounds like if we start with the goal in mind, which is to make sure that people know how to use these systems well.. that helps us look.. work back inside how we're going to do that. Right?
Karen Marhefka 53:37
Yeah. Yeah, we I would encourage
Karen Marhefka 53:40
every organization out there especially I'm using our experience of our.. again going through a massive Go-live
Karen Marhefka 53:46
Don't.. we really loaded up the help the.. might refer to it as shoulder-to-shoulder we're still using at the elbow.
Karen Marhefka 53:57
We've loaded up on our at the elbow, our super user programme, obviously our training to bring everybody live once they're live. Where is everybody?
Karen Marhefka 54:09
Where did.. where did all that go? And because we've moved on to the next wave. So we cut ourselves a little bit short with the.. with the support that needs to be there forever, not just for go-live but forever. So that is.. my piece of advice is don't fall into that trap. Plan for both more robust actually on the post go live side.
Anthony Guerra 54:38
Very good. Ryan, we give you the last word.
Ryan Seratt 54:41
I just like to remind everyone that the cognitive load from classrooms, especially when EMRs and how complex they are, is very, very high. Look for if you really want to see a drastic improvement about the way people are taking
Ryan Seratt 55:00
To software is.. look for how to support them out of the classroom where the work is being done.
Anthony Guerra 55:07
Very good, excellent conversation. Well, that's about all we had time for today regarding continuing education. You can use the final slide in this deck, you'll get an email when the on demand recording of this event is ready for viewing. If you want to sponsor an event with us, you can reach out to Nancy Wilcox from our team and go to our website to register for upcoming webinars. With that, I want to thank our tremendous panel, Jes Cornelius, Mike Restuccia, Karen Marhefka and Ryan Seratt and I want to thank 314e team for making this event possible and our attendees for coming. With that everybody, have a wonderful day. Thank you
