Introducing the CAHPS Cancer Care Survey (Webcast)

Just another WordPress site

Introducing the CAHPS Cancer Care Survey (Webcast)

Stephanie Fry: Good afternoon. Good morning to those of you on the West Coast We wanted to welcome you to the latest CAHPS webinar, where we will be introducing you to the CAHPS Cancer Care Survey. My name is Stephanie Fry, and I will be your moderator today. As you may know, today we are going to introduce you to the CAHPS Cancer Care Survey, which is the latest survey to be added to the suite of surveys under the CAHPS banner. We will introduce you to the survey itself, tell you a little bit about what that includes, and provide you some information regarding both sampling and fielding as well as analyzing and reporting data This survey was developed by the American Institutes for Research and the Mayo Clinic with funding from the Agency for Healthcare Research and Quality, the National Cancer Institute, We have a great agenda today. As you’ll see, we’ll provide you a little bit of information, pause for some questions, and provide some responses at that time. Then delve a little bit deeper into some further information regarding the sampling and administration, and pause for a further set of questions. We will endeavor to get through as many questions as we possibly can today in order to make sure that you leave this webinar fully informed We have a great lineup of speakers today with expertise ranging from in-depth knowledge of the CAHPS program through cancer care and treatment, survey administration, and the development of this particular survey Before we jump into the content itself, just wanted to provide a little bit of technical assistance If you’re having a hard time hearing from your computer speakers, you can join us by phone at the number provided here on the screen If you’re having difficulty with your slides not moving, you can select F5 to refresh your screen and that should help. If there’re other technical questions that you’re having, please go ahead and use the Q&A feature on your webinar panel, and ask for the help that you need and we will get back to you to try and get you up to speed with our webinar here today As you saw from our agenda, we will be pausing at a couple of different points for questions and answers. We will be taking all of our questions via the chat box and so, if you use the Q&A feature, you can type your question right into the box And as I said, we’ll try to get through as many questions as we possibly can here today The materials from today’s presentation will also be available. You can download the slides themselves. So, you don’t have to worry about missing anything using the icon that you see there And we also have some additional resources available for you to follow up, either during the webinar or thereafter. So, without any further delay, I will turn it over to Caren Ginsberg, Director of the CAHPS Division Center for Quality Improvement and Patient Safety at the Agency for Healthcare Research and Quality. Caren, over to you Caren Ginsberg: Thank you. Thank you, Stephanie And good afternoon, everyone. As Stephanie mentioned, I direct the CAHPS program at the Center for Quality Improvement and Patient Safety. We’re delighted to launch this survey on cancer care and welcome it into the family of CAHPS surveys. And I’m very excited to be part of this presentation today I think that there’re probably some new users for CAHPS surveys today on the line sowhat I’d like to do is to give you an overview of the CAHPS program, and the CAHPS surveys, and explain a little bit about what they’re all about So, CAHPS stands for Consumer Assessment of Healthcare Providers and Systems. And these are standardized surveys and related products such as guidance for administration and reporting data that’re all developed according to established principles. And we’ve been around for a while. We started in 1995 with the Health Plan Survey. And as you’ll see in a couple of slides from now, we have grown substantially So, we assess the quality of care, patient experience of care from the patient’s point of view across different settings. And what that means is that when we start our survey development, we begin with asking patients about what is important to measure, what their opinion is about what’s important to measure and about what is high quality healthcare to them. And that’s how we start our whole survey development process. So, the CAHPS surveys are trademarked. The

trademark is held by the Agency for Healthcare Research and Quality. And any survey looking to use the CAHPS trademark must adhere to the design principles of CAHPS – I’ll go over them in a minute – in order to earn that trademark Now, the way to earn that trademark is to work closely with what’s called the CAHPS Consortium. The CAHPS Consortium reviews all of the progress toward completing the survey. It provides comment and guidance. And the CAHPS Consortium is really the oversight body for the CAHPS program It consists of AHRQ staff We support two cooperative agreements or grantees, RAND and the Yale University School of Public Health. And RAND and Yale have been with the CAHPS program since its inception. We have a support contractor, Westat. Again, Westat’s been with the CAHPS program since its inception. And then, as we need to, depending on what we’re working on, we bring in other governmental agencies or other private stakeholders for their advice and consultation. So, for example, the National Cancer Institute was very important in the development of this survey So, here are some examples of what we call our core surveys. And you can see there are a number of them. And this is actually not an exhaustive list. We have some provider-based surveys, Clinician & Group is for our group practice setting, Home Health and Hospice Cancer care represents a condition-specific survey for all realms of cancer care treatment. Facility surveys including hospital CAHPS you might have heard of that that’s HCAHPS You can see some of the others. We still support our Health Plan Surveys. And we have a Home and Community-Based Services survey we just launched as well for supporting a program in the Medicaid program. What’s important to take away from this slide is that the versions include both – for many surveys, both adult and child versions. All are translated into Spanish. And for some of the surveys, there’re other translations as well, specifically those that are mandated for implementation by the Centers for Medicare & Medicaid Services We also produce supplemental item sets And these are additional questions not on the core survey, but address specific topics. And you can customize your survey by adding the supplemental items to the core survey We developed the supplemental items specifically for each survey. So, you can see for Clinician & Group, we have a supplemental item set For Patient-Centered Medical Homes, Health Literacy, Health Information Technology Patient Narratives is our newest. This supplemental item set consists of five questions that patients can answer in a narrative fashion about their experience of care. You can see for Health Plan, for example, some of the supplemental item sets include children with chronic conditions or people with mobility impairments. The supplemental item sets for Cancer Care Survey are access, shared decision making, information from providers. And you can actually make up your own supplemental items to add to your survey. You have to follow the instructions for how to add the questions to your survey And this is because if you add them incorrectly, you might disrupt the measurement properties of the CAHPS survey But you’re free to develop your own supplemental items and add them as you’d like So, I’d like to spend a few minutes talking about the CAHPS design principles As I mentioned, these are patient-focused surveys on topics that patients have told us is important to them. And also, topics for which the patients are the best or sometimes even the only source of the information So, for example, if you want to find out if patients understood their communications from their doctor, the only appropriate respondent for that question is the patient. It’s not the doctor. Only the patient can tell you that information So, that’s an important component of what’s on a CAHPS survey. We include reports of patient experiences of care. And I’m going to differentiate this from a patient satisfaction question. So, a patient experience of care question will ask whether something happened or how often something happened

And the reason why we ask these questions is because they’re actionable for quality improvement. They point to areas specifically for providers to improve quality. Now, a patient satisfaction question is based on a patient’s feelings about their care Were they satisfied? Were they happy or displeased with their care? And really, the answer to those questions are based on what the patient’s expectations were going into the healthcare encounter So, they really address very, very different kinds of issues than an experience of care question does. And they’re not actionable for quality improvement We ask questions and domains that’re under the control of the facility, which means the facility or provider has to be accountable for the things that we ask questions about. And if they’re not accountable, they shouldn’t be on one of our surveys. We ask about patient characteristics, but we can risk adjust for those. We, as I mentioned, develop survey items and protocols based on rigorous scientific development and testing. We include extensive patient input as well as extensive provider input. And importantly, our questions are based on current practices and standards of care. So, we will review a survey or update a survey or a supplemental item set to see – to bring it up to what is current standard of care or current practice might be So, here are some of the uses of CAHPS surveys. The reimbursement and public reporting uses are generally by the Centers of Medicare & Medicaid Services. For public reporting, CMS develops what’re known as Compare websites where you can, for the CAHPS surveys that they mandate, compare the experience of care survey results for a plan or a facility along with other quality indicators on these websites. CAHPS surveys are also used for accreditation, quality improvement, health services research And finally, I’d like to tell you about the AHRQ CAHPS databases. Now, these are voluntary databases that we maintain for our Health Plan Surveys and Clinician & Group Surveys. And you can see a little bit of information about these surveys. They’re voluntary, as I mentioned, but they are widely used And their purpose is so that providers can benchmark their scores over time and also against national – against other submitters to the database So, we also produce annual reports of the information in the databases. And we will also make you a data set for research purposes. A deidentified data set for research purposes, if you send us methodology and we’ll a request for data and your review it for you and determine whether we can accommodate it. So, participation in all database activities and use of the materials are all free and open to the public. So, Stephanie, thank you Stephanie Fry: Thank you, Caren That background on the CAHPS program is a great place to start. And now, we want to move into a bit of a description of why is there a need for a Cancer Care Survey? And we’ll turn it over to Ashley Wilder-Smith, Chief of the Outcomes Research Branch at the National Cancer Institute and partner in development of this latest CAHPS survey. So Ashley? Ashley Wilder Smith: Thank you, Stephanie. And good day to all of you I would first like to thank very much my colleagues at the Agency for Healthcare Research and Quality, or AHRQ, for their tremendous efforts in and partnership in the development and testing of these instruments Really now making them available for primetime Our agencies have had a unique partnership and collaboration on their development, and it’s just – it’s really terrific to see them released I could not be more enthusiastic on behalf of the NCI. And I’d also like to acknowledge my colleague, Dr. Erin Kent, who’s also in the Outcomes Research Branch, who’s been a real collaborator and help to put this presentation together So, by way of some background, I think it’s really important to understand the context of the need for care quality for cancer patients. So, to that end, you’re looking at a slide now that provides some information about the current prevalence estimates. And therefore, potential impact of cancer What you can see is that the number of Americans living with cancer has been increasing steadily

since the 1970s. This is likely due, in part, to successes related to early detection, more effective treatments, as well as the aging of the US population And to that end, currently about 15.5 million Americans are living with cancer. That number is projected to grow to over 26 million by 2040. So, we’re really looking at some unprecedented growth here And what we know from the scientific literature, from reviews and consensus reports such as those that I’ve sort of highlighted and demonstrated here that come out of the National Academies of Science Engineering and Medicine, which was formerly the Institute of Medicine, as well as what we know from patients themselves. High-quality cancer care that really brings the patient’s voice into the care delivery environment is so needed. Patient-centered care has been defined by the National Academies as being respectful of and responsive to individual patient preferences, needs, and values. And ensuring that patient values guide all clinical decisions So, in order for us to really address these sorts of needs, patient-centeredness is required so that we assess patient experience well And the CAHPS for Cancer Care Surveys provide a rigorously tested set of instruments to assess patient experience directly for patients CAHPS for Cancer Care provides the opportunity to leverage the expertise and experience of the CAHPS Consortium. And Caren highlighted and sort of contextualized where these surveys fall within the CAHPS Consortium And really applying some of the rigorously tested and used instruments that have been used in lots of different other settings to cancer treatment This is a new opportunity in cancer care delivery to assess specific experiences of cancer care in both inpatient and outpatient settings And because these surveys were derived under guidance from the CAHPS Consortium, it’s possible to contextualize composites and items from the CAHPS for Cancer Care Surveys to the CAHPS Clinician & Groups Surveys CAHPS for Cancer Care offers real strengths. Developed from this long-standing partnership and collaboration, and many stakeholders, and patient input, these surveys allow for information to be assessed by treatment modality, including surgical, radiation, and medical oncology. There’s relevance for many uses. And I know, as Caren had mentioned, for many stakeholder groups. And I’ll just add some to that list. And they include payers, comprehensive cancer centers, oncology practices, cancer registries, services, researchers have been discussed previously, cancer patient advocacy groups, and professional associations, just to mention a few The domains that’re part of this survey are salient to cancer patients You’ll hear more about these later in the presentation But for example these include enabling patient self-management and care coordination We’ve also explored topics in shared decision-making, among other topics, and are continuing to explore ways to expand our understanding and improve measurements of patient experience. These tools have allowed for an unprecedented opportunity to include the input from clinical partners in oncology settings that. again, originate from the AHRQ Clinician & Group CAHPS Surveys. And at this point, as users, such as yourselves, begin to adopt these surveys, we at the NCI and AHRQ, and I know with the input from California Health Care Foundation previously have been interested in understanding the ways in which these surveys are being used. And we want to understand their application and what you are doing with the data that you are collecting We have a committed collaboration between our agencies to examine some of the implementation topics as their use expands So to that end, we really want to hear from you about your interests, your questions, your thoughts about applications going forward. And so, I’ll just mention as a preview to please feel free to contact us at the email address that we provide at the end of the presentation. So, again, my thanks to this exciting group, and we are thrilled to be presenting these surveys to you today. I’ll transition back to you, Stephanie Stephanie Fry: Ashley, thank you so much And big thanks to both Ashley and Caren for Based on many of the questions that individuals submitted

as part of their registration, we know that you’re anxious to see exactly what the CAHPS Cancer Care Survey includes and how it was developed. So, we will turn it over to Chris Evensen, Principal Researcher at the American Institutes for Research to give us the background and to fill us in on all aspects of the survey Chris? Chris Evensen: Thank you, Stephanie Hi, this is Chris Evensen. As Stephanie mentioned, I’m with the American Institutes for Research. And I would also like to just echo something Caren said, which is that I’m also excited that we have the opportunity to present this survey to a wider audience and get the word out about the survey This survey’s the end product of many years of hard work by a number of people, and it’s very satisfying to see it reach this point. So, as mentioned earlier, the American Institutes for Research in a partnership with Mayo Clinic developed and tested the CAHPS Cancer Care Survey. And in the slides that follow, I’m just going to go over some of the content at a fairly high level to let you know what’s in the survey I just first wanted to acknowledge some of the people on the AIR team who contributed to this survey over the years that we worked on it I probably left a few people off, but I just wanted to put their names up there, because a lot of these people worked quite hard, like I mentioned, over a number of years to get this survey going, and get it tested, and get it trademarked And many of us have been involved in CAHPS work since almost the beginning So, I wanted to just mention a few things about the survey. This CAHPS for Cancer Care Survey is unique among CAHPS surveys for a few reasons. One, it’s the first time AHRQ has co-sponsored a CAHPS survey with an institute in the National Institutes of Health, specifically NCI. It’s the first disease-specific CAHPS survey spanning comprehensive treatment and covering multiple care settings And thirdly, it’s the first CAHPS survey to span both inpatient and outpatient settings The development included many steps. I just quickly want to just show them or list them out for you Formative research, which included – there’s kind of a standard set of things that we do in the beginning, which involves, as Caren mentioned, talking to patients For example, in focus groups. Interviews with stakeholders, an environmental scan, technical expert panel, and so forth. We had two data collections, two field tests. We collected data in two separate data collections Analyzed those data Made several revisions to the survey based both on the statistical analysis of the field test data and also in collaboration with the CAHPS Consortium and AHRQ. One of the big revisions was to align the survey so it’s aligned very closely with the Clinician & Group CAHPS Survey. Some label testing was done for the reporting composites This was not done by AIR, but by a member of the consortium Westat translated the survey into Spanish. So, there’s Spanish versions of the survey available AIR produced quality reports for the participating sites. There are six sites in the first field test. Four sites in the second field test We provided benchmarking reports for those sites. We put together all the materials and the application package for obtaining the CAHPS trademark, which was awarded late last year. And now, our current stage is dissemination, which we’re trying to disseminate the survey by doing things like this webcast We’ve presented at several professional meetings For example, the ASCO Quality Symposium, IsoQual, Academy Health, trying to get the word out about the survey and let people know that it’s out there and ready to use. There’re three parallel surveys that line up with the three major treatment modalities in cancer And this is just one example of one question And you can see that it – the question is identical in each of the three parallel versions, but what’s different is who is being referred to as the referent. It’s, in the last six months, how often did your radiation therapy team listen carefully to you? That’s obviously from the radiation version. And the other two versions, it’s the same question, but you see cancer surgery team for the surgery version. And in the drug version, you see it refers to the drug therapy team. So, all the questions on the survey are like this. They have identical construction, but each one, depending on the version, refers to the specific treatment modality

So, the final (technical difficulty) survey content in the core survey, there are 56 questions total. That includes the substantive questions. It includes screeners, eligibility questions, and also includes About You items, which are basically questions about demographics. Supplemental item set includes 16 questions in total, and there’s actually three screeners in there, and the other 13 items are substantive report items. If you look at a little closer at the core survey, in the core survey, there are 22 survey items that’re measuring six domains of patient experience. There’re also two single-item measures and two global ratings And as mentioned before, the full Clinician & Group Survey is replicated in the CAHPS Cancer Care Survey. The questions have been adapted slightly to refer to the cancer context, but they are the same questions as you’ll find in the core Clinician & Group Survey version 3.0. And one advantage of this is that users of the Cancer Care Survey can compare the scores for those measures to the benchmarks that’re available in the CAHPS Clinician & Group Survey database. As Caren mentioned, there are avenues for getting those data. And if users of the cancer survey wanted to benchmark their performance against the measures from the database, that could be an option because the questions are the same So, with the six core domains, these are the – actually what you’ll see on the slide are the long labels or the detailed labels that were tested that I mentioned And they’re fairly detailed. Some of you, if you’re familiar with CAHPS will recognize the shorter names maybe more easily, but I’m just going to walk through each of these So, getting timely appointments, care and information, this we typically refer to as access to care or just access That’s in the core Clinician & Group Survey How well the cancer team communicates with patients This is the provider communication composite Again, that also is in the clinician group core The cancer care team’s use of information to coordinate patient care. That’s the care coordination composite that Ashley mentioned It’s also in the core survey. And then, helpful, the office staff courteous, and respectful office staff That’s a two question composite that asked about And again, it’s also in the core. Now, the last two items here, the last two composites, are the ones that’re unique to cancer care and unique to this survey Cancer care team supports patients in managing the effects of their cancer and treatment – that’s the enabling patient self-management composite. It’s a very important composite in cancer care. There’s eight items in that composite That’s the one that Ashley had mentioned as well. And then, the second cancer-specific composite is the cancer care team is available to provide information when needed And that’s a three-item composite. The single items in the core survey – there’s an item asking if the patient’s family and friends were involved in discussions, if the patient wanted them to be. And also, if the patient needed an e interpreter, if they were able to get one. And then the two global ratings. One is of thcancer care team itself and then, an overall rating of all cancer care received by the cancer center A supplemental item set has 13 items that reflect three domains. There’re two items that’re part of the – were part of the Clinician & Group 2.0 access to care composite. So, those are in the supplemental item set There were four single items that ask about the ability of the patient to use information from the provider about their care. And then, the shared decision-making composite, there’re seven items Also, very important composite for cancer care as people may know it’s a complicated care process and decisions have to be made constantly With that, I will turn it back over to Stephanie for the – to introduce the next section. Stephanie Fry: Thanks, Chris Really appreciate the overview of the survey We are now going to pause momentarily for some questions and answers So, please go ahead and use the chat box available to you to ask us any questions you may have. And note that your questions are not affiliated with any of your registration information. So, if you think it would be helpful for us to know anything about you or your organization to help frame your question, do go ahead and add that into your question So, a couple of questions have popped up already. So, one question was about defining a single-item measure and

a global rating. So, I can address that one And then, we’ll jump into some of the questions specifically related to the survey. So, a single-item measure is one single question as compared to a group of questions addressing a topic. So, it’s one question that addresses a topic, but unlike a screener question or something else, it’s an item that is designed for measurement for quality improvement, for reporting out to the entities that’re interested. Either patients or cancer centers themselves And the rating items are much like those in that they are single items and they use the 0 to 10 scale to get an overall rating, either of your cancer care, or an overall rating of some aspect of your care. So now, Chris, I’m going to hand it over to you. One of the questions that’s come in is asking, is drug therapy the same as chemo? I wondered if you could say a little bit about that and a little about why you came to the language that you did on the survey Chris Evensen: Yes, it is the same as chemo and I think just in the testing, in the formative research and the testing of the survey, the phrase – Kathleen, you may be able to help me out here The phrase drug therapy was more all encompassing of all the different kinds of treatment that you might get under that treatment modality Stephanie Fry: Great. Thank you Kathleen, did you want to add anything to that or? Kathleen Yost: No, I think that’s correct. It had to do with the testing with the patients and the formative work. That terminology seemed to be more all encompassing Stephanie Fry: Great. Thank you both There’re a couple of questions that ask about, when will the survey become mandated? When will the survey begin? Questions around sort of, when does this endeavor get underway So the survey has just recently been released and is publicly available on the AHRQ CAHPS website Caren, I will turn it over to you to say a little bit more about whether it’s mandated and sort of what the future is for administration of this survey. Caren Ginsberg: So, thank you, Stephanie. AHRQ does not mandate surveys so, any use of the survey now is completely voluntary. It’s not known to me at this point whether CMS is going to mandate use of the survey so I can’t speak to what their plans are. I think you might want to pay attention to what CMS is doing and be ready if they do for some comments If you have some, you’ll be able to comment if, in fact, they do make it mandatory. But right now, it’s only voluntary And I have no additional information on what their plans are Stephanie Fry: Thanks, Caren I know that’s a tricky question and certainly one that’s come up in a couple of different contexts in the questions we have so far. So, important to get that on the table. Chris, I’m going to go back to you, and ask you, what were the basic elements of patient-centeredness that you used to determine the questions? And how confident do you feel that these questions are a comprehensive measure of patient-centeredness? I don’t know if you want to say just a little bit more in depth about the development process that you went through Chris Evensen: Yeah. I mean, we – I think the confidence comes from the great deal of formative research that went in to developing the questions. As I mentioned, I mentioned some of them, but it included an environmental scan, focus groups with patients and caregivers, a technical expert panel, stakeholder interviews. And you approach the topic from all those different angles, with all these different stakeholders, and especially the patients, trying to keep them at the center of developing the concepts. We conducted 16 focus groups Most with patients, but some with caregivers And I think two were in Spanish as well. So, that’s where we get – we try to get as much of the patient perspective as possible. And then, when the survey is drafted, we also conduct cognitive testing, which is where you sit down with the patients, the kind of people who are going to be answering the questions,

and make sure that what the questions are asking are interpreted the way you think they should be by the patient. So, the confidence in the patient-centeredness I think comes from all of that work in the beginning to kind of lay the foundation for the survey Did I answer all the question? Was there more to it than that? Stephanie Fry: No, I think that gives a good background So, we have some questions about what the vision is for which providers would administer the Cancer CAHPS survey. I don’t know, Ashley, if you want to jump in there, or Chris, or Kathleen. There’s a number of you that may have a different or overlapping vision of who you would imagine would be using this survey Ashley, do you want to kick us off? Ashley Wilder Smith: Sure. I mean, honestly, I think that, and in part based on information provided by stakeholders, I think that anyone who is participating in clinical oncology care can be considering the use of these surveys And I think as we learn more about their utility in different environments relative to all of your use, I think that we will have a better sense of additional applications. As we mentioned previously, there are a number of different settings that we could imagine these being used. And there will be clinicians embedded in each of those settings Certainly, based on the modalities that are provided in radiation, medical and surgical oncology, it makes sense that anyone providing that sort of care will be interested to learn more about their patients’ experiences Stephanie Fry: Great Thank you for that So, a few more questions now about the testing and development of this. And Kathleen, you may be best poised to answer this. They’re asking about the field test sites, were they standalone treatment centers, hospitals, or something else? Kathleen Yost: So, we had a nice diverse mix in both the first and the second field tests The first field test, there was several comprehensive cancer centers, health systems that were collections of providers, and hospital-based care In the second field test, we had the HMOs, a very small, private oncology practice and a health system. So, we were able to test it across multiple different care centers and organizational types Stephanie Fry: Thank you. And there’s another question. I’m going to keep you on the hook here, Kathleen, if you don’t mind. Kathleen Yost: Okay. Stephanie Fry: So, the question is, and I know that – so, there’s some questions about sampling and different elements that Kathleen’s going to get into with us in a moment here, but just want to make sure that we get at least this piece tended to. So, they’re asking if there are questions for cancer survivors who are not on active treatment but undergoing follow-up care or more general questions along that nature? So, I don’t know if you want to defer that to your discussion of sampling or if you want to go ahead and answer that one before we get away from the Q&A period. Kathleen Yost: Sure, I can answer that briefly. That in our discussions with stakeholders, we needed to target and tailor how broad the survey was. And the intention for this survey really is intended for more active treatment and not for that follow-up care period. Stephanie Fry: Great. Thank you I know we’re almost at the end of our Q&A period. There’re just a couple of things that have come up with regard to administration. So, a question about, does this survey accept proxies? And the vision is that the individual who had received treatment would respond to the survey him or herself. And for the reasons that Caren discussed with regard to there are some questions for which patients are really the only source of that information, and that includes many of the different measures that you will see on the Cancer Care Survey. So, that said, sometimes individuals need some assistance in completing surveys and depending on the mode of administration, they may need someone to help them check boxes or that sort of thing But the design is that the patient him or herself would respond to the survey And I think it may make sense to hold some of these questions until we get through the discussion of sampling and administration. So, with that, I will hang on to some of these really great questions. And we’ll come back to them

after we have Kathleen walk us through a discussion of the sampling and administration recommendations. So, I will turn it over to Kathleen Yost, Associate Professor of Health Services Research at Mayo Clinic Kathleen? Kathleen Yost: Thank you very much First, I’d like to say that we’re grateful to have had the opportunity to collaborate with the American Institutes for Research on this important work I’d like to briefly just acknowledge the individuals on this slide as well as the physicians and nurse reviewers who helped tremendously with the sampling methodology in the Mayo Clinic Survey Research Center So, target populations for the CAHPS Cancer Care Survey is quite broad and it’s defined as adult patients receiving radiation therapy, drug therapy, in other words, medical oncology, or surgery for cancer in the past six months. The intent of the treatment can be curative or palliative and the patient can be at any point in the cancer care continuum That is, they can be newly diagnosed or being treated for recurrent disease The patient is eligible for any type of cancer in any stage of the disease, except where noted. And finally, patients can receive that care in either an inpatient or an outpatient setting So, the survey’s not intended for measuring care experiences of patients who only receive a diagnosis, but no treatment, follow-up care, or hospice care at that cancer center in the past six months. If they received an active treatment in addition to follow-up care, for example, then that’s fine, but only follow-up care would not be the target audience that we’re looking for this survey. A few in situ carcinomas are also excluded With that patient population in mind, we needed to establish a way to identify patients who would be eligible to receive the survey. We collaborated with diverse cancer centers to determine what source of data they might have in common that could be used in a standardized sampling approach. Their recommendation to us was to utilize billing data So, having a standardized sampling approach is important, because it provides more confidence in the interpretation of differences in survey scores. And the interpretation is that they would reflect actual differences in care rather than differences in how patients were selected So, there’re three parallel surveys. Therefore we have three sample frames. One for each of the modalities of care The instructions for creating the sample frames are laid out in detail in the document Fielding the CAHPS Cancer Care. And I’ll describe them briefly in this next slide So, multiple criteria must be met for adult patients to be considered eligible to receive the survey. Most importantly, the person needs to have a diagnosis of cancer as indicated in the billing data by ICD-10 CM diagnosis of a malignant neoplasm. And they have to have received a treatment for cancer, which is indicated in the billing data with a procedure code. Because drug therapy, radiation therapy, and surgery can all be used for non-cancer purposes, it is critical that the procedure be associated with a cancer diagnosis in the billing data There are lists of diagnosis and procedure codes for each sampling algorithm and those are in the fielding document For the drug and radiation therapy sampling algorithms, the lists of codes are not specific to cancer diagnosis. That is, any cancer diagnosis associated with any chemotherapy administration procedure in the list or any radiation therapy administration procedure in the list would make a person eligible to receive one of those surveys. And I’m using the term chemotherapy administration, but that term is also – would be used for immunotherapy administration or hormone therapy. It’s a general term for that procedure So, for cancer surgery, however, specific types of cancer need to be associated with the surgical procedure that would be used to treat that type of cancer So, for example, a breast cancer diagnosis would need to be associated with some surgery of the breast, such as a resection or excision of the breast So, how facilities implement these criteria will depend on the care setting and the source of the billing data that they have access to So, for outpatient facilities, the cancer for inpatient facilities where the person or the persons conducting the diagnosis must be associated with a CPT code in the list for that treatment modality This is also the case sampling only have access to the professional billing data For outpatient radiation therapy, there’s an additional criterion that the patient has had two encounters. So, that is they had cancer treatment on two separate dates So, two encounters where the cancer diagnosis is associated with the cancer treatment procedure

And the reason for this additional criterion is that many of the core questions in the survey are frequency questions. They’re asking how often a certain behavior was performed So for example, how often did your radiation therapy team listen carefully to you? If a patient only had one encounter with their care team in the past six months, it would be difficult to answer these frequency questions. Moving on to the inpatient facility. Here, the cancer diagnosis needs to be associated with an ICD-10 PCS codes, procedure code. And of course, any inpatient care, and this actually applies to inpatients identified in the professional billing data as well. But for inpatients, the discharge date should be before the sample date. The cancer centers that provide both inpatient and outpatient treatment will want to search for both the CPT and the PCS codes So, this sounds complicated. I understand when you’re hearing this for the first time, it sounds complicated and might feel a bit intimidating, but I can assure you that the collaborating sites that we worked with in the second field test, which is where we developed and tested this approach, they were able to implement the sampling algorithms based on the billing data. They understood the logic of the algorithms, the overarching goal of the approach and they were able to apply the approach in their own administrative data. So, the collaborating sites, I just want to acknowledge them. They were true research partners in this endeavor. We did have some lessons learned in the second field test and they really did help us improve and refine the sampling approach So, once the sampling frames have been created and you have lists of eligible patients, there likely needs to be a little bit of cleanup. So first, you’ll want to access up to date information on patients and screen deceased individuals from the frame Next, each frame should be screened for duplicates. And the process for removing duplicates within a frame is not really important so long as any individual patient is only in the frame once. Finally, since it’s possible for a patient to receive multiple modalities of treatment in a six-month period, it’s also possible for that person to be selected to receive a survey for more than one modality of treatment, which we don’t want. That is, we don’t want to send the same person a cancer surgery survey and a radiation therapy survey, for example. So therefore, it’s necessary to clean up any duplicates across frames. And there’s a couple ways that you can do this. The preferred way would be to randomly assign a person who appears in more than one frame to just one of those frames So, if a person receives both cancer surgery and radiation therapy in the past six months, you might randomly assign them to receive either the cancer surgery survey or the radiation therapy survey. However, there might be some situations in which it may be necessary to manually assign someone to – that’s a duplicate to a specific frame. So, for example, if the cancer center has a high volume of one type of treatment but a low volume of another type of treatment, the number of eligible patients in one frame might be quite high and another frame, it might be low. In that situation, in that scenario, manually assigning a patient to the smaller frame may be necessary to support sample size goals So, once the frames are clean, you look to address to determine who receives the survey based on your sample size goals. So, just a couple of important comments about the sampling method logy. We developed and tested the algorithms prior to October 2015. They were all based on the ICD-9 Diagnosis and Procedure codes. In October of 2015, there was a transition to ICD-10. And the CAHPS Consortium asked us to update the sampling algorithms from ICD-9 to -10. Updating the diagnosis codes, the CM codes, that was pretty straightforward. However, updating the list of the ICD PCS codes was much more complicated, particularly for the cancer surgery sampling algorithm where each cancer type has its own fairly comprehensive list of PCS codes. The CBT codes were not affected by this transition from 9 to 10. So, if you’ve had a chance to review the fielding document currently posted on the website, you’ll see that there’s only one cancer surgery sampling algorithm presented, and that’s for breast cancer, as an example. The list of cancer surgery CBT and PCS codes are available for 14 cancer types, which I will read off slowly: bladder, brain, breast, cervix, colon, esophagus, lung, liver, ovary, pancreas, prostate, rectum, stomach, and uterus So, just to summarize, there’s one algorithm for identifying cancer patients who have received drug therapy. There’s one algorithm for radiation therapy. And there are 14 separate algorithms for cancer surgery based on the type of the cancer Some cancer types do not have a surgery sampling algorithm

When we think of the 17 or 18 most common cancer sites across men and women, a few that do not have algorithms include head and neck, kidney, melanoma, and thyroid Now, if patients with those cancer types also receive drug therapy or radiation therapy in the past six months, they would be eligible to be sampled for those treatment modalities and report their care experiences for those. But they just won’t have the opportunity of being sampled to report on their cancer surgery experiences. So, the remaining 13 surgery algorithms will be ready to be added to the fielding document in the next week or two. And then, it’ll take a few weeks to format that document and upload a revision onto the website. As I mentioned, the logic for these algorithms is not as complicated as it seems. However, unless we do recommend that you work with someone ‘re who understands your practice, your cancer center, and your billing data to really ensure that they implementing the – capturing the intent of the sampling approach and identifying the intended target population. Since these algorithms haven’t been tested in the field, we also recommend a chart review to confirm that individuals identified in the administrative data actually do meet the eligibility criteria I want to just make a few comments on the risk misidentification. The sampling approach relies on administrative data and administrative data are not perfect It’s possible that there could be an error in those data leading to an individual who has not actually received treatment for cancer being sampled and receiving a survey about cancer care. And that could lead to potentially emotional distress. To mitigate this risk, the first three questions in the survey itself confirm eligibility. That is, the survey doesn’t start off with a question, you know, our records show that you received cancer treatment at this cancer center. Rather, they confirm, based on self-report, that the person has been given a diagnosis of cancer and has received the cancer treatment modality for which they’re sampled. In addition, we recommend that the recruitment materials, such as the cover letter, email messages, and telephone scripts include language such as what the example text on the slide to acknowledge the risk of misidentification and mitigate any emotional distress Because drug therapy, radiation therapy, and surgery can all be provided in an inpatient setting, it’s possible for the CAHPS Cancer Care Survey to overlap with the Hospital CAHPS Survey. Similarly, since surgery can be provided in an outpatient setting, CAHPS Cancer Care could also overlap with the Outpatient/Ambulatory Surgery Survey. So, for those facilities where there’s a possibility of this to occur, there’s a rule in this slide to help minimize that overlap by allowing the HCAHPS and OASCAHPS o to be fielded first. So, the fielding document that’s currently posted in the website was uploaded before we had received this text from CMS So, we’re going tsuggest a minor change to the sampling algorithm logic. So, that change along with the addition of the other 13 cancer surgery algorithms will be done within the next couple weeks So, I can’t speak specifically to a timeline, but perhaps check back in 4 to 6 weeks to see if a revised document is available. And I believe you should be able to tell whether the document has been updated by looking at the date in the bottom left corner of the title page, but I’ll let other presenters on this call correct me if I’m wrong So, onto the administration recommendations. So, the recommended method of data collection are presented here The target response rate is at least 40%. Cancer centers should choose a method of data collection and should conduct sufficient follow up of non-respondents to achieve a response rate of at least 40%. The recommended administration protocols, in terms of number of contacts, and time between contacts, et cetera, those are described in the fielding document And I think that brings me to the end of my presentation Stephanie Fry: Kathleen, thank you so much for that really important overview of the sampling and fielding recommendation and as Kathleen reviewed mentioned, there is a detailed document available on the AHRQ CAHPS website that provides the information that Kathleen has for us today and much more. So, please don’t feel that you need to absorb it all in one fell swoop here today There is support documentation available to you There are a couple of questions that have come up And we’re going to hold those just for a moment so that we can turn back to Chris Evensen at AIR to tell us a little bit about the analysis of results. So, Chris, I will turn it back over to you. And then, for those of you with burning questions, we will then open back up to a Q&A. So, Chris, over to you Chris Evensen: Okay, thank you, Stephanie. So, I’m only going to touch on really a couple of analysis topics, primarily scoring. How you score the survey and along with that, benchmarking And I’ll talk a little bit about quality improvement. There are a lot of different kinds of analyses you could do. For example, we conduct psychometric analyses to test the reliability and validity of the measures

Those have been pretty well established now. But other users might want to do those kind of analyses, especially with the shared decision making composite And there are, of course, other substantive analyses people might want to do where they look to see if there’re differences across different kinds of patients in terms of what their experiences are. For example, you might be looking for disparities in experience And let me start with basically how do you score the survey? And the way we score the survey and the way we recommend that users score the survey is to use the CAHPS macro version 4.1. That’s the currently version for the scoring. The macro’s an analysis program in SAS. It was developed by the CAHPS grantees It’s updated occasionally, where various new functionalities added or things are refined or corrected, if they’re – if users spot problems. So, it’s been updated several times throughout the years It’s specifically designed to analyze CAHPS data So, that’s one of the advantages of using it It’s available on the CAHPS website And the link is here It’s kind of a long link, but if you go to the CAHPS website, it’s not too difficult to find it And in addition to the actual programs, the macro itself, and then, some other supporting programs, there’s a fairly detailed instruction document that explains how to use the macro. I would not attempt to use it without familiarizing yourself with that manual, with that instruction manual The macro itself as a computer program is static and you don’t change it. But what you do is customize a control program that executes the macro. And the macro, it’s both standardized and flexible It’s standardized in the sense that it provides a standard way to do several things, including constructing composite scores out of the individual items That’s where you’re combining multiple items into a single composite measure Adjusting for case mix and estimating cancer center scores. Testing the significance of cancer center rankings. Now, both of these things assume that you have data from more than one what we call reporting unit, whether it be a cancer center or community oncology practice There’s no real kind of internal benchmarking that you would do if you only had data from one reporting unit It also lets you apply sampling where it’s appropriate. And it generates the scores in a variety of forms, including both data files and output that you can use to populate reports The flexibility comes in studying the analysis parameters in the control program. So, there’s several different options. For example, how you collapse a global rating or how you – whether you use equal item weighting in constructing the composites, or unequal item weighting, or what you specify as case mix adjustors And there’s a set of recommended adjustors, which I’ll show on another slide And so, there’re a lot of – you can apply various smoothing, if necessary. And these are fairly technical things Again, you’d have to be someone who is quite familiar with statistical analysis and programming in SAS to implement something like this. Case mix adjusters, again, the purpose of case mix adjustment is, it’s an effort to try to create a kind of level playing field What would your scores be if your patient mix looked like everyone else’s patient mix? And of course, it’s not a perfect adjustment. It’s not going to take care of everything that people see as important, but for the cancer survey, we recommended a set of adjusters that’re pretty standard in CAHPS. Includes age, education, the general health rating, and a mental health rating, and then, survey mode, and whether or not the respondents had help with the survey. And that’s a question on the survey. So, all these variables are available in the About You section, except survey mode, but you would know the survey mode if you were implementing a survey

So, benchmarking, again benchmarking – and I mentioned before, there’s an option to benchmark the data that you obtain through the cancer survey with the existing Clinician & Group Survey database. And I noticed someone had asked a question, do you think it’s a valid comparison? And in some ways, it’s problematic, because this is a survey about cancer care. You’re surveying your cancer patients. And the data that’s in that Clinician & Group database is a much broader patient population So, it’s a comparison or a type of benchmarking that can be done. I think it would have to be done carefully and you would have to make it clear what was being compared and the limitations to that comparison. But given sufficient data, meaning if you’re collecting data from a large number of cancer centers or practices, it’s possible to define benchmarks using the macro at different levels. Regional, or state level, or other geographic areas. As I mentioned, cancer centers or other types of cancer providers can compare themselves to the benchmark in the C&G database with limitations Now originally, the scoring was designed to roll up across the three treatment modalities So, if you were collecting data from patients in all of the three different treatment modalities, the original plan was that you would combine all those data, and you’d get one unit level score that combined radiation, medical oncology, and surgery In working with the sites, we discovered that many of them found it more useful to look at each treatment modality by itself. So, they wanted to see how they stacked up against the other sites with respect to radiation, for example, separately from the other two. So, the recommendation right now is if you’re going to roll the scores up across those treatment modalities, it’s to get – try to obtain 200 completed surveys per reporting unit to get reliable estimates. To generate scores separately by treatment modality, again, the recommendation is to get 200 completed surveys per modality. So, that means if you’ve got the three separate modalities, that you’re talking about 600 completed surveys, 200 within each of the modalities And that’s to be able to have enough data to be able to detect the difference among the reporting units, if you have multiple reporting units. Another type of analysis that users of the survey might be more interested in is quality improvement This is where you’re tracking your own scores over time. You collect data in multiple points in time You try to see if the scores change, perhaps in response to some kind of an intervention to improve the quality The macro is not designed to do this kind of comparison So, it’s something that users would have to do themselves. And there’s some different analyses across a range of complexity that could be done. And I just listed a couple here. For example, a difference-in-difference type of analysis for pre-post comparison. Maybe before and after an intervention of some kind. n analysis For multiple year trends, there’s a variety of time-series options, which we don’t really have time to get into here, but this requires a little more sophistication in analysis There’s no payment or accreditation model in consideration right now, but someday if there was, if the survey does start to be used for payment or accreditation, those types of models would probably incorporate both benchmarking and quality improvement, similar to the value-based purchasing program hospitals. This I just pure speculation on my part There’s no plan in place for doing this. But those two dimensions, benchmarking, that’s comparing yourselves to others. And then, quality improvement, comparing yourself now to yourself in the past. It would probably be two dimensions that would be included I wanted to thank you for your time I know that was a lot of kind of complex information in a short time, but now, we’ll have some time for questions, I think And we may be able to address some of your questions I’ll turn it back over to Stephanie. Stephanie Fry: Thanks so much. Thank you, Chris I appreciate that. We have questions ranging across a broad spectrum here. So, we’ll try to get through as many as we can

Again, in your web console, you have a question-and-answer box. And so, please feel free to type any questions you may have into that box Some of you may have asked these questions before Chris and Kathleen got through their full slide deck, but I think it warrants just mentioning quickly, a number of people have asked about how to manage administration of the Cancer Care Survey with other CMS-mandated surveys. And to just point you back to one of the notes that Kathleen provided for us, that there is the 51-day hold for surveys for the Cancer Care Surveys to go out. And that is in order to allow the CMS-mandated surveys, so specifically the HCAHPS survey and the OASCAHPS survey to get out in the field and to be returned before any Cancer Care Survey would go out. So, it is possible, depending on your sampling strategy, that some of these could overlap. And at this point, there isn’t any specific deduplication available across these two surveys. Rather just that the Cancer Care Survey would need to wait for 51 days before being fielded As a follow up to that, some other folks have asked about the administration modes. And again, I would point you to either the fielding document or back to Kathleen’s slides and just to remind you that there are several recommended modes of administration So, mail only, telephone only, or a mixed mode that includes mail and telephone, email and mail, or email and telephone. And so, there are a range of different guidance documents available to support you in your approach to those various modes There were also several questions related to that about response rate. And whether or not it really seems feasible to achieve a 40% response rate. And I’ll turn it over to Kathleen in just a moment here to talk about her experience with the field test. But I would certainly point out that with multiple modes of administration, you have that going for you in terms of increasing response rate. However much relies on the quality of your sampling frame and the quality of contact information that you have available for your patients Kathleen, I’ll turn it over to you to add some of your experience from the field test Kathleen Yost: Sure. I can speak to that question from both field tests. We achieved over 40%response rate in both of them I can specifically – well, I think that just on that note, we think of cancer patients as, especially if they’re in active treatment, as maybe being a sicker population. But I think the fact that we have observed response rates over 40% speaks to the salience of this survey to them That they really do want to share their care experiences with the cancer center where they received their care. So, we have seen that occur I can speak to mail only specifically and email with a mail follow up Those were tested most recently in the second field test. And we achieved greater than 40% rate with both of those. So, I think to the specific question of the likelihood of achieving a 40% response rate is fairly good. Again, I think patients are willing and interested in sharing this information with us. There are a couple that – telephone only, I think may be a little bit more challenging In our first field test, those tended to be lower But again, we don’t have recent data on that from the second field test. So, I think with multiple contacts and maybe mixed modes, I think the opportunity for achieving a 40% response rate with the survey is quite good Stephanie Fry: Thank you for that Kathleen, I’m going to keep you on the hook here. There’s a question specifically about sampling. And what about hematologic malignancy patients? Kathleen Yost: Right. So, those patients will be sampled if they received radiation therapy or drug therapy Surgery’s not a typical treatment modality for those types of cancer, but they could possibly be identified in a sampling frame for the drug or radiation therapy surveys We don’t specifically – if they’re receiving bone marrow transplants, that is a type of a procedure that’s currently not addressed in any of the sampling frames Or it really isn’t applicable with the drug therapy, radiation therapy, or surgery. But the patients themselves could report on their care experiences for radiation and drug therapy Stephanie Fry: Great. Thank you for that. There’s a question about availability of the surveys and including,

are they available to vendors? And the answer to that question is, yes, absolutely All CAHPS surveys are made publicly available free of charge. And so, you will find the three different versions in both English and Spanish on the AHRQ CAHPS website, available for download So, in addition to individuals who may be just curious or those who are fielding it on their own through their organization or institution, if there are vendors who are interested, they certainly can make use of the surveys there as well. And I understand some are well under way with that. With regard to the questionnaire itself, there was a question about patient perception of the effectiveness of the cancer care team. And I know people are just getting familiar with the survey. And Chris, I will turn it over to you to add in a moment here But the majority of the survey does ask about interaction with the cancer care team, knowing that there’s many different individuals who are interacting with patients And so, after asking about a number of different facets of those types of interactions with the cancer care team, there’s also a zero to ten rating question with regard to asking patients to rate their cancer surgery team Chris, I don’t’ know if you wanted to add anything further to that about the effectiveness of the cancer care team and gathering that perspective Chris Evensen: Well, I think that if – that kind of question comes close to asking the patient to evaluate the technical competence of the people that’re giving them treatment, which CAHPS tries to avoid that We did test some questions that the participating sites wanted to add that asked about if the cancer team gave a prognosis or some other types of questions about information like that being given to the patient. And they didn’t test very well. The reliability was quite low. And they tended to have pretty big ceiling effects. The questions weren’t all that great, because they weren’t really cognitively tested. I mean, they weren’t cognitively tested at all So, kind of some of the formative type research that would – that normally goes into questions hadn’t been done with those. So, they ended up not making it to the survey I don’t know if that answers the question or not Stephanie Fry: Thanks, Chris. I appreciate it And again, related, Chris, to the testing of the survey, there’s a question about whether patients who have just had cancer treatment have trouble responding to the question about their general health status Have any insight on that? Chris Evensen: I don’t know. In the sense of actually watching a patient try to answer that question, I can’t really speak to that. I do think, interestingly enough, a lot of patients tended to rate their general health pretty high, which I think was a little bit surprising. I don’t know if, Kathleen, if you have any input on that from – Kathleen Yost: Very anecdotally. So, as the paper survey done and the email surveys came in to the Mayo Clinic Survey Research Center, I did scan all of them for comments that may have been concerning and needed to be brought to the attention of the collaborating sites. So, I did notice just a handful Again, this is very anecdotal. Not scientific at all, but some people would write in the margin They’d mark an answer, but then they’d write, I have cancer. They were appreciating that the cancer was affecting their general health status, but able to – I didn’t see any missing data problems for that particular question, if that gets at the answer for that. So, people are able to reflect on their general health status, even while they’re in active treatment Stephanie Fry: Thanks, Chris and Kathleen We have a few more questions popping up about use of the Cancer Care Survey and how that overlaps with ACO CAHPS or if it’s anticipated that it will be required for any other type of accreditation or participation in various programs. And the answer to that is, at this point, it isn’t required for accreditation that we’re aware of anywhere. It’s a brand new

survey. So, how it finds its place in the world I think is yet to be determined, but we know that there’s certainly a lot of excitement about this very specific survey out there Going back to the content that Ashley walked us through in terms of the need forthis survey right now So, I think there isn’t a whole lot more information about what CMS may do and how it may overlap with other surveys that’re out there except for the couple of notes that we’ve made with regard to the fielding of this particular survey Kathleen, I’m going to turn it back to you for a question that we have about the response rate that you were able to achieve in the field tests. And if you can say anything about what proportion of your sample was considered medically underserved or low SES. And do you have any thoughts around how that will impact your response rate or choice of modality? I don’t know if it’s – Kathleen Yost: Yeah, I don’t off the top of my head without looking at the disposition report. I don’t think we had a lot of information that would answer that question with respect to sort of vulnerable populations or underserved populations. We did include a Spanish language preference so that we could test the Spanish language translation of the survey. We didn’t have a large sample of those individuals And other than the information that was in the about you section about education, we really didn’t have other information that we could use to maybe stratify whether they were vulnerable or not or underserved. Sorry to push this back to Chris, if there was other information in the survey that would help answer that question, but I think the unfortunate answer is that we don’t have enough information at this time to report on that Chris Evensen: Yeah, I agree. We don’t really have that kind of information I mean, I think that – I mean, in this last field test, Kathleen, wasn’t the – I mean, we were basically taking a census of patients. Is that right? Kathleen Yost: We were. To meet our sample size goal, we rarely had the luxury of taking a random sample. So, we weretaking censuses of eligible populations. Just in the geographic area, it’s going to be a mix of people. But I think we just didn’t have a lot of ancillary data on their socioeconomic status that would allow us to evaluate the scores or response rates based on those covariants Stephanie Fry: Great. Thank you A couple more questions around the survey itself. And someone had asked, is the six-month period fixed? And what if treatment is already completed in that time? And the answer is yes, if you want for it to be the CAHPS Cancer Care Survey and to use the CAHPS name, you must use it as is. That said, you can add the supplemental items either developed specifically for this survey or other supplemental items of your choosing. But in terms of the core survey, you have to leave it as is in order for it to be a CAHPS survey. And the look back period of six months is indeed the look back period and syncs up with several of the other CAHPS surveys. Kathleen or Chris, I don’t know if you want to say anything more about the six-month look back period and your experience in fielding it in that way Chris Evensen: Again, well, we used to – we actually in the first field test used a three-month look back window. One of the revisions we made was to – in aligning it with the existing CAHPS surveys was to go to six months I don’t know if we have really any evidence that the six-month window didn’t work I don’t know if any of your anecdotal comment period is fairly long. And so, it’s possible that once you start doing review, Kathleen, you saw anything about that Kathleen Yost: So, just another reason why the recall was extended from three months to six months is that by the time – because the fielding reminder and switching mode, you could be in the field for up to 42 days, for example, or longer. And so then, the person sort of moves out of that window. And so, they’re saying no to the questions because they haven’t received that care in the past three months. So, that was one of the reasons for expanding it to six months is to capture –

to allow them to reflect on a period of care Even if they’ve moved out of that active treatment, they can still reflect on that six-month period. And we wanted the surveys to be parallel. So, we had to pick a time frame that would sort of work for all threemodalities, given that they’re very different. So, surgery’s very intensive and more brief. Radiation is sort of in the middle. And chemotherapy could be quite long. So, trying to find a time frame that would work for all three was another issue that we needed to arrive at and six months seemed to work for all three of those treatment modalities And again, the fact that it also now corresponds with other CAHPS surveys is another benefit of standardizing the way the questions are asked Stephanie Fry: Thanks, Kathleen. And I know that you mentioned this as you went through your sampling section, but I wondered if you could just say one more piece We’ve still got a few questions about what to do if a patient had two treatments in the time period If you could just say a little bit about that again. Kathleen Yost: Sure So, I think the approach to take would be to create separate sampling frames. So, for whatever types of modalities that’re offered at a cancer center that they want to measure the care experience, they would create separate sampling frames for each of those modalities. And then, look across those modalities for duplicates. So, if a person did receive surgery and radiation therapy in that look back period, then the cancer center just needs to come up with some a priori rules for how they want to manage those duplicates So, it’s really up to them. I think if you want to – ideally, if it’s random, then you’re not allowing sort of secular trends to dictate who is selected. So, for example, if it’s more likely for a patient to receive surgery followed by radiation, and you’re always putting the – always selecting the person for radiation, that might be a different kind of experience than allowing that type of a person to be randomly assigned to either the surgery or the radiation frame That’s why the suggestion is if it’s possible and fits within your sampling goals, sample size goals, to maybe randomly assign to one of those modalities to receive one of those surveys That said, sample size goals are – Chris mentioned that the goal is to have either 200 completions per modality or 200 completions across the cancer center. That may dictate what to do and how to manage those duplicates And you may, by necessity, have to force – manually put a person in the smaller of the frames in order to support that reliability. So again, if you create separate sampling frames where each person is within a frame only one time, there’s no duplicates within a frame, then look across the frames for duplicates and have some, again, a priori rule for how to manage those so that at the end of the sampling, a person is only receiving one survey for care If they are in a situation where they have received multiple types of care in the past six months, they’re only getting one survey about that care. I hope that clarified. I know it’s fairly complicated, and I stumbled a little bit, but I think the overall – just thinking the intent of the survey is we recognize that people can receive multiple types of care, but really we don’t want to burden them with more than one survey. So, just have some process for selecting them to receive one survey Stephanie Fry: Thank you, Kathleen. And I think we’ve got time for maybe just one more question. So, I want to touch just briefly on the concept of CAHPS databases There’s a couple of different questions that relate to CAHPS databases and getting results from the Cancer Care Survey. So, to begin with, just as a reminder, for the Clinician & Group Survey and for the Health Plan Survey, AHRQ maintains comparative databases. So, as you field surveys, you can submit the data from your CAHPS survey and receive back comparative results so you can see how you compare to others who have fielded that same survey. And so, related to that, there’s some questions about getting comparative data from the Cancer Care Survey. And Caren, I don’t know if you want to just have the final word about any plans for a comparative database that would compare the Cancer Care Survey Caren Ginsberg: We’re really interested in what you all think about this survey, and how you would use it, and the tools that you need to move forward with whatever

future plans you have. And we don’t have a plan right now, but NCI and AHRQ are committed to this partnership And in order to make the partnership work and move forward, we do need to hear from you about whether a database is helpful or not and we can see how we can accommodate We don’t have plans right now, but we do want to know if you’re going to use the survey, how you might use it, what your concerns are, what a database would mean to you And based on your comments, I think I’d like to continue the discussion with NCI. I think that’s about all I could say right now. We’re really hoping to hear from you at the email address that Stephanie will show you in a couple of slides Stephanie Fry: Perfect. Thank you so much. So, to all of our presenters, a huge thanks for giving this detailed overview of the Cancer Care Survey Well know that people may have additional questions or we may not have had time to get to the specific question that you have So, we are asking you to complete an evaluation at the end of this webcast to let us learn about what we did well and what we need to improve on with these webcasts. And we also want to invite you to follow up with us regarding any questions that you may have about this webcast, about the survey itself, to tell us a little bit about how you think you may use the new CAHPS Cancer Care Survey, how you might field it, or how you might use results. We’re very curious to hear from you and hear how we can support you in using this brand new survey. So, there’s both a telephone number and an email address to reach out to us. And again, the website for the AHRQ CAHPS surveys so that you can go and peruse the information that’s there and available to you So, to all of our speakers, a big thank you And to all of our participants, thank you for joining us today and learning about the new CAHPS Cancer Care Survey. I wish you all a very wonderful afternoon