NCCN’s Bob Carlson talks about the day he stormed out of a meeting with transplanters

In 1995, a group of doctors who advocated treating breast cancer with high dose chemotherapy with bone marrow transplantation made an attempt to include that highly toxic procedure in the guidelines of the nascent National Comprehensive Cancer Network.

The believers asked to argue their case before the guideline-writing committee, and a meeting took place at a small conference room at the Hilton at O’Hare International Airport sometime in 1995, a bit before NCCN’s first annual conference.

“So, in 1995, when there were seven of us, originally, on the breast cancer guidelines panel, and we put together the first version of the guidelines that I don’t believe were ever published outside of NCCN. And they did not include bone marrow transplantation for breast cancer in any specific place,” Robert W. Carlson, then a Stanford oncologist and chair of the breast cancer guidelines panel, told me in an extensive interview recently.

“And that draft was reviewed by members, then, of the board of directors, some of whom thought that the use of high-dose therapy with transplant in breast cancer should have been included in the guideline itself. They were convinced that the data was strong.”

Rodger Winn, who at that time ran the guidelines program, arranged a meeting where the transplanters made a case, based on tumor registry data and non-randomized trials that had used historical controls. Carlson had seen many versions of that presentation before—as had many others. And, like many others, he was determined to await completion of randomized trials that were being performed by NCI.

The discussion between true believers and skeptics focused on the usual subjects: selection bias, the limitation of historical controls.

“It was a spirited, let’s say, discussion, with substantial disagreement; sometimes got heated,” said Carlson, now the NCCN CEO. “At one point in time, I looked at the person who was leading the transplant presentation, and just looked at him and said, ‘If that’s really what you think, then we’re done. We’re done.’

“And I collected my stuff and headed for the door. And Rodger, bless his little soul, ran over to the door, literally locked the door, and turned around and said, ‘You can’t leave, Bob. You’ve got to stay.’”

Carlson and the rest of the NCCN guidelines panel held their ground and transplanters didn’t get their way. But what if they had prevailed? Would NCCN have been able to establish its scientific credibility?  In our conversation, Carlson and I hashed out these what-ifs.

NCCN, an organization that held its 25th annual conference last week, was founded with the goal of promulgating practice guidelines based on the standards of care at elite academic institutions in the U.S. (The Cancer Letter, March 15, 1996).

• The first story in the four-part series on NCCN appeared in The Cancer Letter and the Cancer History Project three weeks ago (The Cancer Letter, March 5, 2021).
• The following week, we published a conversation with Robert C. Young, one of the founders of NCCN (The Cancer Letter, March 12, 2021).
• Last week, we ran an interview with Bill McGivney, who succeeded Bruce Ross as NCCN’s CEO.

A video recording of the conversation with Carlson is posted here.

 

A transcript follows:

Paul Goldberg: Thank you for agreeing to talk with me. We’re going to try to go through the creation of NCCN, show how it began, how you come into it, and the various ways you come into it, and where it is now. But the objective is to show the progression. So, here’s an organization that starts out of, really, nothing—an idea. How do you come into it?

Robert W. Carlson: Well, I think it was spring or early summer of 1995.

I was invited by the board then to chair the breast cancer treatment guidelines committee, which was one of the original 10 guidelines that were starting to be developed in 1995. I’d been nominated at that time by Charlotte Jacobs, who was initially one of my faculty mentors, and ultimately one of my colleagues at Stanford in medical oncology.

Charlotte was the Stanford representative to the NCCN board at the time, and she had proposed me to bring that group together.

 

PG: Let’s go back to February 1996… Florida. That’s kind of where I run into you. The high priority trials [of high-dose chemotherapy with bone marrow transplantation in breast cancer] are still being run by NCI.

They’re trying to get women randomized [but] everybody really wants to transplant, because it’s “the cure.” And there you are, and there are the transplanters. And there’s a closed door, and I never got behind that closed door. I only heard what people told me, but maybe you could tell us what actually happened behind that closed door in Florida. Was it Tampa? [The first meeting was in Ft. Lauderdale.]

RWC: I think it actually took place a little bit before the meetings in Florida. So, in 1995, when there were seven of us, originally, on the breast cancer guidelines panel, and we put together the first version of the guidelines that I don’t believe were ever published outside of NCCN.

And they did not include bone marrow transplantation for breast cancer in any specific place. And that draft was reviewed by members, then, of the board of directors, some of whom thought that the use of high-dose therapy with transplant in breast cancer should have been included in the guideline itself. They were convinced that the data was strong.

And they went to Rodger Winn, who at that time was overseeing the guidelines program at NCCN, and said, “We need to meet with the breast cancer panel and make our case.”

And so, Rodger asked me if we would be willing to meet the evening before the actual breast cancer guidelines panel with the transplanters, a select group of us with the transplanters.

And that was actually a meeting that took place at the Chicago O’Hare Hilton, a hotel that many of us know very well. And Rodger was present at that meeting, and there were two or three representatives from the transplant community from NCCN institutions.

And there were, I think, three or four of us from the breast cancer panel, primarily medical oncologists, who were there for the discussion. And the people who were from the transplant community laid out a very good presentation based upon tumor registry data; national data, but tumor registry data, from purely non-randomized trials that had used historical controls, to argue that transplant should be put on the breast cancer guidelines.

 

PG: This was Bill Peters spearheading it; wasn’t he?

RWC: Bill Peters was not in the room. He was certainly active at that time nationally as a strong proponent of the use of transplantation. And when you look at that data now, you look at that data then, the real-world evidence was really strong and supportive of transplantation.

Those of us on the breast cancer panel, being more disease-related experts rather than modality experts, were concerned that the appearance of superiority of transplantation in the non-randomized trials was based upon unconscious, [and] sometimes probably conscious, evaluation by the treating physicians, that they were essentially selecting a healthy cohort, that someone who looked like they could handle a therapy as strong or as aggressive as transplantation was a healthy cohort patient, and we should expect them to do better.

And so, it became a discussion on are these results the result of selection bias and using historical controls that were not selected in the same way? Or was this really a difference in the effectiveness of the therapies? It was a spirited, let’s say, discussion, with substantial disagreement; sometimes got heated.

At one point in time, I looked at the person who was leading the transplant presentation, and just looked at him and said, “If that’s really what you think, then we’re done. We’re done.”

And I collected my stuff and headed for the door. And Rodger, bless his little soul, ran over to the door, literally locked the door, and turned around and said, “You can’t leave, Bob. You’ve got to stay.”

 

PG: Wow. And you would be leaving, because there was just no point continuing to talk.

RWC: Yeah. It digressed into a non-scientific discussion, and if we’re going to do this based upon opinion, then that’s not the way you get on our breast cancer guideline.

 

PG: How many people are in the room?

RWC: I think there were probably seven or eight people in the room.

 

PG: Wow. And that’s in that Skyline lounge at-

RWC: This was in one of the conference rooms, it was a conference room at the O’Hare Hilton.

And it was a memorable experience. And I know that everyone who was in the room, whom I’ve ever talked with afterwards remembers the whole interaction very clearly.

And it’s interesting, time proved us correct in terms of not placing transplant on the main part of the guideline. What we did agree to do, and I think it was an appropriate response, was we ended up saying, “Well, if the disagreement is really this strong, we’ve got to have randomized trials, and we’ve got to get them done.”

And so, we footnoted the breast cancer guideline in the circumstance of the adjuvant setting with 10 or more positive nodes and in women with the first evidence of metastatic disease, in those two places, that those were places where participation in randomized clinical trials of bone marrow transplantation was especially important.

 

PG: Was Bezwoda data quoted [by transplanters at the Chicago meeting], do you think?

RWC: Bezwoda data wasn’t quoted, because Bezwoda data came out in late 1996, I believe it was September of 1996, which was actually after we had published, or released, the second iteration of the guidelines and that had that footnote in it.

But I do remember vividly the annual conference that you are remembering, where the presentation was made, and afterwards I remember some arrows coming at me from people in the audience, and talking about the Bezwoda data.

And, of course, the Bezwoda data ultimately was proven to be fraudulent. I actually believe that there was evidence in the Bezwoda paper itself, that fraud was being committed.

It was a paper that I used to give to trainees, to say, find the nine errors in this manuscript. And there were errors of logic, there were distribution of numbers of sites of metastatic disease that could not be the correct distribution, based upon the number of patients that they had. There was disagreement between a table and the manuscript about whether there were any toxic deaths.

In one place, there were two, and in another place there were none.

I actually [met with] a statistician colleague of mine at Stanford at the time and said, “Bill, can we prove that this paper is fraudulent based upon this data.” And his ultimate response, after he spent a week studying that, was, it’s either the sloppiest paper he’d ever seen, or it was fraudulent. And he couldn’t be sure which was the correct answer.

 

PG: This is before we knew?

RWC: This is before we knew.

 

PG: That’s astounding. Do you have that paper or that data or anything, any materials from that, because we’d love to post that… Because we should just do a takeout on the entire bone marrow transplantation fiasco.

RWC: So, Paul, if you go currently to JCO, which is where that paper was published, they have removed it from the archives.

 

PG: But your paper. Your work on that and that paper.

RWC: So, the work that I did on the paper, I never published. I presented it verbally from time to time, but it was never formally published.

 

PG: Dick Rettig wrote a book called False Hope, which he wrote in part from our stuff, because I was covering all of this in real time. I’ve asked Dick if we could bring it back and publish it on the Cancer History Project, and that’s when we should probably revisit this whole issue.

And if you don’t mind, just go back to it and look at the documents that are in there, because he really got into it. The book’s called False Hope. Now it’s kind of a difficult to find a copy. So, what would have happened? I mean, this is a big what if—what would have happened to NCCN if the transplanters had won?

I would propose that [NCCN] would have lost its credibility from the outset.

RWC: Well, I’ve heard you previously make that comment, and I’ve heard others, who have made that comment.

I do have to tell you that in real time, as it was actually happening, that thought never even occurred to me, that this was a decision that could make or break NCCN. And we’ll never know whether it would have broken NCCN, had we made the opposite decision.

But I think it was, in retrospect, a very prominent way where an NCCN panel made the decision that we are going to follow science. We are going to follow good clinical practices in terms of evaluating studies, and moving forward. And show us the data, we’ll change the guideline, but show us data that is meaningful, that’s important, that really addresses the issue in a straightforward fashion.

And if it’s an important question and we don’t have that data, we will support you in trying to recruit patients to those trials. And we also know that a number of the NCCN institutions were in the forefront of the bone marrow transplant scenario, overall, not just in breast cancer—just overall. But breast cancer was rapidly becoming the most commonly treated malignancy there.

And it hurt the member institutions financially that the panel said, “transplantation should not be done routinely. It should be done as part of the clinical trials.”

 

PG: We’re talking about a year before the high priority trials data are reported, I think. Correct me if I’m off the timeline here, but it’s not very long, it’s close to being done.

RWC: I think it was a few years later when the ASCO meeting happened, where the five, I think it was five, transplant studies were reported and all of them were negative trials. No difference in outcomes, except for the Bezwoda trial. The second Bezwoda trial.

 

PG: I remember sitting in the front row. It was pretty amazing watching that big presentation [at ASCO] in Atlanta (The Cancer Letter, April 16, 1999). He looked so happy presenting this garbage.

RWC: Yeah. You know, how somebody can do that… I don’t know. I was, actually, on the program committee for the breast cancer section of the ASCO meeting that year, and I remember-

The Bezwoda abstract, when it was first submitted to  ASCO, actually had a single author on it. Ultimately, at the time he presented it formally, there were multiple authors on it.

And a number of us thought that it was just unusual that a single high-priority study like that would have a single author on it… And it made us skeptical of the study, really from the get-go.

And, of course, the ultimate audit that was overseen by NCI, showed our skepticism was well founded. [The Cancer Letter, Feb. 11, 2000].

 

PG: Yeah. That was Ray Weiss.

RWC: Yes. Ray and Richard Theriault, I think.

Roy Beveridge was there as well. They went and looked at the data, and found that there was nothing. It was just made up. He just did what he wanted.

Theriault was in that group, too. Yeah.

 

PG: I should actually go back to them, but do a separate… I don’t know if we’ve veered off or we haven’t. We actually haven’t. We’re still on the subject, I declare.

I stay with my opinion that you saved NCCN’s credibility with that, but can we talk about expert-based guidelines? NCCN never tried to do evidence-based [comprehensive] guidelines. What was your thinking about them as a tool, and what they can do and what they can’t do?

RWC: Well, when we first published the first series of NCCN guidelines, there was a lot of criticism about so-called cookbook medicine: “You’re telling us what to do. Patients are unique. You can’t show all the different scenarios on a guideline.”

Our take on it was quite different, and it was that there’s so much literature published every year across all the different oncology space that how can we expect someone who sees multiple cancer types to stay current?

We really need to assist them in delivering care. And we aren’t trying to micromanage. We are trying to give them a tool that will actually make their care better. The analogy that I come up with today is like a GPS.

I remember the time when we used maps, paper maps, in the car, and almost ran off the road all the time, trying to read the map, and so on. Now, we have a GPS that gives us a suggested route. Sometimes it gives us multiple different suggestions, but we still have to look for deer running across the road.

We have to avoid accidents, do defensive driving. We have to be sure there’s gas in the tank. We have to do a lot of different tasks that make us a bad driver or a good driver. And I think that guidelines in the oncology space are similar. They assist you in getting to where you are going, but the individual physician, the individual nurse, the social worker, the team of people taking care of the individual still matter to an incredible degree.

So, we’re not really taking ownership away from those people, of the quality of care. We are trying to facilitate, we are trying to give them a tool that will really assist them in doing that. I think that’s especially important for the provider that takes care of more than one or two different diseases.

There are literally thousands of articles published every year that are important articles in the oncology space now. No one physician could ever keep up with that literature. And if they tried to, they wouldn’t be able to see patients.

 

PG: And looking back, before NCCN, you needed to see one of the luminaries of the field. But here, it’s more of a standardization of care. But it did exclude things like bone marrow transportation, which was performed by luminaries like Bill Peters, for example, at Duke.

So, there’s still a lot that was accomplished through that, but let’s go back to NCCN being started. When Bruce Ross was the CEO, the money was coming in through memberships that cancer centers started out with.

At one point there was also some thought about contracting. But very soon thereafter, after Bill McGivney came in,  [NCCN] started seeing [other] revenues—and t the real growth in the guidelines, and commercializing the guidelines. 

So, where are you in this? Are you in Stanford, not paying attention? Are you paying attention? What’s your role in this?

I don’t know the answer to this…

RWC: Very early on, I was not a member of the board of directors. I was not in what I would call, the senior leadership of the organization, but definitely was involved in the Breast Cancer Guidelines Panel.

Many of those issues impacted the Breast Cancer Guidelines Panel. Funding does; where does it come from? Potential financial conflicts of interest… Are we going to charge or not charge for access to the guidelines? I mean, all of those things were issues that those of us involved in developing the guidelines thought were important.

And, fortunately, I think that the right decisions were made at that time. There were some really bold decisions made early on. One of which was that the guidelines would be published publicly and accessible free of cost for non-commercial use.

Now, let’s think about what the implications of that are. So, you have the member institutions who are paying dues to develop guidelines. They are contributing the time and effort, and expertise of their faculty and not a trivial amount of time or expertise.

So, tens of thousands of hours at this point, of time to develop the guidelines. They’re having their faculty as a group, put in the guidelines, the secret sauce of the premier institutions… How did the premier institutions treat these groups of patients and how did they go about doing it?

And then they post them on their website for access, free of charge, to their competitors.

It’s a really wonderful story, I think, and capability of NCCN, but that’s how they approached this. It’s really about improving the standard of care and not about funds, not about trying to make more money for our institutions.

 

PG: Well, but also it helped that the competitors were falling down and dying by the side of the road, because the [drug] compendia [used for reimbursement] were dying. There were two drug compendia [that stopped publication], literally, very quickly after Bill took over. And then he started seeing that this is how oncology starts regulating itself in a way; right?

RWC: Yes. Yes.

 

PG: And I’m sure patients are using them; right? So that improves the quality of care. Does that drive people toward NCCN institutions, you think?

RWC: You mean patients or faculty?

 

PG: Well, I was thinking patients, but if you want to talk about faculty…

RWC: The faculty one, I think is the easiest one to deal with. And the faculty one is that many of our member institutions use the potential for participation in NCCN activities as a recruitment tool.

It’s not unusual that faculty will say, “Well, you’re not a member institution of NCCN, but if you promise me, if I come, that I can assist you in becoming a member institution …”

That’s not infrequent.

Or, alternatively, at one of our member institutions who’s recruiting a faculty member, they’ll say, “Come and join our faculty, and we will put you on an NCCN guidelines committee. They’re incredibly fun and they’re very prestigious, and just professionally very good and rewarding.”

Patients, in terms of getting to NCCN institutions, I think, actually have a number of barriers.

One is restricted panels, that some insurance companies will restrict the institutions or practices that they will cover with insurance.

That sometimes works for NCCN institutions and sometimes works against them. I think that most patients though, don’t really understand the role of NCCN and what we do. Within the oncology space, if you mention NCCN, everybody knows who we are and what we do.

If you go to the person on the street and say, “What’s NCCN?” very few will know.

So, I think that in terms of driving patients, by patient initiative, that at the current time and historically, the NCCN designation has had a modest impact.

Now, that’s starting to change, and it’s starting to change because we now have a library that’s up to more than 50 of our guidelines that not only have a professional component, but they have a patient component. There’s a patient guideline that’s actually written in language that patients can understand.

It’s content that’s derived entirely from the professional guidelines, but it’s written in language that patients can understand. There are graphics associated with it, and so forth. And the readership of those guidelines has just skyrocketed.

We’re over a couple million downloads of those guidelines now, per year. There are also print copies. You can get it on Amazon, and so forth. So, I think that the patients are beginning to understand who we are and the role that we actually have.

 

PG: So, you probably are driving patients toward NCCN institutions, because a couple of million downloads is not nothing.

RWC: Except that those downloads can be used in working with their local physicians also. Some of the feedback we get is that, “I took the patient guideline in to my doctor, to talk with him about it. He said, ‘Oh, I use the professional version.’”

And it’s really, I think, helpful in that circumstance, because the patient, if they read our guidelines before they see their provider, and their provider is using the professional guidelines, there’s a synchronicity where they realize where the major decisions are that need to be made.

There’s general agreement about what the options of therapy are, and the patient has some background information that they’ve been able to think about before they are even talking with their physician about what options they want to move forward.

 

PG: Well, there’s also, going back, the beginning [when there are] mostly the freestanding cancer centers that are NCCN.

Then that becomes more of a matrix, centers coming in as well. And then the money really switches over from money for licensing of the guidelines, and use of the guidelines, and payment. So, it’s a very interesting…

It’s not an organization that Joe Simone, Bob Young and Bruce Ross had in mind. Certainly not even in the first iteration, it’s completely different.

RWC: I think that none of us envisioned what NCCN would be 25 years later. But I also think that the founders would be proud of where we’ve gotten. I talked with this, with Bob Young about this, and he certainly is proud of what the organization has achieved.

He had no small part in setting it on its original course. And I know Joe Simone, before his death last month, was also very proud of what we had accomplished with NCCN.

It isn’t necessarily where we would’ve predicted it would be, but I think people are really happy with it.

 

PG: Absolutely. I was just thinking about how it evolved because the field evolved, the world changed.

RWC: Yes.

 

PG: There was also this very interesting third item of business for one of the items, or three things really, one of which was guidelines. The other was contracting, maybe, but that fell by the wayside.

But the item of business that was fascinating to me was always trying to demonstrate to payers that cancer centers produce a better value than community oncology. That is something that did not seem to ever materialize, because there’s really, I haven’t seen any data that shows that academic oncology is any better or worse than community oncology.

RWC: There is some data that would suggest that, but it’s relatively limited data. But what the data does show is that therapy delivered, concordant with the NCCN guidelines has better outcomes than care not delivered in concordance with NCCN guidelines.

There was an interesting study that was performed in conjunction with United Healthcare and NCCN that looked at an interactive prior authorization system in delivering systemic therapy for cancer done in the State of Florida.

Over a year, what they found was that if you had a prior authorization system that forced concordance with systemic therapies that were listed in the NCCN guidelines, that in the State of Florida, the drug spend went down by 10% during a year where the drug spend in the rest of the country actually went up by 10%.

It was about a 20% relative cost savings.

tThere have been some comparisons of guideline-concordant care versus not. Those that look at disease-related endpoints always show, either no difference in outcome, or, usually, they show substantial improvements in outcome where survival is an endpoint. Not just toxicity or whatever, but survival as an endpoint.

 

PG: You’re talking about the studies—like the ones that Don Berry goes completely nuts when he sees it, the kind of study where you look at survival by disease, and then [compare] academic versus community. That kind of stuff..

RWC: Well, this is guideline concordance versus not.

So, it could be in the same community, same state, but it’s guidelines-concordant care versus not. And in the current environment that we’re in, current social awareness that we have, there are a number of studies that show that, for instance, racial and ethnic minority patients less frequently get guidelines-concordant care than non-Hispanic whites.

If the ethnic minority and racial minority patients receive guideline-concordant care, most of the disadvantage and outcome differences between that racial and ethnic group and the non-Hispanic whites goes away.

So, we do believe that one of the really important ways for minimizing disparities of outcomes in healthcare, in the oncology space, is to work towards everyone getting guideline-concordant care.

 

PG: Well, you’ve been running NCCN for eight years. What prompted you to change the course of your career and come to-

RWC: I had been at Stanford for 40 out of the 42 years, before coming to NCCN, and I loved my job at Stanford. I also got to the point where my mentor there, Frank Stockdale, and I had built the breast cancer program at Stanford to the point where, when I looked around me, there were not just the two of us. There were dozens of us in the space. I realized that I could go away and the program would do just fine. There were a couple of rising stars within the medical oncology group in breast cancer, and it was clear that the leadership of the breast cancer program, going forward, was going to be in good hands.

And then, at the same time, I got to the point where I wondered… How can I become a better professional? What can I do differently that would really make me happy again, would reinvigorate my energy professionally?

Those three aha moments independently happened at the time when Bill McGivney had just left NCCN. I had been involved with NCCN so much in my career, was so devoted to the organization professionally, personally, emotionally, that it was just an opportunity that when all was said and done, I couldn’t pass by.

I was uncertain whether or not I would be able to be a good CEO at NCCN. I’d never done a position like that before, but people that knew me thought I could do it and do it well. It’s been such a wonderful organization to work for. It’s such a fantastic group of people, and the mission is so solid. It’s just so solid. So, it was a good move.

 

PG: [NCCN] is massive. Not in terms of only its influence, but also in terms of the money that is involved. It’s what’s? Third after AACR. I mean, there’s ASCO, AACR and NCCN in terms of … I just looked at the 990s today—because I look at them every 10 to 15 years—-whether I need to or not. But it’s a pretty massive organization.

RWC: We’ve done well. We’re now at about 130 employees. So, we’re a reasonable sized organization, but we’re not enormous.

But I think we do have a huge impact, and I think that we don’t make decisions at NCCN, based upon funding. Obviously, we have to float our boat, otherwise, we can’t survive, but we really have been an organization that has been truly mission-driven. We operate on a formal strategic plan that is developed with the board of directors, senior leadership and NCCN, and all, literally all of the NCCN staff.

I think one of the things that really made a difference for us over the last eight years has been, we have a strategic plan that everybody is participating in. We’re all rowing in the same direction, and it really provides a harmonization between the staff and the board of directors that I think is, I won’t say it’s unique, but I’ll say it’s almost unique, among the organizations that I participated in previously.

 

PG: What’s the most interesting thing you’ve done in the past eight years at NCCN? What’s been the most challenging? And also, where do you see it going next?

RWC: If I had to select a single accomplishment in eight years, it would be developing with others, the NCCN Harmonized Guidelines for Sub-Saharan Africa.

At the current time, we have about 1.4 million registered users on our website. Almost half of those registered users come from outside the United States.

Last year, there were 11.7 million copies of our guidelines downloaded. Again, about 40 to 45% of those downloads actually go outside the United States. So, we take very seriously the role of the NCCN and our guidelines, not only in the U.S. but now globally, because we’re impacting cancer care globally now.

We try to develop content that’s useful in every place around the world that our content is used. The American Cancer Society, the Clinton Health Access Initiative, the IBM Foundation and NCCN came together now, about three and a half years ago, to work on developing systems to facilitate quality cancer care in Sub-Saharan Africa.

We did this in partnership with what’s become known as the African Cancer Coalition, which is a group now probably 120, 130 oncology providers in Sub-Saharan Africa, but also including administrators, representatives from the ministries of health, and so forth. And one of the major activities of that group has been to take the NCCN guidelines, to work with experts in sub-Saharan Africa to modify those guidelines for the reality that they practice in in sub-Saharan Africa.

So, limited drugs that are available. Sometimes no or limited radiation therapy is available. They have few dedicated surgical oncologists and so forth. So, we modify the guidelines. We harmonize them for use in sub-Saharan Africa.

I think as of today, we have nearly 50 of our guidelines that have been harmonized with the experts in sub-Saharan Africa. And in addition to that, there are programs for steeply discounted purchasing of oncology agents. There are educational programs for pharmacists and oncology nurses.

There’s a whole system of things that are in place in this activity, to try and improve the quality of cancer care in Africa. And we believe that this whole effort that’s really led by our expert colleagues in sub-Saharan Africa, who are incredibly bright, incredibly well versed in oncology care and dedicated to oncology care, that it’s transforming the quality of oncology care in sub-Saharan Africa.

 

PG: That’s fascinating to see. There was originally some concern about giving toxicity without, really, the efficacy. When you cut something back down, if you can’t do three cycles, do two kind of thing. Or maintaining things refrigerated. That hasn’t materialized?

RWC: Those are all issues in Africa, especially in sub-Saharan Africa. Storage, just acquisition of drugs. Things as simple as transportation to the healthcare clinics is an issue. I mean, I remember first time I went to sub-Saharan Africa.

I spent a couple of wonderful weeks in Ghana, and in their cancer care clinics, you’d walk in in the morning and the waiting room would be literally packed with people who would wait in line to get care.

And by the end of the day, that room would be empty. It was, why don’t we have appointment times? And the response was, the transportation is so unreliable that people can’t make appointments. So, they come in when they can, and they wait.

Some of the simple realities of care that we just don’t consider very often anyway, in the United States are every-day, every-patient issues in that environment.

But the other thing about the efforts in places like sub-Saharan Africa, and we have similar efforts elsewhere in the world, is that what we’re really trying to do is to optimize the care that can be delivered in the reality of the situation that these providers are practicing.

With some of our guidelines being tiered and so forth, they can also be aspirational. If we had this radiation therapy equipment available, we could also do this, and this would be the number of patients, the outcomes that would be achieved with it and so forth.

It also provides a roadmap for health ministries, for individual cancer centers, individual providers within the resource constrained areas of the world.

 

PG: That’s fascinating. And that is totally not what was on the original list of goals, of NCCN.

RWC: It wasn’t.

 

PG: If they could have ever thought of and are very … Bob Young is very happy about this, I’m sure. Joe [Simone] would’ve been totally delighted, and was. So, that’s all fascinating. What happens next? What’s the next step for NCCN?

RWC: Well, we are always increasing our guideline coverage. We’re now, for adults, into the rare diseases, rare cancers. And we, a year or so ago, re-initiated an effort for pediatric cancers as well.

So, we can expect the library of guidelines to continue to expand. But we also have a lot of efforts to try to better address the needs of excluded and disadvantaged populations, and many of those exist solidly in the United States.

So, the racial and ethnic minorities, the impoverished patients, geographically limited patients, people who are in rural areas that have to drive hundreds of miles to get oncology care, sexual and gender minorities that are distrustful of the oncology healthcare system, often with good reason.

We’re trying to develop systems and policies that are exportable to oncology care in the United States, to try to minimize those disadvantages and minimize the number of people who are excluded from good oncology care in the U.S.

And  we touched also on the sub-Saharan Africa initiative. We have other initiatives around the world, and we’re trying to expand those initiatives in a really collaborative way, with the providers in different regions of the world.

We have initiatives in Asia. We have initiatives in South America. We have initiatives in Africa. And we have initiatives in Europe. And we’re all working together to figure out how can we make each other better and more useful? So, we will certainly see those efforts expand in the future.

 

PG: What about real-world evidence? Big Data, that kind of thing?

RWC: Our panels, in developing guidelines and recommendations, use the best data that we have. Sometimes the best data is the classic, placebo-controlled, randomized, large clinical trial, and that’s wonderful.

There are other situations in oncology where there is very little, if any, meaningful data, and it’s all based upon experience. We use whatever the best data is that we have. Real-world data, Big Data sometimes is the best data that we have.

It’s probably the most, what I would call practical data that we have, but I don’t think that real-world data really can take the place of the randomized clinical trial in many situations.

We started this conversation talking about bone marrow transplantation. If we were using real-world data, we would still be trying to figure out how to give higher and higher doses of cytotoxics to patients with breast cancer. And yet, we know, from what we’ve learned since the randomized trials became available, that that strategy with breast cancer, doesn’t work.

What it’s forced us to do is to look at a whole variety of new strategies.

The immunotherapies, the personalized medicine, the signal transduction inhibitors, all of those strategies that are not only more effective, they are easier to move forward in the science. There are more pathways, there are more options of investigations to try to make things better than I think that there would have been had high dose therapy proven to be the winner in those early trials.

So, we’ve gotten to a really complicated situation with all the different options of therapy now, and all the different pathways or avenues that we can go down.

And then, from NCCN’s perspective, we love that. We love that because, not only are patients doing better, but if there are 42 new oncology drugs in 2020. That means that NCCN solidly has a role to play in its space because we need to help educate, and help advise oncologists and other healthcare providers who are trying to ingest all this new data and information, and new agents in a sensible, understandable way.

That’s what NCCN does.

 

PG: Well, in a way, you created a rapid learning system.

RWC: Yes.

 

PG: What about actual, rapid learning systems as they exist or as they’re imagined, really now? Is that the future or is that … I’m asking this based on nothing.

RWC: Well, I think that rapid learning has many different definitions or considerations. I think the answer is yes, that rapid learning systems do have a role.

But I would also argue that if you look at the history of NCCN, we have a rapidly learning system. For instance, when the first generations of guidelines came out, they were updated intentionally once a year.

Now, because our guidelines are utilized by the vast majority of payers in the U.S., in making coverage determinations and decisions, we never want a patient to be denied an effective drug because our guidelines are out of date.

So, we now have a library of 81 guidelines. We had 184 versions of those guidelines published last year. So, the FDA approves a new agent, we are committed to having our panels review the new agent, the data supporting its use, determining where in our guideline, there should be a modification, if any, making the modification, publishing on our website, including updating the associated compendia products with that update, within two weeks. We actually do that. So, we are actually rapidly learning from the data that becomes available in real time.

We’d love to get that two weeks down to two or three days. It just because it’s impossible to get a panel together, to review something in a day, although we have historically once.

NCCN really is a rapid learning, rapid morphing system of content, at the current time. Whether a computer could do that for us, don’t know. We know that there’ve been at least a few failures of rapid learning systems in the oncology space.

 

PG: Right.

RWC: So, at least for the foreseeable future, we think we’re okay.

My suspicion is that, because there are so many circumstances where the data is just really, really limited, and the complexity of the problem is such that computer systems will probably never totally replace people.

And if you can’t replace people, then I think the judgments that the experts bring to the table, in developing our guidelines, the importance of that will never go away.

 

PG: Fascinating. I think we’ve just covered a quarter of a century.

RWC: And more.

 

PG: And more, of course. Is there anything we forgot? Anything that you’d like to add?

RWC: We focused primarily on the guidelines and that’s what NCCN is really known for, obviously, but there are so many other programs that we have ongoing.

We have our Oncology Research Program (ORP), we have a continuing education program. We have a chemotherapy order template program. We have a policy program. We have the NCCN Foundation. Huge number of initiatives that are ongoing, not to mention Best Practices Committee, Products and Services Committee, and so forth.

NCCN is an incredible organization. It has many more moving parts and pieces that I had never imagined before I came to headquarters. It continues to grow, continues to morph. We embrace change.

I think we’re solidly in the space, and look forward to continuing to work with the rest of the oncology and patient community.

 

PG: Thank you so much.

RWC: Thank you.