| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
ORIGINAL ARTICLES |
From the Departments of Surgery (PJT, OEN, BBRK) and Radiotherapy (AAMH), The Netherlands Cancer Institute, Amsterdam, the Netherlands.
Correspondence: Address correspondence and reprint requests to: P. J. Tanis, MD, Department of Surgery, the Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, the Netherlands; Fax: 31-20-512-2554; E-mail: ptanis{at}nki.nl
| ABSTRACT |
|---|
|
|
|---|
Methods: Four learning phases were analyzed: 25, 50, 75, and 150 procedures with a corresponding number of 10, 20, 30, and 60 tumor-positive cases. Critical values of nonidentification rate and false-negative rate were defined. The binomial distribution was used to calculate the probabilities of correctly or incorrectly accepting the quality of the performance, given a certain long-term nonidentification or false-negative rate.
Results: The chance of incorrectly reaching a favorable false-negative rate of <10% (critical value) in 20 metastasized patients was 18% for a surgeon with a long-term probability of false-negative procedures of 15%. This chance was reduced to 10% with a learning phase of 60 tumor-positive cases. When this chance has to be further reduced to 5%, the critical value has to be lower in smaller groups of patients: 5% false-negative rate in 20 tumor-positive procedures.
Conclusions: A learning phase of at least 150 procedures with 60 tumor-positive cases is needed to draw any reliable conclusion about the quality of sentinel node biopsy. In general, a compromise has to be made between the reliability of the results and the practically achievable number of procedures.
Key Words: Sentinel node Learning phase Sensitivity Identification rate Binomial distribution Critical value
| INTRODUCTION |
|---|
|
|
|---|
Some data about the initial experiences in sentinel node biopsy of individual surgeons have been published. Eleven surgeons participated in a multicenter validation trial in breast cancer, published by Krag et al.6 The number of sentinel node biopsies per surgeon ranged from 16 to 51, with a total of 443 procedures. The success rate differed significantly among the participating surgeons, with a minimum of 79% and a maximum of 98% identification. The false-negative rate ranged from 0% to 29%. In a study by Cody et al.,5 the identification rates of the four most experienced surgeons were from 92% to 96%, and the false-negative rates ranged from 7% to 20%. Bass et al.4 found that five surgeons obtained a 90% success rate (±4.5%) after 23 sentinel node biopsies on average, and this rate was 95% (±2.3%) after 53 sentinel node biopsies in breast cancer. In melanoma, the existence of a learning curve for sentinel node biopsy was demonstrated at The Netherlands Cancer Institute.7 The identification rate with the use of patent blue dye alone increased from 87% to 96% for one surgeon and from 76% to 86% for another surgeon. Almost all sentinel nodes (99.5%) were identified by adding the gamma-ray detection probe.
An institutional experience in sentinel node biopsy is made up of the experiences of the individual surgeons. A starting surgeon either can learn from his or her experienced colleagues or has to introduce the new procedure starting from scratch. The surgeon may find it helpful to attend a teaching course first.8 There are also several books and CD-ROMs that may help in the preparation.914
This article discusses the effect of the number of patients in the learning phase for sentinel node biopsy on the precision with which the identification rate and sensitivity can be assessed and hence on the probability to correctly or incorrectly accept the quality of the performance of a surgeon or institution.
| METHODS |
|---|
|
|
|---|
|
| (1) |
The sensitivity is defined as the complement of the false-negative rate:
|
| (2) |
Calculations were performed for four learning phases, with a total number of sentinel node procedures of 25, 50, 75, and 150. The numbers of tumor-positive procedures corresponding with the chosen group sizes were based on the assumption that 40% of patients with breast cancer would have lymph node metastases.15 This resulted in 10, 20, 30, and 60 tumor-positive procedures for the different learning phases. The acceptable long-term nonidentification rate, as well as false negativity, was assumed to be 5%. In addition, results for long-term false negativity of 10%, 15%, and 20% were also calculated for the groups of 20 and 60 tumor-positive procedures.
Critical values were defined for nonidentification rate and false negativity in such a way that the quality of the procedure was accepted if both the observed nonidentification rate and the false negativity were lower than their critical value. By using the binomial distribution, it is then possible to calculate the probability that an observed nonidentification rate or false negativity is lower than its critical value, given a certain long-term nonidentification rate or false-negative rate for the surgeon or center. Depending on the assumed long-term value, the calculated probability is the chance to correctly or incorrectly accept the quality of a surgeon or center. The chance of not accepting the quality is defined as the probability that an observed result is equal or higher than its critical value. Critical values to be used are given in Results.
| RESULTS |
|---|
|
|
|---|
|
Similar histograms can be made for the false-negative rate (Fig. 2). A striking difference in comparison to Fig. 1 is the more extended range of values with the same probability of .05. This phenomenon is caused by the fact that the false-negative rate is based on a subgroup of all patients who undergo sentinel node biopsy, namely, those with tumor-positive lymph nodes. Consequently, the group size of a learning phase should be based on the desired certainty about the safety of the procedure in terms of a low false-negative rate. For a surgeon with an actual false-negative rate of 5%, the chance of finding a false-negative rate of at least 10% (critical value) is 40%, 26%, 19%, and 8% if the learning phase contains 10, 20, 30, and 60 tumor-positive procedures, respectively (Fig. 2). These probabilities of incorrectly rejecting the sensitivity of the sentinel node procedure can be reduced to 5% if a critical value of 30% (n = 10), 20% (n = 20), 17% (n = 30), or 12% (n = 60) is used. However, this approach will increase the probability of accepting a long-term false-negative rate that is too high.
|
10% in a group of 20 patients with involved lymph nodes. When the number of patients with lymph node metastases is increased to 60, the binomial distribution histograms become more separated from each other, enabling a more reliable conclusion about the ultimate false-negative rate (Fig. 4). The previously calculated chances in 20 tumor-positive procedures are now reduced from 18% to 10% and from 26% to 8%.
|
|
|
| DISCUSSION |
|---|
|
|
|---|
This article is focused on the surgeons learning phase, but clearly other factors also play a role. Several studies have pointed out that a combined technique of both patent blue dye mapping and the use of a gamma-ray detection probe after administration of a technetium-labeled colloid is superior to a single-agent technique.1619 Tracer migration from the dense dermal lymphatic network in melanoma results in a clearer visualization of the sentinel node in comparison to breast cancer and probably influences the length of the learning curve in these two malignancies. There is increasing evidence that preoperative lymphoscintigraphy is helpful.20 It is important that the surgeon review the images together with the nuclear medicine physician. A pathologist who scrutinizes the lymph node by using serial sectioning and immunohistochemical staining is also essential. Therefore, lymphatic mapping requires a team effort. A surgeon, no matter how skilled, will never find 95% of the sentinel nodes with a 95% sensitivity if not backed by a competent nuclear medicine physician and a competent pathologist. Different teams may have different results. However, a standardized technique within an institution allows evaluation and comparison of the performance of individual surgeons.
Do identification rate and sensitivity go hand in hand? Theoretically, rerouting of lymph flow to a neosentinel node because of tumor blockage does not hamper identification of the neosentinel node that may still be tumor free. In other words, such a false-negative case will not impair the identification rate. Krag et al.6 described a surgeon with a 98% success rate and a 27% (4 of 14) false-negative rate and another surgeon with 79% identification and a false-negative rate of only 7% (1 of 14). From this point of view, it is not sufficient to consider only identification rate as a parameter in determining the length of a learning curve. The clinical effect of a false-negative sentinel node procedure seems more pronounced than not finding the node, which will be followed by an axillary lymph node dissection in breast cancer patients. The false-negative rate should be the most important factor in discussions about the number of procedures that have to be completed before reliable conclusions can be drawn about the performance of a team.
The term learning curve suggests that most failures occur in the initial phase of the new procedure. If this is true, our calculations on the required number of patients should be considered optimistic. The binomial distribution assumes a constant failure probability. Valid statistical inference on the basis of this model has to be restricted to the subgroup of later patients in the learning curve for whom this can be assumed. Other models for which this assumption is not necessary will always lead to less precision than inference from a binomial distribution with the same number of patients. Curves of sentinel node identification in breast cancer published by Bass et al.4 indeed show a high initial nonidentification rate of >30% after a few procedures. Subsequently, the percentage of nonidentification diminished gradually and stabilized between 0% and 10% after approximately 15 to 40 procedures. In their recent experience with more participating surgeons, these investigators found two other types of curves.21 The first one was a late onset of unsuccessful sentinel node biopsies. The second one was an increasing nonidentification rate from the beginning. Late failures were significantly associated with a low individual surgical volume index (number of procedures per month per surgeon). These observations support the idea that good results in the initial series of procedures are not always indicative for future procedures.
In the experience of Cody et al.,5 false-negative cases occur early in the experience of a surgeon. Two of the four most experienced surgeons had their false-negative biopsy in the first 10 cases (cases 1 and 6) and the other two in the next 10 cases (cases 12 and 20). With the exclusion of the first six biopsies of each surgeon, the false-negative rate decreased from 11% to 5%. These observations are in contrast to our own findings in 82 patients with breast cancer who underwent sentinel node biopsy with confirmatory axillary lymph node dissection performed by three surgeons. The sentinel node was false negative in the 62nd and 78th procedure. It is hard to draw conclusions from these findings because of the low incidence and the complex etiology of false-negative procedures. The contribution of the surgeon to the total false-negative rate of a multidisciplinary team in variable patient populations based on different selection criteria cannot be determined exactly. One aspect of the relationship between false-negative results and the number of procedures is indisputable: the false-negative rate will never be 0%.
It is unclear how rigorously one should adhere to a certain false-negative rate. This rate depends on how it is determined. Do false-negative biopsies have to declare themselves during follow-up, as is customary for a learning phase in melanoma patients? If completion regional lymph node dissection is performed, as is customary for a learning phase in breast cancer, how rigorously are metastases in nonsentinel nodes pursued? Is a truly complete nodal clearance performed? Does the pathologist find all the nodes in the chunk of fatty tissue that the surgeon submits? A team with a skillful surgeon performing radical node dissections and a dedicated pathologist examining all nonsentinel nodes with serial sectioning and immunohistochemistry may find more tumor-positive nonsentinel nodes. Could it be that a better team will have a higher false-negative rate?
How many procedures should a surgeon perform to be qualified to implement lymphatic mapping in routine patient management? A total number of 20 to 30 procedures, as proposed by Giuliano,22 seems to be inadequate to reliably calculate the safety of the procedure in the hands of a specific surgeon and the other members of the multidisciplinary team. Morton23 suggested that an acceptable level of technical skill requires success in 30 to 50 melanoma patients and 60 to 80 cases of breast cancer. Our results show that at least 150 procedures, including 60 with lymph node involvement, are needed to make a somewhat reliable conclusion about the quality of sentinel node biopsy. Even with this high number of procedures, there is a chance of inadvertently characterizing the quality as sufficient or insufficient. It will take 750 patients with 300 tumor-positive basins to establish with 95% certainty that a surgeon who has a nonidentification rate of 5% and a false-negative rate of 5% indeed has these capabilities within a range of 0% to 7%. It is unrealistic to pursue these standards. Increasing numbers of well-informed breast cancer patients nowadays are not willing to undergo a confirmatory axillary lymph node dissection when a nearby hospital has abandoned confirmatory axillary node dissection. These practical problems in controlled implementation of sentinel node biopsy were illustrated by a survey among surgeons in the Netherlands.24 Routine complete lymph node dissection in breast cancer patients was omitted by 43% of surgeons who had performed <10 procedures, by 44% of surgeons with <25 procedures, and by 70% after a learning phase of <50 procedures. A survey among American surgeons who had attended a university-sponsored course showed that almost all surgeons (95%) did not complete 30 validation cases and that 55% of them completed <10.25 In melanoma, the situation is even more complicated because the incidence of the disease is less, the percentage of involved basins is less, and confirmatory lymph node dissection is difficult to justify. Long-term follow-up is needed to determine the actual false-negative rate. Still, increasing numbers of melanoma patients wish to be informed about their lymph node status.
An important aspect in the decision about the length of a learning curve is to determine a desired certainty that a result will be below a critical value. The chosen critical value has an important effect on that certainty, particularly in relatively short learning phases, as shown in this study. When the chance of incorrectly accepting an actual false-negative rate of 15% or higher has to be reduced to a minimum of 5%, the critical value has to be lower in smaller groups of patients (Table 1): no false-negative procedure (critical value 5%) is allowed in 20 tumor-positive procedures.
Series from high-volume centers and experienced investigators confirmed that the sentinel node is the first lymph node reached by tumor cells in approximately 95% of all metastasized tumors, with a small confidence interval. These series, however, are no guarantee of success for starting surgeons. Statistical analysis shows that a short learning phase may not be representative of the ultimate sensitivity and may conceal an unacceptably high recurrence rate in the future.
| CONCLUSIONS |
|---|
|
|
|---|
Received for publication May 7, 2001. Accepted for publication October 12, 2001.
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
S. H. Estourgie, O. E. Nieweg, R. A. Valdes Olmos, C. A. Hoefnagel, and B. B. R. Kroon Review and Evaluation of Sentinel Node Procedures in 250 Melanoma Patients With a Median Follow-Up of 6 Years Ann. Surg. Oncol., July 1, 2003; 10(6): 681 - 688. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |