Methylene blue sentinel lymph node biopsy for breast cancer learning curve in the COVID-19 era: How many cases are enough?

Introduction

Sentinel lymph node (SLN) biopsy has replaced axillary lymph node dissection (ALND) as the standard minimally invasive staging procedure in patients with clinically node-negative disease.^1–3 Dual tracers, such as blue dye and radiotracer mapping, are recommended in the Asia Pacific and Europe to achieve a higher SLN identification rate especially for node negative after neoadjuvant chemotherapy than blue dye alone. However, because radiotracer mapping is more expensive and has several disadvantages, methylene blue dye as a single agent is well-tailored to use in developing countries without significantly compromising test quality.^4–6

There is a debate in the collegium of surgical oncology about what number of surgeries a trainee (prospective oncology surgeon) should be achieve to be competent in performing specific actions. Sentinel Lymph Node Biopsy (SLNB) is one of the basic procedures that a breast cancer surgeon must master. The American Society of Breast Surgeon recommends that a trainee only achieve this competency if he has performed this SLNB action 20 times with a False Negative Rate of no more than 5% and an identification rate from SLN of more than 85%. The low False Negative number is a symbol of high accuracy, while the identification rate reflects the competence of a surgeon performing SLNB procedures regardless of the type of agent and volume used to perform SLNB, patient selection, and injection location of SLNB marker agents.

The Indonesian Board-Certified oncologist surgeon admitted that the learning curve used to assess the fellow surgeon objectively might be shorter, especially during the COVID-19 pandemic, when surgery volume and timeframe should be reduced.⁶ A method for plotting learning curves that can be used to check and predict the performance of others.^7,8

It is necessary to develop a method for plotting learning curves to test and predict individual performance concerning a standardized degree of proficiency. Cumulative sum chart (CUSUM) plots are an excellent method for determining learning curves for any technique with specific and difficult output variables. They will be able to customize qualifications, knowledge, and skill certification criteria and deal with training issues throughout the COVID-19 pandemic.⁹

Methods

Results

Then a calculation of CUSUM will be carried out to get the value of $S_j$. The calculations for the 3 trainees are as follows:

Based on the values $S_j$ in the table above, a CUSUM chart will be created using the MINITAB 21 software. The graph of the 3 fellows is as follows:

Figure 1. Cusum Curve based on time.

G1: Trainee A with the red line, G2: Trainee B with the yellow line, G3: Trainee C with the purple line. Note the different behavior of the G1, G2, and G3 curves, regardless of the same end result between G1 and G2. The black line on each plot describes the upper decision limit (h1) and the lower decision limit (h0).

In total, there were 66 procedures SLNB that were assessed based on the time was needed to identify (Tables 3 & 4). Trainee A performed 25 procedures, Trainee B performed 20 procedures, and Trainee C performed 21 procedures. All trainees achieved a minimum number of procedures so the analysis had an 85% success rate. Based on the graph above, by being in two boundary lines (which means that the result is indeterminate and the actual failure rate is different from the acceptable failure rate), and if it passes the Upper decision boundary (h1) and Lower decision boundary (h0), then the actual failure rate does not differ significantly from the acceptable failure rate. It can be presented as follows.

Trainee A passes the Upper decision boundary (h1) in the 8^th to 25^th operation, the 1^st to 7^th attempt is within two boundary lines, so it can be concluded that the minimum number of surgeries that a surgical trainee A must perform to achieve competence based on identification time is 7 times.

Trainee B passes the Upper decision boundary (h1) in the 5^th to 20^th operation, while in the 1^st to 4^th operation it is within two boundary lines, so it can be concluded that the minimum number attempt that surgical trainee B must perform to achieve competence based on identification time is 4 operations.

Trainee C passes the Upper decision boundary (h1) in the 4th to 21^st attempt, while in the 1^st to 3^rd operation it is within two boundary lines, so it can be concluded that the minimum number of operations that the C surgical trainee must perform to achieve competence is 3^rd attempt.

By the definition the accuracy to performed SLNB is determined by Correct Identification Rate more than 85%. Identification by SLNB which achieved standard $X_j=1$ and failed $X_j$ (Figure 1).

Then a calculation of CUSUM will be carried out to get the value of $S_j$ . The calculations for the 3 trainees are as follows (Table 5):

Based on the values $S_j$ in the table above, a CUSUM chart will be created using MINITAB software. The graph of the 3 fellows is as follows:

Figure 2. Cusum Curve for Negative Positive Data identified by SLNB and by gold standard.

G1: Trainee A with the red line, G2: Trainee B with the yellow line, G3: Trainee C with the purple line. Note the different behavior of the G1, G2, and G3 curves, regardless of the same end result between G1 and G2. The black line on each plot describes the upper decision limit (h1) and the lower decision limit (h0).

In total, there were 66 procedures (operations) that were assessed based on accuracy to correct identification rate as gold standards. Trainee A performed 25 procedures, Trainee B performed 20 procedures, and Trainee C performed 21 procedures. All trainees achieved a minimum number of procedures. Based on the graph above, by being in two boundary lines (which means that the result is indeterminate and the actual failure rate is different from the acceptable failure rate), and if it passes the Upper decision boundary (h1) and Lower decision boundary (h0), then the actual failure rate does not differ significantly from the acceptable failure rate. It can be presented as follows.

Trainee A passes the Upper decision boundary (h1) in the 5^th to 25^th attempt, the 1^st to 4^th attempt within two boundary lines, so it can be concluded that the minimum number of procedures that surgical trainee A must perform to achieve competence based on standard is 4 procedures.

Trainee B passes the Upper decision boundary (h1) in the 5^th to 20^th operations, while on procedures 1 to 4 it is within two boundary lines, so it can be concluded that the minimum number of procedures that trainee B must perform to achieve competence based on standard is 4^th.

Trainee C passes the Lower decision boundary (h0) in operations 6 to 21, in procedures 1^st to 5^th is within two boundary lines, so it can be concluded that the minimum number of operations that surgical trainee C must perform 5 procedures to achieve competency (Figure 2).

We conducted a separate analysis for the achievement SLNB by conducting an analysis based on the level of accuracy and time required to identify blue nodes when conducting SLNB. The number of glands identified through SLNB and the number of glands identified in paraffin block, the average size of the tumor and the location of the tumor are summarized in Table 1.

Discussion

The SLNB is one of the essential procedures in breast cancer surgery that oncologist surgeons should master during their training. Many criteria influence whether a surgeon can do a particular operation, including their medical knowledge, specific training, and level of expertise. Under the supervision of a supervisor, the skill to perform a surgical technique is usually acquired through observation, learning, and repetition.

In general, a combination of informal assessment and peer review and more formal accreditation, credentialing, or privilege can be used to ensure the quality of medical practice. The evaluation, review, or credentialing process is frequently subjective and lacks specific standards of practice. It has been suggested that comparative treatment result data on specific physician performance benchmarks are required to establish a single process’s credibility.⁸

The CUSUM chart can be used as one tool to assess the level of competence and has been widely used to evaluate the achievement of the learning curve for some new procedures in surgical fields. The CUSUM curve is a control chart that can monitor shifts in the process mean. The target is the plot should not be widely variable from the reference value (attending performance).^8,9

In our result the required competence of fellow breast surgeons in Indonesia can be achieved quickly. The fellow surgeon in this study comes from the general surgeon who has experience and is familiar with mastectomy procedures, including axillary clearance, as a part competency that should be achieved to be a general surgeon in rural areas. Successful SLN biopsy depends primarily on the accurate identification of the metastatic route. Knowledge of the anatomy of the lymphatics is essential. Fellow surgeons can easily attain this acquisition part on 4 to 5 attempts, when we used the time identification as target point there is one fellow who need a longer time to identified the blue nodes.^10,11

However, there are several problems with using CUSUM analysis to assess performance in procedural skills. First, no nationally agreed definitions exist for success or failure at any given procedure, and those used in the literature vary greatly.¹²

There is currently no consensus on where the acceptable and unacceptable boundaries should be set or to what degree alpha and beta errors should be tolerated. Tight boundaries are essential for quality control and for assessing trained individuals. Still, we should set these boundaries to be much more comprehensive for the fellows to allow for their learning curve and to provide encouragement and a sense of achievement. The number of competent surgeons produced can increase dramatically simply by altering the boundaries. Therefore, if procedural competency is to be defined by CUSUM, it would be necessary to establish national rates. These would need to be tailored to the fellow’s experience.^8,13

Second, in this research, we have not included the characteristics of patients in the analysis; the size of the tumor, location of the tumor, and type of surgery probably have influenced the procedure’s achievement.^14,15

This study does not specifically discuss the technique of Sentinel Lymph Node Biopsy using Methylene Blue so we do not discuss the complications of SLNB using Methylene Blue but still observed patients according to standards.

No systemic anaphylactic reaction and postoperative complication related to the subdermal injection of methylene blue, such as allergic reactions and skin or parenchymal necrosis occurred throughout the study. One Patient who underwent BCS exhibited blue skin staining in the skin around the injection site, which remained for approximately 1-2 months after the procedure.

Conclusion

Using the CUSUM chart, a reasonable choice of other parameters shows that experienced breast surgeons have completed the SLNB learning curve after 4-5 successful methylene blue attempts. In the future, this learning curve analysis can be applied to trainee breast surgeons by utilizing a proxy measure for failure, such as failure to identify the SLN.

Data availability