Discrepancies in BRAF and EGFR Mutation Detection
Discrepancies in BRAF and EGFR Mutation Detection
The cohort comprised 13 clinical laboratories. Case volume of each laboratory is illustrated in figure 2. Table 1 provides an overview of participant methodology.
(Enlarge Image)
Figure 2.
Annual number of cases screened by each participant for epidermal growth factor receptor (EGFR) and BRAF mutations demonstrates that the cohort had experience detecting EGFR and BRAF mutations, on average screening >10 and >7 samples per week for EGFR and BRAF mutations, respectively. The volume of our cohort (489) is more than double that of 112 participants in a European external quality assurance scheme, suggesting our findings may be relevant to other laboratories.
The range of DNA recovery from the total number of samples analysed in the ring trial (n=104) is shown in figure 3. Yield variance was highest for (control) sample 6 (mean=4473.1 ng, SD=5508.4 ng, CV=123.1%, n=13) and lowest for sample 2, an engineered sample (mean=1016 ng, SD=1124.7 ng, CV=58.4%, n=13). Significant difference in DNA yields across all samples was seen using ANOVA (p<0.0001); however, variation among engineered samples was not significant (p=0.3782). Consistency among the engineered samples was to be expected due to their artificial nature, demonstrating their suitability for use as reference material. On average, reported yields from engineered samples (n=78) were 1.3-fold greater than the calculated theoretical yield, suggesting potential problems arising during the pre-PCR process (see online supplementary figure S1 http://jcp.bmj.com/content/68/2/111/suppl/DC1). Surprisingly, 10/13 laboratories reported yields averaging 235.8 ng (SD=266.7, CV=1.13%, n=13) from cell-negative sample 5.
(Enlarge Image)
Figure 3.
Range of total DNA (in nanograms) recovered from each sample by participants. In total, 104 samples were analysed. Samples 1–4 comprise theoretical DNA yields of approximately 1100 ng, sample 5 comprises a cell-negative curl, sample 6 comprises a tonsil tissue specimen, sample 7 comprises 1100 ng theoretical DNA yield, in turn harbouring BRAF V600E and EGFR G719S, and sample 8 comprises 1100 ng theoretical DNA yield, in turn harbouring BRAF V600E and EGFR L858R. The range of DNA recovered from the engineered samples, after dropping laboratory K outlier values, was >55-fold for two of the six and >5-fold for the four others. Range in DNA recovered from control sample 6, excluding outliers from laboratory K, was >40-fold.
Comparing DNA recovery from engineered samples shows laboratory D was most consistent (CV=17.1%) and laboratory K was least consistent (CV=146.7%) (see online supplementary figure S2 http://jcp.bmj.com/content/68/2/111/suppl/DC1). Laboratory K failed DNA recovery from 7/8 samples. Laboratory A failed DNA extraction from 2/8 samples.
In total, 11.9% (5/42) of potential calls failed to detect the EGFR or BRAF mutations present (Table 2). Laboratory K failed to detect mutations in samples 7 and 8 due to insufficient DNA recovery (two calls missed as did not routinely screen EGFR mutations). Laboratory A failed to test sample 7 due to insufficient DNA recovery (two calls missed). Laboratory H extracted adequate DNA but failed to detect 20% EGFR p.L858R in sample 8 using Sanger sequencing (one call missed). This is likely to be due to the limit of detection of Sanger sequencing approaching 20% mutant: wild type.
Extraction. Quantities of DNA extracted from the engineered samples (measured by Qubit) were compared with respect to the various extraction methodologies employed. The results demonstrate that Qiagen EZ1 had the lowest yield variance (CV=52%), while Qiagen QIAamp had the highest (CV=82%). Despite small sample size, overall yield variance between the methods was relatively low (30%) (figure 4).
(Enlarge Image)
Figure 4.
Variance in DNA recovered by the different extraction methods was calculated using Qubit measurements for engineered samples 1–4. It should be noted that one of the laboratories used a modified version of RecoverAll; however, for the purpose of the analysis, these were treated the same. N refers to the number of laboratories using each method.
Quantitation. Correlation between Nanodrop and Qubit measurements for identical samples (n=78) was poor (R=0.48, p<0.0001) (see online supplementary figure S3 http://jcp.bmj.com/content/68/2/111/suppl/DC1). Comparing Qubit versus Nanodrop measurements for identical samples, represented as the Nanodrop:Qubit (N/Q) ratio, demonstrates median Nanodrop readings were 5.1-fold higher than Qubit measurements for the same samples (figure 5). In addition, Qubit measurements of cell-negative sample 5 from all laboratories were undetectable in all cases.
(Enlarge Image)
Figure 5.
Average Nanodrop:Qubit ratio for each laboratory as well as the average and median ratio for the entire cohort.
DNA integrity. Sample quality (n=78) assessed using multiplex-PCR showed degradation of DNA from control sample 6 (figure 6C). In contrast, DNA from the engineered samples was intact (figure 6A). Analysis of cell-negative sample 5 confirmed absence of amplifiable material (figure 6B), underscoring the significance of DNA recovery reported for these samples. Average N/Q ratio for the degraded control samples (N/Q=15.6) was higher than intact engineered samples (N/Q=10.9).
(Enlarge Image)
Figure 6.
Biomed 2 multiplex PCR results for assessing DNA integrity. (A) Representative set of engineered samples indicates high-quality DNA with four strong bands at 400, 300, 200 and 100 bp. (B) Comparison between engineered samples (B2 and C2) and a representative set of cell-negative samples 5. No bands were seen for cell-negative samples except in samples from laboratories E, I and K. These were not reproducible on repeat. (C) Representative set of control samples from each laboratory shows just one band at 100 bp indicating degraded DNA.
Asuragen assessed samples from a subset of laboratories (D, H, J, L and M) using QFI-PCR, which demonstrated DNA from engineered samples 1–4 to have similar 'functional' quality, with approximately 1 in 4–6 input templates amplified. In contrast, <1 in 100 templates in DNA from control sample 6 were amplified. These findings have close correlation with the multiplex-PCR data.
PCR inhibition. 'SPUD' PCR analysis demonstrates absence of PCR inhibitors in all but one sample. Mean Cq across all samples (Cq=30.21) was consistent with mean Cq from the standard curve for 600 copies of exogenous DNA target (Cq=30.14) in all but one sample, M6 (see online supplementary figure S4 http://jcp.bmj.com/content/68/2/111/suppl/DC1).
Results
Participant Demographics
The cohort comprised 13 clinical laboratories. Case volume of each laboratory is illustrated in figure 2. Table 1 provides an overview of participant methodology.
(Enlarge Image)
Figure 2.
Annual number of cases screened by each participant for epidermal growth factor receptor (EGFR) and BRAF mutations demonstrates that the cohort had experience detecting EGFR and BRAF mutations, on average screening >10 and >7 samples per week for EGFR and BRAF mutations, respectively. The volume of our cohort (489) is more than double that of 112 participants in a European external quality assurance scheme, suggesting our findings may be relevant to other laboratories.
Variation in DNA Recovery
The range of DNA recovery from the total number of samples analysed in the ring trial (n=104) is shown in figure 3. Yield variance was highest for (control) sample 6 (mean=4473.1 ng, SD=5508.4 ng, CV=123.1%, n=13) and lowest for sample 2, an engineered sample (mean=1016 ng, SD=1124.7 ng, CV=58.4%, n=13). Significant difference in DNA yields across all samples was seen using ANOVA (p<0.0001); however, variation among engineered samples was not significant (p=0.3782). Consistency among the engineered samples was to be expected due to their artificial nature, demonstrating their suitability for use as reference material. On average, reported yields from engineered samples (n=78) were 1.3-fold greater than the calculated theoretical yield, suggesting potential problems arising during the pre-PCR process (see online supplementary figure S1 http://jcp.bmj.com/content/68/2/111/suppl/DC1). Surprisingly, 10/13 laboratories reported yields averaging 235.8 ng (SD=266.7, CV=1.13%, n=13) from cell-negative sample 5.
(Enlarge Image)
Figure 3.
Range of total DNA (in nanograms) recovered from each sample by participants. In total, 104 samples were analysed. Samples 1–4 comprise theoretical DNA yields of approximately 1100 ng, sample 5 comprises a cell-negative curl, sample 6 comprises a tonsil tissue specimen, sample 7 comprises 1100 ng theoretical DNA yield, in turn harbouring BRAF V600E and EGFR G719S, and sample 8 comprises 1100 ng theoretical DNA yield, in turn harbouring BRAF V600E and EGFR L858R. The range of DNA recovered from the engineered samples, after dropping laboratory K outlier values, was >55-fold for two of the six and >5-fold for the four others. Range in DNA recovered from control sample 6, excluding outliers from laboratory K, was >40-fold.
Laboratory Variation
Comparing DNA recovery from engineered samples shows laboratory D was most consistent (CV=17.1%) and laboratory K was least consistent (CV=146.7%) (see online supplementary figure S2 http://jcp.bmj.com/content/68/2/111/suppl/DC1). Laboratory K failed DNA recovery from 7/8 samples. Laboratory A failed DNA extraction from 2/8 samples.
Mutation Analysis
In total, 11.9% (5/42) of potential calls failed to detect the EGFR or BRAF mutations present (Table 2). Laboratory K failed to detect mutations in samples 7 and 8 due to insufficient DNA recovery (two calls missed as did not routinely screen EGFR mutations). Laboratory A failed to test sample 7 due to insufficient DNA recovery (two calls missed). Laboratory H extracted adequate DNA but failed to detect 20% EGFR p.L858R in sample 8 using Sanger sequencing (one call missed). This is likely to be due to the limit of detection of Sanger sequencing approaching 20% mutant: wild type.
Variation in pre-PCR Steps
Extraction. Quantities of DNA extracted from the engineered samples (measured by Qubit) were compared with respect to the various extraction methodologies employed. The results demonstrate that Qiagen EZ1 had the lowest yield variance (CV=52%), while Qiagen QIAamp had the highest (CV=82%). Despite small sample size, overall yield variance between the methods was relatively low (30%) (figure 4).
(Enlarge Image)
Figure 4.
Variance in DNA recovered by the different extraction methods was calculated using Qubit measurements for engineered samples 1–4. It should be noted that one of the laboratories used a modified version of RecoverAll; however, for the purpose of the analysis, these were treated the same. N refers to the number of laboratories using each method.
Quantitation. Correlation between Nanodrop and Qubit measurements for identical samples (n=78) was poor (R=0.48, p<0.0001) (see online supplementary figure S3 http://jcp.bmj.com/content/68/2/111/suppl/DC1). Comparing Qubit versus Nanodrop measurements for identical samples, represented as the Nanodrop:Qubit (N/Q) ratio, demonstrates median Nanodrop readings were 5.1-fold higher than Qubit measurements for the same samples (figure 5). In addition, Qubit measurements of cell-negative sample 5 from all laboratories were undetectable in all cases.
(Enlarge Image)
Figure 5.
Average Nanodrop:Qubit ratio for each laboratory as well as the average and median ratio for the entire cohort.
DNA integrity. Sample quality (n=78) assessed using multiplex-PCR showed degradation of DNA from control sample 6 (figure 6C). In contrast, DNA from the engineered samples was intact (figure 6A). Analysis of cell-negative sample 5 confirmed absence of amplifiable material (figure 6B), underscoring the significance of DNA recovery reported for these samples. Average N/Q ratio for the degraded control samples (N/Q=15.6) was higher than intact engineered samples (N/Q=10.9).
(Enlarge Image)
Figure 6.
Biomed 2 multiplex PCR results for assessing DNA integrity. (A) Representative set of engineered samples indicates high-quality DNA with four strong bands at 400, 300, 200 and 100 bp. (B) Comparison between engineered samples (B2 and C2) and a representative set of cell-negative samples 5. No bands were seen for cell-negative samples except in samples from laboratories E, I and K. These were not reproducible on repeat. (C) Representative set of control samples from each laboratory shows just one band at 100 bp indicating degraded DNA.
Asuragen assessed samples from a subset of laboratories (D, H, J, L and M) using QFI-PCR, which demonstrated DNA from engineered samples 1–4 to have similar 'functional' quality, with approximately 1 in 4–6 input templates amplified. In contrast, <1 in 100 templates in DNA from control sample 6 were amplified. These findings have close correlation with the multiplex-PCR data.
PCR inhibition. 'SPUD' PCR analysis demonstrates absence of PCR inhibitors in all but one sample. Mean Cq across all samples (Cq=30.21) was consistent with mean Cq from the standard curve for 600 copies of exogenous DNA target (Cq=30.14) in all but one sample, M6 (see online supplementary figure S4 http://jcp.bmj.com/content/68/2/111/suppl/DC1).
Source...