In a world centered on information, knowledge about ourselves and the conditions that affect us are the greatest medical tool we can wield. When it comes to diagnosis, modern technologies are continually creating systems that are easier and cheaper to implement, and vastly more accurate. Developed in the mid-1990s and now used nearly universally for the rescreening of pap smears (the primary method of diagnosis of cervical cancer), Redmond-based NeoPath’s AutoPap system exemplifies the application of digital image analysis to this field. The algorithms developed for this purpose have served as the basis for the development of a wide array of more recent automatic digital cytometry and detection tools, such as Deus Technologies’ system for the detection of lung nodules. However, the most significant development that the use cases presented by AutoPap have led to is that of the advanced system of flow cytometry. Through a more accurate and consistent approach to the same issue addressed by AutoPap, flow cytometry has the potential to serve as a low-cost method of pathology for a much wider range of situations and diseases in the near future.
The application of image analysis to the situation of automatic cytometry, that is, the computerized measurement of various properties of individual cells (size, morphology, coloration, and a number nucleic properties, among others), represents the unique challenge of presenting high levels of uniformity in the background that must initially be eliminated (this process will be further discussed below). The background referred to above consists of precisely those cells and other materials which result in the high rate of false positives in manual pathology (particularly of pap smears). The greatest confounding factor in visual cancer diagnosis is the Atypical Squamous Cell of Undetermined Significance (ASCUS). In the pursuit of pre-cancerous Low-Grade Squamous Intraepithelial Lesions (LSIL) and High-Grade Squamous Intraepithelial Lesions (HSIL), as well as actual cancer cells, pathology technicians frequently encounter ASCUS’, which can skew conclusions erroneously (Imperial). As such, given the widely diverse applications of image analysis (from motion tracking to “computational photography”), the development of specific algorithms that appropriately lower the error rate caused by the aforementioned confounding factor became necessary.
The first step to AutoPap’s algorithmic method to achieve this was the filtration of images gathered. As described by Randall L. Luck and Richard Scott in their patent titled “Morphological Classification System and Method,” prior art for AutoPap, this began with repeatedly determining cells larger and smaller than the “approximate size of a malignant or premalignant cell” (Luck). Having done this, these images were subtracted from the original to form an output solely consisting of cells fitting the parameters being sought.
Following filtration, images were fed through a set of algorithm consisting of decision trees to complete two tasks: 1) rank samples on a scale of 0 to 1 based on risk of abnormality, and 2) determine specific locations with high risk of abnormality. As described by Dr. Michael J. Seo, a former employee of NeoPath who collaborated on the development of AutoPap, in an online interview, “[NeoPath] had the leaders of cytopathology and trained [their] algorithms on their feedback.” Further factors considered in NeoPath’s algorithms, aside from simple size and shape (the latter being further flushed out by the successive lung nodule detection system by Deus Technologies described below), were later discussed by Dr. Seo. These primary factors were “nuclear to cytoplasm [ratio] and hyperchromasia” (Seo), though over a 100 correlated factors were considered by the decision trees. Of course, there are many other distinct factors considered by manual cytopathologists in distinguishing pre-cancerous cells (as cancerous cells themselves have more distinct differences from their environment). The online Eurocytology Training Package lists these for cervical cancer (pap smears): “nuclear size, nuclear shape, structure of chromatin in interphase nucleus, chromatin content of interphase nucleus, hyperchromasia, multinucleation, nucleoli, cohesiveness, and mitoses” (Imperial). A number of these (nucleoli, mitoses, both factors concerning chromatin) are unfortunately infeasible in the context of static snapshots of slides (whether based on scale, requirement for movement, or non-visual analyses). However, the rest certainly hold the potential to act as differentiating factors in determining the scale of risk factor and specific high-risk regions through automatic image analysis.
The next step undertaken by researchers, this time at Rockville, Maryland-based Deus Technologies, LLC, was to apply the methods developed by NeoPath to a larger scale: detection of lung nodules in radiological images. The most significant addition to the procedure utilized by Deus Technologies was a technology which NeoPath had been unable to use because of patent issues: neural networks (Yeh). Neural networks transcended the capabilities of decision trees because, as an Artificial Intelligence system capable of continuous machine learning, they transitioned the process of filtration from a sequential, taxonomical analysis to a more efficient synchronous consideration of factors according to a highly flexible system that inherently increases its accuracy over time. Interestingly, the greatest result of this flexibility was the fact that, despite radiological imaging still having a consistent scale, this system allowed baseline comparisons to be individually calculated for each sample.
Deus Technologies’ patent for the lung nodule detection system discussed here describes the final imaging phase, focused on classification: “the data is first analyzed using background correction, followed by an edge operation, histogram generation, marginal distribution generation, standardization, and neural network classification and integration” (Yeh). Of course, the applicability of these custom evaluative distributions for each iteration, and the efficacy of the decision flexibility allowed by neural networks, is rooted in the consideration of a variety of truly innovative factors considered by the algorithm. Beyond simple morphological filtering in the same style pioneered by AutoPap, this system drew off of the 3D characteristics of their distinct situation from cytopathology, and created an even more significant technique for testing the sphericity of nodules. Once again dependent on the median of a calculated distribution, and capable of classifying based on this comparison, this distinctly added to the role of image analysis in not only rescreening, but even accurate diagnosis as a whole.
A final technique utilized here succeeded in achieving what is referenced by the patent as “body part segmentation” (Yeh). This is a technique called Image Block Segmentation which adds an even greater level of accuracy to group analysis based on divisive factors. In a paper titled “Strategic Approach to Image Block Segmentation,” Avanish Shrivastava and Mohan Awasathy describe this technique as follows:
Many image processing tasks require to know the meaning (e.g. object or background) of the image pixels. Image segmentation is an important process to furnish such information to many image processing applications such as pattern recognition and object identification. Image segmentation is a process of dividing an image into different regions such that each region is nearly homogeneous, whereas the union of any two regions is not (Unnikrishnan R. et. Al. 2007)
Of course, the applicability of this to situations like AutoPap is difficult due to the fact that these are dependent on massive-scale analysis for very small amounts of suspect cells, and thus segmentation primarily results in empty areas, and thus a potential skew in the percentage of suspect cell presence. However, it is an essential precedent that establishes an efficient method to approaching a wide variety of such analytical tools in the future.
Present research, such as that being done by Clemex, is attempting to create universal methods that would allow simple customization for the detection of varying irregularities in differing environments, by non-technical personnel. In fact, even from the top of their description of their Vision PE Multipurpose Image Analysis System, they claim that “even without image analysis expertise, the short learning curve means you’ll be performing analyses within hours of setup” (Clemex). As an integrated microscopy system, it is capable of being customized to examine virtually any visual parameter of a sample, including “cell count, fiber length, grain size, layer thickness, nodularity, particle size, phase area, porosity, shape analysis, and surface roughness” (Clemex). With “automatic object separation [and] multi-layer grab” (Clemex), the best techniques demonstrated by the previously discussed tools have been implemented to allow individuals with absolutely no digital image analysis background (many manual cytopathologists) to easily utilize their existing knowledge to form automatic, large-scale automatized versions of themselves.
The primary reason AutoPap remains a mechanism purely for re-screening is the inherent unreliability of its basis technology: pap smears themselves. Dr. Seo discussed the inescapable presence of confounding materials in the images analyzed by AutoPap: Despite the elimination of many such factors through AutoPap’s high-speed imaging system and image filtration algorithms, the possibility for cancerous/pre-cancerous cells to be hidden in a non-visible area of a smear remained. Through the combined use of liquid-suspended cell samples and an analysis technique known as flow cytometry, this issue can be overcome. Note that, though AutoPap’s present-day counterpart, BD’s SurePath, requires storage and transport of full samples in liquid form, analysis persevered in slide form, as opposed to a fluidic imaging system (BD).
As stated by Dr. Seo, “Flow cytometers allow… researchers to focus cells one at a time… through a fluidic system.” In contrast to AutoPap’s methods of analyzing groups of cells, flow cytometry purely relies on gathering large amounts of data on individual cells, and allowing interpretation of this based on the context of sample size. Due to the universality of this method, it can easily be adapted to virtually any pathological application. Given appropriate sample collection techniques, this can also relegate the jobs of visual analysis algorithms like AutoPap’s cell group analysis (to reduce confounding by ASCUSs) to purely data-based [formulaic] methods. As such, users would avoid the more distinct, more physical issue of healthy cells literally burying the highly limited number of suspect cells, resulting in false negatives.
One of the leading companies in the field of Flow Cytometry at present is the Seattle-based Amnis Corporation. The three major foci of Amnis’ unique development of the tool have been Time Delay Integration (TDI), Extended Depth of Field (EDF), and Multispectral Imaging. The first of these works to allow Amnis’ devices to “[image] 1000X faster than standard imaging systems without loss of sensitivity” (Amnis, “Breakthrough”) by synchronizing the conversion of image photons into photocharges (the technique used by digital cameras to record images) with the velocity of cells in the fluidic system. The second factor, EDF, is rooted in “[projecting] all structures within the cell into one crisp plane of focus” (Amnis, “Extended”). This allows for more reliable analysis of smaller scale “spot counting” (Amnis, “Extended”) based analyses, such as nuclear translocation, or, in the case of AutoPap, multinucleation. Finally, the goal of multispectral imaging is to increase ability to analyze nuances in morphology and function by facilitating spectral decomposition of images. Thus, the output actually becomes separable into three distinct modes: “brightfield, darkfield, and fluorescence” (Amnis, “Multispectral”). This has great potential for considering unique parameters within samples without the use of additional hardware, such as spectrophotometers.
Of course, the true efficacy of Amnis’ tools can only be gauged by records of successful use for advanced, innovative purposes. With the enormous list of research utilizing Amnis flow cytometers available on Amnis’ website, it was ideal to select a case that involved a future pathological application. Such a case turned out to exist in the form of a J Immunol article by researchers at the University of Rochester Medical Center, titled “Characterization of platelet-monocyte complexes in HIV-1-infected individuals: possible role in HIV-associated neuroinflammation.” In essence, the conclusion of the study was that HIV-1 infections trigger an increase in the presence of “complexes between inflammatory monocytes and activated platelets” (Singh). These complexes express a distinct platelet marker apparent in flow cytometers. With such information, not only could researchers initially determine the presence of an HIV-1 infection, but they could further use it as basis for inhibiting platelet activation vital to the negative feedback loop that the aforementioned complexes are known for. Having done so, risks of neuroinflammation and neurologic deficit are reduced (Singh). Not only are pathologists able to diagnose a disease, but they are able to do so in such a way that gives them a direct initial treatment path.
With applications ranging from radiological analysis of not only the lungs, as described here, but also the brain, cytopathology, and any other form of cytometry or small-scale microscopy, image analysis holds great potential in terms of the greatest rising medical innovations. At present, with a basis in optimized algorithms for very specific conditions, such as AutoPap for cervical cancer, a universal goal in the industry is to facilitate customization by non-experts. The idea is to allow researchers with experience in entirely distinct disciplines to make use of this great equipment. Nevertheless, with ever cheaper and more efficient techniques of both diagnosing and treating diseases, it is essential for such research to serve as the next generation of innovation. By considering the question of whether costs can be lowered to the point of accessibility in the developing world, the increasing necessity for said tools provides a definitive future in this light. Development not only indicates more unique uses here in a research environment, but true change for healthcare internationally. Continuous development in this direction could someday entirely evolve pathology internationally.
Amnis Corporation. “Breakthrough Technology.” Amnis. Amnis Corporation, 2015. Web. 25 Feb. 2015.
Amnis Corporation. “Extended Depth of Field: How It Works.” Amnis. Amnis Corporation, 2015. Web. 26 Feb. 2015.
Amnis Corporation. “Multispectral Imaging.” Amnis. Amnis Corporation, 2015. Web. 26 Feb. 2015.
Amnis Corporation. “Time Delay Integration: How It Works.” Amnis. Amnis Corporation, 2015. Web. 25 Feb. 2015.
BD. “BD FocalPoint™ Slide Profiler.” BD. Becton, Dickinson and Company, 2015. Web. 22 Feb. 2015.
Clemex Technologies Inc. “Clemex Vision PE: The Most Complete Image Analysis Solution.” Clemex Intelligent Microscopy. Clemex Technologies Inc., 2015. Web. 25 Feb. 2015.
Imperial College, The Karolinska Institute, and Pomeranian Medical University. “Cervical Cytology.” Eurocytology. Eurocytology, n.d. Web. 08 Nov. 2014.
Luck, Randall L., and Richard Scott. Morphological Classification System and Method. Neuromedical Systems, Inc., assignee. Patent US 5257182 A. 26 Oct. 1993. Print.
Nelson, Alan C., and Shih-Jong J. Lee. Method for Testing Proficiency in Screening Images of Biological Slides. NeoPath, Inc., assignee. Patent US5797130 A. 18 Aug. 1998. Print.
Seo, Michael J. “AutoPap and Flow Cytometry.” Online interview. 10 Feb. 2015.
Shrivastava, Avanish, and Mohan Awasathy. “Strategic Approach to Image Block Segmentation.” International Journal of Scientific Research Engineering and Technology 2.10 (2014): 604-09. IJSRET. Jan. 2014. Web. 21 Feb. 2015.
Singh, M. V., D. C. Davidson, J. W. Jackson, V. B. Singh, J. Silva, S. H. Ramirez, and S. B. Maggirwar. “Characterization of Platelet–Monocyte Complexes in HIV-1–Infected Individuals: Possible Role in HIV-Associated Neuroinflammation.” J Immunol 192.10 (2014): 4674-684. PubMed. Web. 25 Feb. 2015.
Yeh, Hwa-Young M., Yuan-Ming F. Lure, and Jyh-Shyan Lin. Method and System for the Detection of Lung Nodule in Radiological Images Using Digital Image Processing and Artificial Neural Network. Deus Technologies, LLC., assignee. Patent US 6760468 B1. 6 July 2004. Print.