Three-Dimensional Vessel Segmentation in Whole-Tissue and Whole-Block Imaging Using a Deep Neural Network
2020; Elsevier BV; Volume: 191; Issue: 3 Linguagem: Inglês
10.1016/j.ajpath.2020.12.008
ISSN1525-2191
AutoresTakashi Ohnishi, А. А. Теплов, Noboru Kawata, Kareem Ibrahim, Peter Ntiamoah, Canan Fırat, Hideaki Haneishi, Meera Hameed, Jinru Shia, Yukako Yagi,
Tópico(s)Medical Imaging Techniques and Applications
ResumoIn the field of pathology, micro–computed tomography (micro-CT) has become an attractive imaging modality because it enables full analysis of the three-dimensional characteristics of a tissue sample or organ in a noninvasive manner. However, because of the complexity of the three-dimensional information, understanding would be improved by development of analytical methods and software such as those implemented for clinical CT. As the accurate identification of tissue components is critical for this purpose, we have developed a deep neural network (DNN) to analyze whole-tissue images (WTIs) and whole-block images (WBIs) of neoplastic cancer tissue using micro-CT. The aim of this study was to segment vessels from WTIs and WBIs in a volumetric segmentation method using DNN. To accelerate the segmentation process while retaining accuracy, a convolutional block in DNN was improved by introducing a residual inception block. Three colorectal tissue samples were collected and one WTI and 70 WBIs were acquired by a micro-CT scanner. The implemented segmentation method was then tested on the WTI and WBIs. As a proof-of-concept study, our method successfully segmented the vessels on all WTI and WBIs of the colorectal tissue sample. In addition, despite the large size of the images for analysis, all segmentation processes were completed in 10 minutes. In the field of pathology, micro–computed tomography (micro-CT) has become an attractive imaging modality because it enables full analysis of the three-dimensional characteristics of a tissue sample or organ in a noninvasive manner. However, because of the complexity of the three-dimensional information, understanding would be improved by development of analytical methods and software such as those implemented for clinical CT. As the accurate identification of tissue components is critical for this purpose, we have developed a deep neural network (DNN) to analyze whole-tissue images (WTIs) and whole-block images (WBIs) of neoplastic cancer tissue using micro-CT. The aim of this study was to segment vessels from WTIs and WBIs in a volumetric segmentation method using DNN. To accelerate the segmentation process while retaining accuracy, a convolutional block in DNN was improved by introducing a residual inception block. Three colorectal tissue samples were collected and one WTI and 70 WBIs were acquired by a micro-CT scanner. The implemented segmentation method was then tested on the WTI and WBIs. As a proof-of-concept study, our method successfully segmented the vessels on all WTI and WBIs of the colorectal tissue sample. In addition, despite the large size of the images for analysis, all segmentation processes were completed in 10 minutes. Whole-slide imaging has gained importance as an imaging modality in pathology because it enables analog to digital conversion for integration with image analysis and/or artificial intelligence in future diagnosis. However, digitization in the field of pathology lags behind other medical imaging modalities because analog observation with microscope remains popular among many pathologists. More than two decades of research have revealed important roles of whole-slide image (WSI), among which three-dimensional (3D) imaging is a symbolic achievement.1Casero R. Siedleck U. Jones E.S. Gruscheski L. Gibb M. Schneider J.E. Kohl P. Grau V. Transformation diffusion reconstruction of three-dimensional histology volumes from two-dimensional image stacks.Med Image Anal. 2017; 38: 184-204Crossref PubMed Scopus (12) Google Scholar, 2Gaffling S. Daum V. Hornegger J. Landmark-constrained 3-D histological imaging: a morphology-preserving approach.in: Vision, Modeling, and Visualization. Geneva, Switzerland, Eurographics Association2011: 309-316Google Scholar, 3Schwier M. Böhler T. Hahn H.K. Dahmen U. Dirsch O. Registration of histological whole slide images guided by vessel structures.J Pathol Inform. 2013; 4 Suppl: S10Crossref Google Scholar, 4Cifor A. Bai L. Pitiot A. Smoothness-guided 3-D reconstruction of 2-D histological images.NeuroImage. 2011; 56: 197-211Crossref PubMed Scopus (34) Google Scholar, 5Xu Y. Pickering G. Nong Z. Glibson E. Arpino J.M. Yin H. Ward A.D. A method for 3D histopathology reconstruction supporting mouse microvasculature analysis.PLoS One. 2015; 10: e0126817Crossref PubMed Scopus (23) Google Scholar, 6Roberts N. Magee D. Song Y. Brabazon K. Shires M. Crellin D. Orsi N.M. Quirke R. Quirke P. Treano D. Toward routine use of 3D histopathology as a research tool.Am J Pathol. 2012; 180: 1835-1842Abstract Full Text Full Text PDF PubMed Scopus (93) Google Scholar However, whole tissues from the patient cannot be examined because in accordance with institutional regulations, 50% of the tumor volume must remain untouched after the study. In addition, WSI-based 3D imaging is time-consuming, labor intensive, and costly. Therefore, to realize a clinical application for pathology, a noninvasive and simple 3D imaging technique is required. In recent years, micro–computed tomography (micro-CT) has become an attractive imaging modality because it enables full analysis of the 3D characteristics of a tissue sample or organ in a noninvasive manner.7Teplov A. Tabata K. Fu X. Uraoka N. Roehr H.A. Ntiamoah P. Humm J.L. Sirintrapun S.J. Murray M.P. Shia J. Travis W.D. Brogi E. Hameed M. Yagi Y. Development of standard operating procedure (SOP) of micro-computed tomography (micro-CT) in pathology.Diagn Pathol. 2019; 5https://doi.org/10.17629/www.diagnosticpathology.eu-2019-5:273Crossref Google Scholar, 8Ketsamenis O.L. Olding M. Wamer J.A. Chatelet D.S. Jones M.G. Sgalla G. Smit B. Larkin O.J. Haing I. Richeldi L. Sinclair I. Lackie P.M. Schnider P. X-ray micro-computed tomography for nondestructive three-dimensional (3D) X-ray histology.Am J Pathol. 2019; 189: 1608-1620Abstract Full Text Full Text PDF PubMed Scopus (21) Google Scholar, 9Troschel F.M. Gottumkkala R.V. DiCorpo D. Mario J. Ott H.C. Wright C.D. Muniappan A. Lanuti M. Yang K. JO Shepard Nardi V. Michaelson J.S. Hariri L.P. Fintelmann F.J. Feasibility of perioperative micro-computed tomography of human lung cancer specimens.Arch Pathol Lab Med. 2019; 143: 319-325Crossref PubMed Scopus (6) Google Scholar, 10Albers J. Markus M.A. Alves F. Dullin C. X-ray based virtual histology allows guided sectioning of heavy ion stained murine lungs for histological analysis.Sci Rep. 2018; 8: 7712Crossref PubMed Scopus (16) Google Scholar, 11Tanabe N. Vasilescu D.M. Kirby M. Coxson H.O. Verleden S.E. Vanaudenaerde B.M. Kinose D. Nakano Y. Paré P.D. Hogg J.C. Analysis of airway pathology in COPD using a combination of computed tomography, micro-computed tomography and histology.Eur Respir J. 2018; 51: 1701245Crossref PubMed Scopus (45) Google Scholar, 12Mai C. Verleden S.E. McDonough J.E. Willems S. Wever W.D. Coolen J. Dubbeldam A. Raemdonck D.E.V. Verbeken E.K. verleden G.M. Hogg J.C. Vanaudenaerde B.M. Wuyts W.A. Verschakelen J.A. Thin-section CT features of idiopathic pulmonary fibrosis correlated with micro-CT and histologic analysis.Radiology. 2017; 283: 252-263Crossref PubMed Scopus (39) Google Scholar The imaging principle is the same as that of conventional clinical CT. Although the field of view is much smaller, microscale images of high spatial resolution can be obtained. The best spatial resolution of micro-CT image is inferior to that of WSI with ×40 magnification; however, micro-CT images have potential as a substitute for low-magnification WSI because they enable observation of the morphologic structure of the entire tissue noninvasively. Teplov et al7Teplov A. Tabata K. Fu X. Uraoka N. Roehr H.A. Ntiamoah P. Humm J.L. Sirintrapun S.J. Murray M.P. Shia J. Travis W.D. Brogi E. Hameed M. Yagi Y. Development of standard operating procedure (SOP) of micro-computed tomography (micro-CT) in pathology.Diagn Pathol. 2019; 5https://doi.org/10.17629/www.diagnosticpathology.eu-2019-5:273Crossref Google Scholar examined fresh samples, fixed samples, and formalin-fixed, paraffin-embedded (FFPE) block samples of various organs. They developed operational procedures and effective imaging parameters for fresh, fixed, and FFPE block samples; in addition, they found that the X-ray dose from micro-CT scanning does not affect protein expression. In their study of interstitial pneumonia, Ketsamenis et al8Ketsamenis O.L. Olding M. Wamer J.A. Chatelet D.S. Jones M.G. Sgalla G. Smit B. Larkin O.J. Haing I. Richeldi L. Sinclair I. Lackie P.M. Schnider P. X-ray micro-computed tomography for nondestructive three-dimensional (3D) X-ray histology.Am J Pathol. 2019; 189: 1608-1620Abstract Full Text Full Text PDF PubMed Scopus (21) Google Scholar demonstrated that the small airways, blood vessels, and alveoli of the lungs could be clearly visualized on micro-CT. Albers et al10Albers J. Markus M.A. Alves F. Dullin C. X-ray based virtual histology allows guided sectioning of heavy ion stained murine lungs for histological analysis.Sci Rep. 2018; 8: 7712Crossref PubMed Scopus (16) Google Scholar scanned murine lung samples with free-propagation phase-contrast micro-CT and reported that micro-CT has the potential to improve workflow aspects of histologic analysis, such as determination of the cutting position. Xu et al13Xu B. Teplov A. Ibrahim K. Inoue T. Stueben B. Kitabi N. Hameed M. Yagi Y. Ghosein R. Detection and assessment of capsular invasion, vascular invasion and lymph node metastasis volume in thyroid carcinoma using microCT scanning of paraffin tissue blocks (3D whole block imaging): a proof of concept.Mod Pathol. 2020; 33: 2449-2457Crossref PubMed Scopus (5) Google Scholar assessed thyroid carcinoma using micro-CT images of paraffin tissue blocks. According to their report, capsular invasion and vascular invasion could be detected and volume of nodal metastasis could be assessed by micro-CT images, even if the images were not present on serially sectioned hematoxylin and eosin (H&E)–stained slides. In addition, they mentioned that the volume of the metastatic foci measured by micro-CT image differed by 13.125-fold compared with that measured by WSI of H&E slide. The reported spatial resolution of a micro-CT image of an FFPE block sample is approximately 5 to 10 μm,7Teplov A. Tabata K. Fu X. Uraoka N. Roehr H.A. Ntiamoah P. Humm J.L. Sirintrapun S.J. Murray M.P. Shia J. Travis W.D. Brogi E. Hameed M. Yagi Y. Development of standard operating procedure (SOP) of micro-computed tomography (micro-CT) in pathology.Diagn Pathol. 2019; 5https://doi.org/10.17629/www.diagnosticpathology.eu-2019-5:273Crossref Google Scholar,8Ketsamenis O.L. Olding M. Wamer J.A. Chatelet D.S. Jones M.G. Sgalla G. Smit B. Larkin O.J. Haing I. Richeldi L. Sinclair I. Lackie P.M. Schnider P. X-ray micro-computed tomography for nondestructive three-dimensional (3D) X-ray histology.Am J Pathol. 2019; 189: 1608-1620Abstract Full Text Full Text PDF PubMed Scopus (21) Google Scholar which is comparable to that of WSI (up to ×10.0 magnification) used for evaluation of morphologic structure. To three-dimensionally visualize the FFPE block sample with 4-mm–thick tissue by WSI, approximately 1000 slides must be generated; however, micro-CT can achieve it without sectioning and tissue exhaustion. The findings of these previous studies suggest that, in terms of clinical diagnosis, micro-CT is a useful tool that can provide 3D structural tissue information that enriches the histopathologic diagnosis. However, because it is difficult to interpret the vast amount of 3D information, there is a need for analytical methods, such as those implemented in the software used with clinical CT. The most highly demanded technique is the accurate identification of each tissue component. Tissue identification has great potential for examining the extension of each component feature and the relationship between these components. For example, vessel extension or tumor growth can be analyzed after tissue identification, thus providing the pathologist with additional diagnostic information that enables a more accurate morphologic assessment. Several previous studies have reported the development of tissue identification techniques. Wollatz et al14Wollatz L. Johnston S.J. Lackie P.M. Cox S.J. 3D histopathology—a lung tissue segmentation workflow for microfocus X-ray-computed tomography scans.J Digit Imaging. 2017; 30: 772-781Crossref PubMed Scopus (7) Google Scholar built a semi-automatic segmentation workflow with open-source software for segmenting airways and blood vessels from lung tissue samples. Although their system could run on a standard personal computer, it needed an improvement in performance. Zhao et al15Zhao F. Gao P. Hu H. He X. Hou Y. He X. Efficient kidney segmentation in micro-CT based on multi-atlas registration and random forests.IEEE Access. 2018; 6: 43712-43723Crossref Scopus (10) Google Scholar developed a kidney segmentation method based on multi-atlas registration and random forests, and applied it to murine micro-CT images. Although their method shows great potential for segmentation purposes, it will not be easily applied to clinical histopathologic diagnosis as it was developed specifically for analysis of murine kidney. While there is high demand for the development of various image analyses, this manuscript focuses on vessel segmentation because there is strong interest in the 3D structural relationships between vessels and tumors in many clinical fields. Sometimes a contrast between vessel and other tissues in the CT is low, depending on the object. In such case, angiography, which uses a contrast agent, has been generally applied to enhance the vessel region.16Lee J. Beighley P. Ritman E. Smith N. Automatic segmentation of 3D micro-CT coronary vascular images.Med Image Anal. 2007; 11: 630-647Crossref PubMed Scopus (77) Google Scholar Classic methods, such as Hessian matrix–based method,16Lee J. Beighley P. Ritman E. Smith N. Automatic segmentation of 3D micro-CT coronary vascular images.Med Image Anal. 2007; 11: 630-647Crossref PubMed Scopus (77) Google Scholar make it difficult to segment the vessel from low-contrast micro-CT images. Because one of our targets is FFPE block, generated according to the standard clinical procedure, use of the additional angiogram step is not assumed. Therefore, to achieve highly accurate segmentation on the micro-CT image without the contrast agent, we constructed a deep neural network (DNN), which has become a widely used technique in recent years. As a proof-of-concept study, we focused on the diagnosis of colorectal cancer and performed vessel segmentation on three micro-CT images acquired from one colorectal tissue sample. The following sections will explain the image acquisition procedure, outline the network structure for vessel segmentation, and report the experimental results. Figure 1 outlines the procedure developed for application of micro-CT to clinical histopathologic diagnosis. A resected fresh or fixed tissue sample is examined by micro-CT scanner in low-resolution mode. The spatial resolution of the obtained micro-CT image is ≥50 μm. The micro-CT image of those samples is called a whole-tissue image (WTI) in this study. FFPE block samples are prepared, and these are also examined by micro-CT scanner. The micro-CT image of the FFPE block sample is termed a whole-block image (WBI). After micro-CT examinations, stained slide samples are prepared and digitized by whole-slide scanners. All acquired micro-CT images and whole-slide images are stored in the image server linked with each scanner, and synchronized with a storage server via a database server. The database server manages the synchronized images on the storage server and associates the study identifier and the corresponding images. The pathologist searches and views the arbitrary images through the database server. If image analysis is requested from a client personal computer, the database server submits the tasks to a high-performance computing server and provides the analysis results to the client personal computer. Figure 2 shows the data flow from tissue sample attainment to performance evaluation in this study. Three resected colorectal tissue samples were used in the experiment. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study. Of these, two fixed tissue samples were divided into several sections along their long axis, and each dissected colorectal tissue sample was divided into two block samples. Each block sample underwent tissue processing and was then embedded into a cassette. Because the block samples were larger than the standard block cassette (28 mm × 40 mm) used for colorectal tissue samples, according to the standard procedure of our institute, we used whole-mount block cassettes of 75 mm × 52 mm. This procedure allowed us to reduce the number of samples to be scanned and to extend the field of view on micro-CT. Forty FFPE blocks were then immobilized with a carbon fiber paddle and individually examined by micro-CT scanner, resulting in 40 WBIs. After micro-CT image acquisition, the FFPE blocks were thin sliced. All slices were deparaffinized, stained using hematoxylin and eosin, and then scanned by a whole-slide scanner. The resulting images were termed whole-mount slide images (WMIs). Because following the annotation process was a highly time-consuming task, 10 pairs of WBIs and WMIs were selected to generate a data set for network training. As the original WBIs were too large and the vessels were sparsely located, 25 arbitrary-sized volumes containing vessels were manually extracted from WBIs. The average size of the extracted volumes was 200 × 200 × 80 voxels. Vessels with 2 to 15 voxel diameters were annotated as the ground truth (GT) on each extracted volume by two individuals, with reference to the corresponding WMI. Annotation was complete when there was agreement between the annotators. One remaining fixed tissue sample was immobilized in the polystyrene foam container and examined by micro-CT scanner after formalin fixation, resulting in one WTI. A total of 30 FFPE blocks were obtained from the fixed tissue sample and examined by micro-CT scanner. For cost and labor reasons, two WBIs (B1 and B2) were selected and used to evaluate the performance of our method. These two WBIs were selected because the tissue blocks were unadjacent, and had plenty of vessels versus those in the other WBIs. After micro-CT examination, 20 consecutive H&E slices were obtained from the FFPE blocks corresponding to B1 and B2, and their WMIs were acquired. GT for the performance evaluation was obtained by manual annotation on WMIs, and corresponding slices of WBIs to WMIs were also manually selected. Annotators were the same individuals who generated the GTs for the training data set. As it is extremely time-consuming to annotate all vessels on a WMI, in each case we focused on an area of high-vessel density, and evaluated performance on the area. The micro-CT imaging system comprises a custom-built micro-CT scanner (Nikon Metrology NV, Hertfordshire, UK) and CT Pro image reconstruction software version 2.0 (Nikon Metrology NV). The following scan parameters were adjusted for WTI and WBI, as determined previously7Teplov A. Tabata K. Fu X. Uraoka N. Roehr H.A. Ntiamoah P. Humm J.L. Sirintrapun S.J. Murray M.P. Shia J. Travis W.D. Brogi E. Hameed M. Yagi Y. Development of standard operating procedure (SOP) of micro-computed tomography (micro-CT) in pathology.Diagn Pathol. 2019; 5https://doi.org/10.17629/www.diagnosticpathology.eu-2019-5:273Crossref Google Scholar: beam energy, beam current, exposure time, number of projections, frames per projection, and the distance from the X-ray source to the sample (Table 1). For fixed tissues, scan duration was set to 15 minutes, and the best combination of scan parameters was investigated. After every micro-CT image examination, we confirmed successful image reconstruction by visual assessment using VGSTUDIO MAX image analysis software version 1.2.1 (Volume Graphics, Heidelberg, Germany). The whole-mount slides were scanned by NanoZoomer S60 (Hamamatsu Photonics K.K., Hamamatsu, Japan) in ×20 magnification mode. In this mode, pixel size was 0.46 × 0.46 μm2. For each WBI case, we selected the four WMIs that had the greatest similarity with the WBI slices. The same annotators in the GT generation visually evaluated the similarity between WMI and WBI slices by focusing on the position and shape of the vessels. The WMI positions were manually corrected by referring to the corresponding WBI slice. The image format of the WMIs was converted from NDPI to BMP on an image viewer (NDP.view2; Hamamatsu Photonics K.K.). Three pairs of corresponding coordinates were selected manually from a slice of WBI and WMI, and transformation parameters based on affine transformation with shift, rotation, and flip operations17Chang S. Cheng F. Hsu W. Wu G. Fast algorithm for point pattern matching: invariant to translations, rotations and scale changes.Pattern Recognit. 1997; 30: 311-320Crossref Scopus (147) Google Scholar were calculated. WMIs were transformed by calculated transformation parameters using ImageJ software version 1.52a (NIH, Bethesda, MD; https://imagej.nih.gov/ij).Table 1Acquisition Parameters of Micro-CTObjectBeam energy, kVBeam current, μAExposure time, millisecondsProjections, nFrames per projectionSource-to-object distance, mmFixed tissue9022225018012325FFPE block70100325048244120–175FFPE, formalin fixed, paraffin embedded; Micro-CT, micro–computed tomography. Open table in a new tab FFPE, formalin fixed, paraffin embedded; Micro-CT, micro–computed tomography. In the present study, we implemented a vessel segmentation function using DNN. A U-Net18Ronneberger O. Fischer P. Brox T. U-Net convolutional networks for biomedical image segmentation.2015Crossref Scopus (21534) Google Scholar–like network was created (Figure 3). Inspired by V-Net19Milletari F. Navab N. Ahmadi S.A. V-Net fully convolutional neural networks for volumetric medical image segmentation.2016Crossref Scopus (3098) Google Scholar and 3D U-Net,20Çiçek Ö. Abdulkadir A. Lienkamp S.S. Brox T. Ronneberger O. 3D U-Net: learning dense volumetric segmentation from sparse annotation.2016Google Scholar volumetric semantic segmentation was constructed. These previously reported DNNs used the same sized convolution kernels at each convolutional block. Many convolutional blocks must be stacked and repeated for segmenting thick vessels. Particularly in the case of 3D DNN, this greatly increases the number of adjustable parameters and leads to low processing efficiency and a long training time. Accordingly, our convolutional block design aimed to extend the convolution area and increase the variation of the convolution processes while limiting the number of parameters. First, input patch images were multichannelized by convolution layer with a 1 × 1 × 1 kernel and C channels. The convolutional block and down convolution were then conducted recurrently. The convolutional block, four convolution paths, consisting of small convolution layers, were built (Figure 4) . The input feature volumes were divided into four groups along a channel dimension, and then input into each convolution path. This adopted structure was inspired by residual inception block21Szegedy C. Loffe S. Vanhoucke V. Alemi A. Inception-v4, inception-ResNet and the impact of residual connections on learning.2016Google Scholar to extend the effective area and increase the convolution styles. If the input feature volumes had 4c (= C) channels, standard 3 × 3 × 3 and 5 × 5 × 5 convolutions adjusted 112c and 504c parameters, respectively. However, the number of adjustable convolutional parameters in our convolutional block became 138c. Although the effective area of the convolution block was a little sparse, it was maximally 9 × 9 × 9 voxels, and the number of parameters was similar to the standard 3 × 3 × 3 convolution. The concatenated four kinds of convolution results were added with the input feature volumes. Batch normalization22Ioffe S. Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift.2015Google Scholar and leaky-rectified linear unit23Maas A.L. Hannun A.Y. Ng A.Y. Rectifier nonlinearities improve neural network acoustic models.Proc ICML. 2013; 30Google Scholar activation were conducted after the addition process. Batch normalization was defined as follows:xi¯=xi−μBσB2+ε,(1) where xi and xi¯ represent layer inputs and standardized layer inputs, respectively; and μB, σB, and ε represent the mean and SD of the batch and a small constant value, such as 1.0 × 10−3, respectively. The layer output is obtained with scale and shift operations as yi=γxi¯+β. The γ and β represent trainable scale and bias parameters, respectively. Leaky-rectified linear unit is a rectified linear unit base activation function, defined as follows:f(x)={x ifx>0,0.3x otherwise(2) The slope parameter was originally proposed as 0.01 but we used 0.3, the default value of Keras API (Google LLC, Mountain View, CA), to implement the DNN. The down convolution provided twofold channels of downsized feature volumes by applying the 2 × 2 × 2 kernel and stride 2. After bottom convolutional block, up convolution consisted of up sampling with linear interpolation, and convolution was conducted with a 1 × 1 × 1 kernel. The number of channels was halved through the up-convolution process. Up-convolved results were concatenated with feature volumes provided at the same depth, after which convolutional block was executed again. After repeating up convolution and convolutional block, the feature volumes were dechannelized into one channel and activated with a sigmoid function. The activation result was used as a predicted value. The loss between the predicted value and the GT was calculated by binary cross entropy, as described by the following equation:H=−1N∑i=0N{yi·log[p(yiˆ)]+(1−yi)·log[1−p(yiˆ)]}(3) where y, yˆ represent the GT having 0 or 1, respectively; and predicted value p(·) represents the probability of the vessel for all N voxels. Herein, batch normalization and leaky-rectified linear unit layers were executed after every convolution process in multichannel, down-convolution, up-convolution, and dechannel layers. In general, although the inception module reduces the number of parameters, DNN still uses large amounts of graphics processing unit memory to analyze the 3D structure. Therefore, the patch image size must be small, which means that large-caliber vessels cannot be segmented by our DNN. To solve this problem, a multiresolution strategy was adapted in the prediction step. A Gaussian blur filter with σ = 1 was applied to the original image. An image pyramid was then generated from the original image, and patch images to be input to the network were extracted from the image at each resolution level. The network outputs at low resolution levels were up sampled to the original resolution with linear interpolation. The final segmentation result was obtained by applying the voxel-wise maximum operation with all up-sampled results. In machine learning field, intensity normalization or standardization is generally applied for all data before network input. However, because the range of intensity strongly varied with each WBI, the conventional method+, using mean intensity and SD, was unsuitable for this study. Therefore, several vessel intensities were measured manually on each WBI by using ImageJ software, and a mean intensity of the vessel was calculated. All intensities of each WBI were then divided by the mean intensity. After standardization, the vessel intensities on WBI approached 1. Small patch images with 64 × 64 × 32 voxels were randomly extracted from the standardized volumes for training. Contrast control, rotation, and flip processes were conducted as data augmentation during the extraction process. Contrast was adjusted simply, according to the following equation:Vout=aVin+b(4) where Vin and Vout represent input and output intensities, respectively. The a and b represent adjustive parameters that were determined randomly for each patch image. The ranges of a and b were from 0.95 to 1.05 and from −0.05 to 0.05, respectively, and the distribution of random numbers was uniform. Rotation of 0, 90, 180, and 270 degrees was conducted along the z axis only. Flip was conducted for each axis. Execution of flip for each axis was determined randomly. The implementation environment was as follows: Intel Xeon E5-2698 version 4 2.2 GHz with 20 cores, 80 GB RAM, and 4-Tesla V100 with 5120 cores and 16 GB RAM (Nvidia, Santa Clara, CA). The proposed network was implemented by Tensorflow 1.14.0 (Google LLC) and Keras 2.2.5 (Google LLC) in a Python 3.6 (Python Software Foundation, Beaverton, OR) environment. The number of channels in a multichannel layer was empirically set to 20. The numbers of network depth and convolutional blocks at each time were 4 and 4, respectively. Adam optimizer was used for network training, and the initial learning rate was 1.0 × 10−5. The batch size was set to 48. A total of 100,000 epochs were conducted with a train-on-batch mode, and batch was updated on every epoch. Therefore, total training time was approximately 4 days. Performance evaluation was conducted with true positive rate (TPR) and positive predictive value (PPV), defined as follows:TPR=TP/(TP+FN),(5) PPV=TP/(TP+FP),(6) where TP, FN, and FP represent the numbers of true positives, false negatives, and false positives, re
Referência(s)