Retinal Vessel Centerline Extraction Using Multiscale Matched Filters, Confidence and Edge Measures

This page gives a high level overview of our advances in the retinal extraction work. For more details, please refer to our article in IEEE Transactions on Medical Imaging.

Contents

• Overview
• Introduction
• Algorithm
• Results
• Conclusion
• Bibliography
• Download Fast Tracing Executable
• Download Pixel-wise Detection Executable

Overview

Motivated by the goals of improving detection of low-contrast and narrow vessels and eliminating false detections at non-vascular structures, a new technique is presented for extracting vessels in retinal images. The core of the technique is a new likelihood ratio test that combines matched-filter responses, confidence measures and vessel boundary measures. Matched filter responses are derived in scale-space to extract vessels of widely varying widths. A vessel confidence measure is defined as a projection of a vector formed from a normalized pixel neighborhood onto a normalized ideal vessel profile. Vessel boundary measures and associated confidences are computed at potential vessel boundaries. Combined, these responses form a 6-dimensional measurement vector at each pixel. A training technique is used to develop a mapping of this vector to a likelihood ratio that measures the ``vesselness'' at each pixel. Results comparing this vesselness measure to matched filters alone and to measures based on the Hessian of intensities show substantial improvements both qualitatively and quantitatively. The Hessian can be used in place of the matched filter to obtain similar but less-substantial improvements or to steer the matched filter by preselecting kernel orientations. Finally, the new vesselness likelihood ratio is embedded into a vessel tracing framework, resulting in an efficient and effective vessel centerline extraction algorithm.

Introduction

Reliable vessel extraction is a prerequisite for subsequent retinal image analysis and processing because vessels are the predominant and most stable structures appearing in the images. Several challenges of vessel extraction in retinal images are illustrated by the images shown in Figures 1 and 2. These challenges may be outlined as follows:

• There is a wide range of vessel widths -- from less than a pixel to 12 pixels wide in the example shown.
• Vessels may be low contrast. The central intensity of some vessels differ from the background by as little as 4 grey levels, yet the background noise standard deviation is 2.3 grey levels. Narrow vessels often have the lowest contrast.
• A variety of structures appears in the images, including the retina boundary, the optic disc, and pathologies. The latter are particularly challenging for automatic vessel extraction because they may appear as a series of bright spots, sometimes with narrow, darker gaps in between.
• Wider vessels sometimes have a bright strip running down the center (the ``central reflex''), causing a complicated intensity cross-section. Locally, this may be hard to distinguish from two side-by-side vessels.

Our focus in this work is on techniques needed to solve the first three problems -- detecting low-contrast vessels and narrow vessels, while avoiding false responses near pathologies and other non-vascular structures.

Figure 1: Illustration of the challenges of retinal vessel extraction. Arrows drawn on the image [in yellow / with dashed lines] indicate pathologies, the boundary of the optic disc and the boundary of the retina, all of which tend to cause false positives in vessel detection. Arrows drawn [in white / with solid lines] highlight narrow or low-contrast vessels which are difficult to detect.

The limitations of some existing methods are illustrated in Figure 2. All these ``vesselness measures'' produce substantial responses to non-vascular structures such as the optic disc and the pathologies. All measures produce stronger responses at the boundary of the retina, near the optic disc, and along central pathological structures than for the thin and low-contrast vessels.

Figure 2: Illustration of the challenges of retinal vessel extraction on the image shown in Figure 1. (a) Matched filter response based on a Gaussian vessel profile; (b) Lindeberg's ridge measure [4]; (c) Frangi's Hessian-based vesselness measure [2]; and (d) Mahadevan's robust model-based measure [5]. All measures show spurious responses to non-vascular structures such as the fundus boundary, the optic disc boundary and pathologies. Edge response image (e) shows numerous edges to which the vesselness measures are sensitive. The proposed Likelihood Ratio Vesselness (LRV) measure (f) suppresses the response to the bright pathology on the right, highlights thin and low-contrast vessels (e.g. the vessel above the same pathology), and completely removes the boundary of the retina.

(a) Matched filter (b) Lindeberg (c) Frangi (d) Mahadevan (e) Edge responses (f) LRV measure

Algorithm

The main contribution of this paper is the development of an enhanced vesselness measure that addresses the issues illustrated in Figure 1. Motivated by the apparent effectiveness of the matched filter in highlighting low-contrast and narrow vessels and by recent success in using matched filters for retina vessel segmentation [3], we introduce a multi-scale matched filter for vessels, using an appropriate normalizing multiplier to allow the combination of responses across scales. We then augment the matched-filter responses with a new vessel ``confidence'' measure, analogous to the edge-based measure presented in [6]. It can be viewed as normalized cross-correlation or cosine distnace between model and sample vectors and determines how closely an image region follows an ideal vessel profile. Importantly, unlike the matched filter, this measure is independent of amplitude. To these vessel response and confidence values we add edge detection filter responses and confidences taken from the boundary of the purported vessel. This produces a six degree-of-freedom measurement vector at each pixel. Then, we use a training technique to develop a mapping from this vector to a single likelihood ratio that serves as the final ``vesselness'' measure. This gives a measure at each pixel which may be used either for segmentation of vessel pixels or for identifying the centerline vessel pixels and vessel widths when used in combination with non-maximum suppression. We focus on the latter because the measures are designed to have maximum response along the centerline of the vessel, and because this provides a more compact, geometric description of the vessels than segmentation alone.

We show that the new Likelihood Ratio Vesselness (LRV) measure outperforms the multiscale matched-filter and existing Hessian-based measures using quantitative analysis and using visual inspection of results on retinal images. We also show how the multiscale Hessian may be used to steer the matched filter by selecting its orientation at each pixel and scale. This way, the matched filter is applied only once at each pixel and scale, eliminating much of the excess computation. Alternatively, the Hessian may be used in place of the matched-filter in the LRV measure, producing substantial improvements in Hessian-based vesselness measures. The advantage of this is a lower overall computational cost than the matched-filter-based measure in exchange for a slight decrease in effectiveness.

Results

In our evaluation, we first quantify the improvement gained by using vessel confidences and edge measures together with the matched filter. We then compare detection results using LRV and Frangi's measure on all retinal vessels in an image. We then show qualitative comparison of the LRV measure and the vessel matched filter.

Quantitative Results

Our quantitative analysis uses overall results combined across all 20 images in the STARE data set (Figure 3). The use of the matched filter and its associated confidence -- a two-component measurement vector at each pixel -- in forming the likelihood ratio dramatically improves the performance of the matched filter, while adding the edge responses and confidences needed to form the complete 6-component vector provides significant further improvements. Both the Hessian-based (Frangi) and the matched-filter-based LRV measures substantially outperform the original measures, but the matched-filter LRV is clearly superior. Finally, the seemingly-strange non-monotonic shape of the matched-filter and Frangi measures alone is easily explained. The highest responses for these measures occur at the retina boundary, the optic disc boundary, and the boundary of pathologies (though perhaps slightly offset from the true boundary, as discussed above), because they are of much higher-contrast than even the more distinct vessels. Hence, for very high thresholds, these are the only responses that survive thresholding.

Figure 3: (a) (1-Precision)-Recall curves showing the effectiveness of the LRV measure. Comparisons are made between the full measure (LRV), the measure with only vessel confidences, and the multiscale matched filter alone. (1-Precision)-Recall curves are more suitable for comparison because of large number of negatives (non-vessel pixels) in the ground truth than ROC curves. Notice that the matched filter with vessel confidences is as powerful as the full LRV measure until about 50% of all traces are detected. Plot (b) compares the LRV measure (with the matched filter) to the LRV using Frangi's Hessian-based measure and to the matched filter and Frangi's measure alone.

(a) (b)

Qualitative Results - Pathologies

In order to give a qualitative feel for the results, several ``chips'' from the most challenging images are shown in Figure 4. These include a variety of pathologies and thin vessels that are so subtle they sometimes completely fade into the noise.

Figure 4: Results on 5 difficult image chips showing a variety of challenges, including pathologies, other non-vascular structures, and low-contrast, narrow vessels. The figure shows the source images in the 1st column, the vessel matched-filter response images after non-maximum suppression in the 2nd column, the Likelihood Ratio Vesselness (LRV) after non-maximum suppression in the 3rd column and the pixels that remain after thresholding the LRV measure at = 2.0 in the 4th column.

Vessel Tracing

Starting from a set of oriented seed locations, exploratory tracing algorithms recursively trace the vascular structure by stepping along the vessel tangent direction and locally detecting the next point on the vessel. Usually, the ``centerline'' location, vessel width, and tangent direction are estimated simultaneously. Tracing stops for a given seed point when it intersects a vessel already traced or when the vessel is not reliably detected. Our approach follows this strategy as well, with a small but important variation. The natural approach would be to use the 6-D measurement vector of matched-filter, confidence and edge responses, computed over a small neighborhood in location, orientation and scale to compute the likelihood ratio and then apply a threshold to decide the presence of a vessel. We have found, however, that this can lead to missed traces for the most subtle vessels, prematurely halting tracing. Instead we apply only the vessel matched filter during tracing. This is very sensitive and detects both subtle vessels as well as results in many traces at non-vessel locations. We then compute the 6-D measurement and apply the likelihood ratio subsequently to these ``over-traced'' results. In effect we are using the vessel matched filter in tracing to generate candidate locations at which to evaluate the LRV measure. This is less expensive than starting from large number of seed points and applying LRV measure during tracing.

Figure: 5 Source image (1st column), tracing results using parallel edge algorithm [1] (2nd column), Likelihood Ratio Vesselness (LRV) based tracing (3rd column) and the LRV after non-maximum suppression and thresholding below 2.0 (4th column). True positives are in blue, false negatives in red, and false positives in green. The new tracing algorithm successfully handles difficult vessel branching and crossover points (two overlapping vessels in the 3rd and 4th example), more accurately determines when to stop tracing in unclear regions (thin vessels near pathologies in the 1st and 3rd example), correctly ignores pathologies (1st and 4th example) and finds many of the thinnest vessels which usually have very low contrast (5th example). Notice in a number of cases false positives are marked alongside false negatives (green line along red line) because of localization differences with the ground truth. The tracing results are comparable to the LRV measure evaluated at every point. Vessels with fewer than 4 connected pixels were removed.

One final example that reinforces the effectiveness of LRV tracing is shown in Figure 6 where both tracing algorithms are applied to an image from a slit-lamp biomicroscope. The likelihoods are powerful enough to highlight vessels but ignore most of the background noise. This noise causes detection of spurious traces by the parallel-edge based algorithm. The new tracing algorithm correctly finds vessels that are missed by the parallel-edge based tracing.

Figure 6: Example of a slit lamp image (top left), LRV measure computed image wide (top right), parallel-edge based tracing (bottom left), and LRV-based tracing (bottom right). The LRV measure is powerful enough to highlight vessels but ignore the background noise which is the main reason for spurious traces of the parallel-edge based algorithm. The new tracing algorithm correctly finds vessels that are missed by the parallel-edge based tracing.

Conclusion

Each component of the Likelihood Ratio Vesselness (LRV) measure is designed to help address the problem of detecting narrow and low-contrast vessels, while avoiding responses to other retinal image structures. In particular, the elongated template of the multiscale matched filter tends to preserve vessels that are only a pixel wide (and typically low-contrast), whereas isotropic measures such as the Hessian tend to substantially blur these structures. The edge responses are useful in distinguishing between offset edges near pathologies and true vessels. The confidence measures emphasize the shape of the intensity surface rather than the magnitude of the responses, enhancing the ability of the LRV to detect low-contrast vessels. The 6-component measurement vectors are mapped to the final LRV measure using pdfs learned from training data. Significantly, after designing multiscale filtering and training procedures there are no run time tuning parameters in the computation of this measure. The computation is made more efficient by using the Hessian to select matched-filter orientations and by embedding the LRV measure in a tracing framework. The latter substantially outperforms our existing retinal vessel tracing algorithm. Both quantitative and qualitative analysis on challenging retinal images have shown the effectiveness of the LRV measure. The new measure may be used in place of the Hessian and the matched filter in existing vessel detection and segmentation algorithms [2,7]. Based on the experimental evaluations reported here, this should lead to substantially-improved results. The fact that the Hessian may be used to steer the application of the matched-filter and confidence measures makes the computation tractable.

Publications and Further Reading

1. Sofka, M., V. Stewart, C., 2006. Retinal Vessel Extraction Using Multiscale Matched Filters, Confidence and Edge Measures. IEEE Transactions on Medical Imaging 25, 1531–1546.
   abstract bibtex

   Motivated by the goals of improving detection of low-contrast and narrow vessels and eliminating false detections at non-vascular structures, a new technique is presented for extracting vessels in retinal images. The core of the technique is a new likelihood ratio test that combines matched-filter responses, confidence measures and vessel boundary measures. Matched filter responses are derived in scale-space to extract vessels of widely varying widths. A vessel confidence measure is defined as a projection of a vector formed from a normalized pixel neighborhood onto a normalized ideal vessel profile. Vessel boundary measures and associated confidences are computed at potential vessel boundaries. Combined, these responses form a 6-dimensional measurement vector at each pixel. A training technique is used to develop a mapping of this vector to a likelihood ratio that measures the “vesselness” at each pixel. Results comparing this vesselness measure to matched filters alone and to measures based on the Hessian of intensities show substantial improvements both qualitatively and quantitatively. The Hessian can be used in place of the matched filter to obtain similar but less-substantial improvements or to steer the matched filter by preselecting kernel orientations. Finally, the new vesselness likelihood ratio is embedded into a vessel tracing framework, resulting in an efficient and effective vessel centerline extraction algorithm.

   @article{sofka:tmi06,
     author = {Sofka, Michal and V.\ Stewart, Charles},
     title = {Retinal Vessel Extraction Using Multiscale Matched
                       Filters, Confidence and Edge Measures},
     journal = {IEEE Transactions on Medical Imaging},
     year = {2006},
     volume = {25},
     number = {12},
     pages = {1531--1546},
     month = dec,
     url = {http://www.sofka.com/vessels.html}
   }
   

   
   

Bibliography

[1] A. Can, H. Shen, J. N. Turner, H. L. Tanenbaum, and B. Roysam. Rapid automated tracing and feature extraction from live high-resolution retinal fundus images using direct exploratory algorithms. IEEE Trans. Inform. Technol. Biomed., 3(2):125-138, 1999.

[2] A. Frangi, W. J. Niessen, K. L. Vincken, and M. A. Viergever. Multiscale vessel enhancement filtering. In Proc. 1st MICCAI, pages 130-137, 1998.

[3] A. Hoover, V. Kouznetsova, and M. Goldbaum. Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans. Med. Imag., 19(3):203-210, 2000.

[4] Tony Lindeberg. Edge detection and ridge detection with automatic scale selection. Int. J. Comp. Vis., 30:117-156, November 1998.

[5] V. Mahadevan, H. Narasimha-Iyer, B. Roysam, and H.L. Tanenbaum. Robust model-based vasculature detection in noisy biomedical images. IEEE Trans. Inform. Technol. Biomed., 8(3):360-376, 2004.

[6] Peter Meer and Bogdan Georgescu. Edge detection with embedded confidence. IEEE Trans. Pattern Anal. Machine Intell., 23(12):1351-1365, December 2001.

[7] J.J. Staal, M.D. Abramoff, M. Niemeijer, M.A. Viergever, and B. van Ginneken. Ridge based vessel segmentation in color images of the retina. IEEE Trans. Med. Imag., 23(4):501-509, Apr 2004.