Computer Vision and Human Computation for Cell Image Analysis -- A Project at Boston University

Click here to access our cell data
Click here to jump to down to newest work - 6 papers in 2016

The Cell Imaging Project at Boston University started as a collaboration between Margrit Betke from Boston University's Department of Computer Science and Joyce Wong from Boston University's Department of Biomedical Engineering and their former students Danna Gurari, David House, Diane Theriault, and Matthew Walker in 2008. The project has been funded by NSF since Fall 2009 as part of a larger effort to develop intelligent tracking systems that reason about group behavior and specifically for studying how to involve "Humans in the Loop to Collect High-quality Annotations from Images and Time-lapse Videos of Cells" since 2014.

2008 Project on Collection of Phase-Contrast Microscopy Image Data and Development of Cell Tracking Algorithms

We collected time-lapse microscopy videos of various mammalian cells. The data includes image sequences of hundreds of Balb/c fibroblast cells on polyacrylamide hydrogel substrates with varying mechanical properties and on tissue culture plastic substrates. The living cells were seeded on the substrates at a density of 1,000 cells/cm^2. The images of 1300 by 1030 pixels were acquired with a Princeton Instruments D1299421 camera mounted on a Zeiss Axiovert S100 microscope over the course of 10-24 hours.

We first experimented with an interval between images of 15 minutes and then extended our capture rate to 30 seconds. It established ground-truth classification of cell shape for 800 images with about 2-10 cells per image. The class label indicate whether a cell is "spread," "non-spread," "polarized," etc. The group is in the process of developing a machine learning approach to classify cell shape in single images of cells.

We established ground-truth motion trajectories in image sequences of over 400 cells and evaluated the performance of various tracking algorithms, including algorithms that evaluate cell shape. To solve the data association problem, the assignment of current measurements to cell tracks, we tested various cost functions with both an optimal and a fast, suboptimal assignment algorithm (see video below).

We measured how cell populations respond to the physical stimuli presented in the environment, for example, the stiffness property of the substrate. The analysis of hundreds of spatio-temporal cell trajectories revealed significant differences in the behavioral response of fibroblast cells to changes in hydrogel conditions.

We proposed metrics to quantify cell migration properties, such as motility and directional persistence. The group is in the process of comparing actual measurements of cell movement with a theory, the "random walk model," which has been used widely in the literature to describe cell migration.

Our results were published this paper:
D. House, M. L. Walker, Z. Wu, J. Y. Wong, and M. Betke. "Tracking of cell populations to understand their spatio-temporal behavior in response to physical stimuli." In MMBIA 2009: IEEE Computer Society Workshop on Mathematical Methods in Biomedical Image Analysis, Miami, FL, June 2009. 8 pp. pdf.
and discussed in a conference poster
M. L. Walker, D. M. House, M. Betke, and J. Y. Wong. "Using automated cell tracking tools to quantify durokinesis and durotaxis in real time." Proceedings of the Biophysical Society 53rd Annual Meeting, Boston, MA, USA, February 2009. Abstract.

2010 Project on Determining the Period in the Life Cycle of a Cell based on Shape

Cell shape analysis plays an important role for studying biological processes, developing biomaterials, and diagnosing and fighting diseases. We used machine learning techniques to classify cells based on their shape:

D. H. Theriault, M. Walker, J. Y. Wong, and M. Betke. "Cell Morphology Classification and Clutter Mitigation in Phase-Contrast Microscopy Images Using Machine Learning." Machine Vision and Applications, 23(4):659-673, 2012, paper on Springer, local pdf (restricted access).

2012 Project on Segmentation of Interacting Cells

The goal of our work is to address a key bottleneck in computer vision systems that automatically analyze cells - being able to accurately maintain an estimate of the shape of an individual cell as it undergoes continuous deformation while interacting with other deforming cells over long durations. Our method relies on the observation that the boundary of the merged object is made up of segments of the individual object boundaries from before they merged. We published this work at MICCAI 2012:

Z. Wu, D. Gurari, J. Y. Wong, and M. Betke, "Hierarchical Partial Matching and Segmentation of Interacting Cells." Proceedings of the 15th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Nice, France, October 1-5, 2012. 8 pp., Poster. Pdf. Video.

2013 Project on Analysis of Segmentation Quality

Segmentation analysis is important for establishing that contours of objects found accurately represent the true contours of objects observed in images. The common approach for segmentation analysis involves selecting a measure and then computing a score indicating how similar the query segmentation is to the gold standard segmentation. Observing that the score depends on the gold standard segmentation, we proposed an analysis framework that introduces the consideration of how to establish the gold standard segmentation. We call this framework SAGE standing for Segmentation Annotation Collection, Gold Standard Generation, and Evaluation. We conducted three case studies addressing how to establish trusted gold standard segmentations for cell and artery images. We prepared a freely available implementation of SAGE and published our work at WACV:

D. Gurari, S. Kim, E. Yang, B. Isenberg, T. Pham, A. Purwada, P. Solski, M. Walker, J. Y. Wong, and M. Betke, "SAGE: An Approach and Implementation Empowering Quick and Reliable Quantitative Analysis of Segmentation Quality," Proceedings of the Workshop on Applications in Computer Vision (WACV), Clearwater, Florida, January 17-18, 2013. 7 pp. pdf. Best Paper Award. One of two awards selected among 161 submitted and 75 accepted papers.

2016 Projects on Crowdsourcing and Interactive Cell Image Analysis

M. Sameki, M. Gentil, D. Gurari, E. Saraee, E. Hasenberg, J. Wong, and M. Betke. " CrowdTrack: Interactive Tracking of Cells in Microscopy Image Sequences with Crowdsourcing Support." GroupSight Workshop at The Fourth AAAI Conference on Human Computation and Crowdsourcing (HCOMP 2016), Austin, Texas, October 30-November 3, 2016.

D. Gurari, M. Sameki, and M. Betke. "Investigating the Influence of Data Familiarity to Improve the Design of a Crowdsourcing Image Annotation System." The Fourth AAAI Conference on Human Compuation and Crowdsourcing (HCOMP 2016), October 30-November 3, 2016.

D. Gurari, M. Sameki, Z. Wu, and M. Betke. "Mixing Crowd and Algorithm Efforts to Segment Objects in Biomedical Images." The 3rd Interactive Medical Image Computation Workshop (IMIC) held in conjunction with MICCAI 2016 in Athens, Greece, October 21, 2016. 8 pages.

M. Gentil, M. Sameki, D. Gurari, E. Saraee, E. Hasenberg, J. Y. Wong, and M. Betke. "Interactive Tracking of Cells in Microscopy Image Sequences" The 3rd Interactive Medical Image Computation Workshop (IMIC) held in conjunction with MICCAI 2016 in Athens, Greece, October 21, 2016. 10 pages.

M. Sameki, D. Gurari, and M. Betke. "ICORD: Intelligent Collection of Redundant Data -- A Dynamic System for Crowdsourcing Cell Segmentations Accurately and Efficiently." The CVPR Workshop on Computer Vision for Microscopy Image Analysis (CVMI) on July 1, 2016. 10 pages.

D. Gurari, S. D. Jain, M. Betke and K. Grauman. "Pull the Plug? Predicting If Computers or Humans Should Segment Images." IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016, Las Vegas, June 2016. 10 pages.


Video of tracking cells in time-lapse microscopy images

Active Contour Segmentation


Last update by Margrit Betke 1/2017 | W3C | CSS