Research Interests: 

  • Multimedia Database,
  • Content-based Multimedia Information Retrieval,
  • Data Mining,
  • Machine Learning,
  • Pattern Recognition,
  • Computer Vision and Image Understanding. 

Ph.D. Dissertation Scope:

The general scope of my Ph.D. research on image database modeling and retrieval.

Projects: [Automatic Image Annotation] [Concepts Discovery] [Semantic Repository Modeling] [BALAS] [FAST] [CORAL] [Med-Archive] [Multimedia-Edu] [Motion Detection] [Image Morphing]


Automatic Image Annotation and its Application to Multi-Modal Image Retrieval

This project addresses automatic image annotation problem and its application to multi-modal image retrieval. The contribution of our work is three-fold. (1) We propose a probabilistic semantic model in which the visual features and the textual words are connected via a hidden layer which constitutes the semantic concepts to be discovered to explicitly exploit the synergy among the modalities. (2) The association of visual features and textual words is determined in a Bayesian framework such that the confidence of the association can be provided. (3) Extensive evaluation on a large-scale, visually and semantically diverse image collection crawled from Web is reported to evaluate the prototype system based on the model. In the proposed probabilistic model, a hidden concept layer which connects the visual feature and the word layer is discovered by fitting a generative model to the training image and annotation words through an Expectation-Maximization (EM) based iterative learning procedure. The evaluation of the prototype system on 17,000 images and 7,736 automatically extracted annotation words from crawled Web pages for multi-modal image retrieval has indicated that the proposed semantic model and the developed Bayesian framework are superior to a state-of-the-art peer system in the literature. [more]

     Hidden Semantic Concepts Discovery in Image Databases

This project focuses on developing a hidden semantic concept discovery methodology to address effective semantics-intensive image retrieval. In our approach, each image in the database is segmented to regions associated with homogenous color, texture, and shape features. By exploiting regional statistical information in each image and employing a vector quantization method, a uniform and sparse region-based representation is achieved. With this representation a probabilistic model based on statistical-hidden-class assumptions of the image database is obtained, to which Expectation Maximization (EM) technique is applied to analyze semantic concepts hidden in the database. [more]

  Semantic Repository Modeling with Alpha-Semantics Graph for Image Classification

This project investigates content based image database mining and retrieval, focusing on developing a classification orientated
methodology to address semantics-intensive image retrieval. In our approach, with Self Organization Map (SOM) based image feature grouping, a visual dictionary is created for color, texture, and shape feature attributes respectively. Labeling each training
image with the keywords in the visual dictionary, a classification tree is built. Based on the statistical properties of the feature
space we define a structure, called Alpha-Semantics Graph, to discover the hidden semantic relationships among the semantic repositories embodied in the image database. With the Alpha-Semantics Graph, each semantic repository is modeled as a unique fuzzy set to explicitly address the semantic uncertainty existing and overlapping among the repositories in the feature space. An algorithm using the classification accuracy measures is developed to combine the built classification tree with the fuzzy set modeling method to deliver semantically relevant image retrieval for a given query image. [more]

                 BALAS: Stretching Bayesian Learning in Relevance Feedback of Image Retrieval

This project addresses user relevance feedback in image retrieval. We take this problem as a standard two-class pattern classification problem aiming at refining the retrieval precision by learning through the user relevance feedback data. However, we have investigated the problem by noting two important unique characteristics of the problem: small sample collection and asymmetric sample distributions between positive and negative samples. We have developed a novel approach to stretching Bayesian learning to solve for this problem by explicitly exploiting the two unique characteristics, which is the methodology of BAyesian  Learning in Asymmetric and Small sample collections, thus called BALAS. Different learning strategies are used for positive and negative sample collections, respectively, in BALAS based on the two unique characteristics. By defining the relevancy confidence as the relevant posterior probability, we have developed an integrated ranking scheme in BALAS which complementarily combines the subjective relevancy confidence and the objective similarity measure to capture the overall retrieval semantics. [more]

  FAST: Fast And Semantics-Tailored (FAST) Image Retrieval

This project develops a Fast And Semantics-Tailored (FAST) image retrieval methodology. Specifically, the contributions of FAST methodology to the CBIR literature include: (1) development of a new indexing method based on fuzzy logic to incorporate color, texture, and shape information into a region based approach to improving the retrieval effectiveness and robustness (2) development of a new hierarchical indexing structure and the corresponding Hierarchical, Elimination-based A* Retrieval algorithm (HEAR) to significantly improve the retrieval efficiency without sacrificing the retrieval effectiveness; it is shown that HEAR is guaranteed to deliver a logarithm search in the average case (3) employment of user relevance feedbacks to tailor the semantic retrieval to each user's individualized query preference through the novel Indexing Tree Pruning (ITP) and Adaptive Region Weight Updating (ARWU) algorithms. [more]

  CORAL: Money Laundry Pattern Learning and Detection Using Data Mining Techniques

This project studies the problem of applying data mining to facilitate the investigation of money laundering crimes (MLSs). We have identified a new paradigm of problems --- that of automatic community generation based on uni-party data, the data in which there is no direct or explicit link information available. Consequently, we have proposed a new methodology for Link Discovery based on Correlation Analysis (LDCA). We have used MLC group model generation as an exemplary application of this problem paradigm, and have focused on this application to develop a specific method of automatic MLC group model generation based on timeline analysis using the LDCA methodology, called CORAL. A prototype of CORAL method has been implemented, and preliminary testing and evaluation based on a real MLC case data are reported. [Online Demo] [more]

  Web Based Medical Data Archive System

This project develops web-based medical data archive system. This system is a prototype system of a proposed paradigm of research aiming at building up a distributed archival system over the Internet to facilitate medical practitioners for maintaining, sharing, updating, searching, and processing medical information conveniently and consistently. Our system differs from the existing systems in the sense that it not only offers a full spectrum of online communications, processing, and annotation tools, but also provides powerful multimodal search functionalities to the users. In addition, the database is always kept in a "live" mode such that information contributed by users is periodically indexed. [Online Demo] [more]

  An Internet-Based Multimedia Diabetes Education System

This project develops an Internet based diabetes education system for school nurses. Compared with existing systems fro medical education using computer-assisted technologies, our system exhibits the advantages of powerful browsing capability, friendly GUI, individualized presentation schedule tailored to each user based on their special browsing interest, powerful online processing and annotation capabilities, and powerful querying capabilities. The system is being evaluated for school nurses in Onondaga County, NY. [Online Demo] [more]

  Independent Motion Detection Directly from Compressed Surveillance Video

This project addresses the problem frequently encountered in military/intelligence surveillance: automatic mining video data for independently moving targets. Instead of attempting to detect each individual independently moving target in each frame, this project focuses on determining whether or not each frame has independent motion. Consequently, it differs from the existing literature on this topic in the following two aspects: (1) fast detection (2) detection directly from the compressed video streams instead of from the image sequences. These two aspects are motivated from the applications that this research aims to address, and therefore, are the contributions to the literature of independent motion detection. The solution is based on the Linear System Consistency Analysis, and thus called LSCA. [more]

  Feature-based Image Morphing

A new approach to image morphing. is developed. The approach gives the animator high-level control of the visual effect by providing natural feature-based specification and interaction. When used effectively, this technique can give the illusion that the photographed or computer generated subjects are transforming in a fluid, surrealistic, and often dramatic way. The new method is extended to accommodate keyframed transformations between image sequences for motion image work. [more]