Loading...

Semi-Supervised Kernel Learning for Pattern Classification

Rohban, Mohammad Hossein | 2012

2400 Viewed
  1. Type of Document: Ph.D. Dissertation
  2. Language: Farsi
  3. Document No: 43218 (19)
  4. University: Sharif University of Technology
  5. Department: Computer Engineering
  6. Advisor(s): Rabiee, Hamid Reza
  7. Abstract:
  8. Supervised kernel learning has been the focus of research in recent years. Although these methods are developed based on rigorous frameworks, they fail to improve the classification accuracy in real world applications. In order to find the origin of this problem, it should be noted that the kernel function represents a prior knowledge on the labeling function. Similar to other learning problem, learning this prior knowledge needs another prior knowledge. In supervised kernel learning, only naive assumptions can be used as the prior knowledge. These include minimizing the ℓ1 and ℓ2 norms of the kernel parameters.
    As an alternative approach, in Semi-Supervised Learning (SSL), unlabeled data is employed as well as the labeled data in the learning process. In order to use the unlabeled data, certain assumptions should be held on the relation of the labeling function and distribution and geometry of the unlabeled data. These assumptions make it possible to reduce the number of kernels in the hypothsis space. Therefore, the kernel learning could be well posed problem in SSL. As a result, it is expected that SSL kernel learning leads to more significant improvements than the supervised kernel learning. Our aim is to develop efficient SSL kernel learning methods in this dissertation.
    Manifold assumption is one of the most applicable and important assumption used in SSL. The SSL methods that are based on this assumption are called graph-based methods. This assumption states that the labeling function changes smoothly on the data manifold. The neighborhood graph has a key role in modeling the manifold structure and evaluation of the labeling function smoothness.Hence, a successful graph construction scheme is necessary for every graph-based SSL method.We will show that learning the neighborhood graph is equivalent with the graphbasedsemi-supervised kernel learning. Therefore, we first survey the methods of graph construction. Since labeled data proves to be effective for graph construction,we will focus on supervised graph construction methods. We will show that three recently proposed graph construction methods, optimize similar objective functions, implicitly. Analysis of this objective function would be helpful in developing a graph construction framework.
    We will provide a novel framework for graph construction to address the quality evaluation of neighborhood graphs and appropriate prior knowledge on the structure of the neighborhood graph. Specifically, we provide two graph quality measures based on the Marginal Likelihood in Gaussian Processes framework and generalization error bound of graph-based SSL methods. We show that these two measures result in similar objective fucntions for graph construction. Moreover, we will show that original neighborhood graph is a subgraph of a k′-NN graph in the Euclidean space.
    We propose two graph construction methods based on the novel framework. In the first method, we use an edge classification scheme to remove between class edges from the k′-NN graph. We show that in this method the expected Marginal Likelihood will be maximized. In the second proposed method, we explore the metric learning methods to be used for graph construction. Based on the proposed framework, we will show that a local ranking based metric learning is the suitable choice for graph construction.
    We use a number of real world and synthetic dataset to evaluate the proposed methods. The generalization bound is used to validate whether the manifold assumption is held on different datasets. The accuracies of classification shows significant improvements with respect to the state-of-the-art graph construction methods
  9. Keywords:
  10. Kernel Learning ; Semi-Supervised Learning ; Patterns Classification ; Manifold Assumption ; Neighborhood Graph ; Semi-Supervised Graph Construction

 Digital Object List

  • محتواي پايان نامه
  •   view

 Bookmark

...see more