Loading...

Semi-Supervised Kernel Learning for Pattern Classification

Rohban, Mohammad Hossein | 2012

2472 Viewed
  1. Type of Document: Ph.D. Dissertation
  2. Language: Farsi
  3. Document No: 43218 (19)
  4. University: Sharif University of Technology
  5. Department: Computer Engineering
  6. Advisor(s): Rabiee, Hamid Reza
  7. Abstract:
  8. Supervised kernel learning has been the focus of research in recent years. Although these methods are developed based on rigorous frameworks, they fail to improve the classification accuracy in real world applications. In order to find the origin of this problem, it should be noted that the kernel function represents a prior knowledge on the labeling function. Similar to other learning problem, learning this prior knowledge needs another prior knowledge. In supervised kernel learning, only naive assumptions can be used as the prior knowledge. These include minimizing the ℓ1 and ℓ2 norms of the kernel parameters.
    As an alternative approach, in Semi-Supervised Learning (SSL), unlabeled data is employed as well as the labeled data in the learning process. In order to use the unlabeled data, certain assumptions should be held on the relation of the labeling function and distribution and geometry of the unlabeled data. These assumptions make it possible to reduce the number of kernels in the hypothsis space. Therefore, the kernel learning could be well posed problem in SSL. As a result, it is expected that SSL kernel learning leads to more significant improvements than the supervised kernel learning. Our aim is to develop efficient SSL kernel learning methods in this dissertation.
    Manifold assumption is one of the most applicable and important assumption used in SSL. The SSL methods that are based on this assumption are called graph-based methods. This assumption states that the labeling function changes smoothly on the data manifold. The neighborhood graph has a key role in modeling the manifold structure and evaluation of the labeling function smoothness.Hence, a successful graph construction scheme is necessary for every graph-based SSL method.We will show that learning the neighborhood graph is equivalent with the graphbasedsemi-supervised kernel learning. Therefore, we first survey the methods of graph construction. Since labeled data proves to be effective for graph construction,we will focus on supervised graph construction methods. We will show that three recently proposed graph construction methods, optimize similar objective functions, implicitly. Analysis of this objective function would be helpful in developing a graph construction framework.
    We will provide a novel framework for graph construction to address the quality evaluation of neighborhood graphs and appropriate prior knowledge on the structure of the neighborhood graph. Specifically, we provide two graph quality measures based on the Marginal Likelihood in Gaussian Processes framework and generalization error bound of graph-based SSL methods. We show that these two measures result in similar objective fucntions for graph construction. Moreover, we will show that original neighborhood graph is a subgraph of a k′-NN graph in the Euclidean space.
    We propose two graph construction methods based on the novel framework. In the first method, we use an edge classification scheme to remove between class edges from the k′-NN graph. We show that in this method the expected Marginal Likelihood will be maximized. In the second proposed method, we explore the metric learning methods to be used for graph construction. Based on the proposed framework, we will show that a local ranking based metric learning is the suitable choice for graph construction.
    We use a number of real world and synthetic dataset to evaluate the proposed methods. The generalization bound is used to validate whether the manifold assumption is held on different datasets. The accuracies of classification shows significant improvements with respect to the state-of-the-art graph construction methods
  9. Keywords:
  10. Kernel Learning ; Semi-Supervised Learning ; Patterns Classification ; Manifold Assumption ; Neighborhood Graph ; Semi-Supervised Graph Construction

 Digital Object List

 Bookmark

  • مقدمه
    • تعاریف اولیه
    • اهمیت موضوع
    • تعریف مساله
    • راهکار پیشنهادی
    • نوآوری‌ها
    • ساختار رساله
  • یادگیری نیمه‌نظارتی با فرض منیفلد
    • بیان رسمی فرض منیفلد
    • روش‌های اعمال فرض منیفلد
      • روش انتشار برچسب
      • روش انتشار برچسب تعمیم یافته
      • روش منظم‌سازی Tikhonov
      • روش سازگاری محلی و عمومی
      • ورارسانی گراف از طریق کمینه‌سازی تناوبی
    • روش‌های ساخت گراف همسایگی
      • روش‌های سنتی
      • روش b-matching
      • روش حذف نویز منیفلد
      • روش تنظیم ابرپارامترهای گراف
      • روش نمایش تنک
      • روش یادگیری تطبیقی منیفلد
      • روش بهینه‌کردن احتمال حاشیه‌ای
      • روش تیزکردن لبه‌های گراف
      • یادگیری طیفی هسته
      • روش یادگیری متریک برای ساخت گراف همسایگی
      • ارتباط روش‌های یادگیری طیفی، تیزکردن لبه‌های گراف و بیشینه کردن احتمال حاشیه‌ای
    • نتیجه‌گیری
  • چارچوب نظری پیشنهادی
    • مقدمه
    • ارتباط یادگیری هسته و ساخت گراف
    • پیش‌فرض مناسب روی گراف همسایگی
    • بهینه‌سازی متوسط درست‌نمایی
    • بهینه‌سازی باند خطای تعمیم
    • درجه متصل بودن گراف
    • نتیجه‌گیری
  • روش‌های پیشنهادی
    • مقدمه
    • روش اول: دسته‌بندی اتصالات
      • تابع هدف
      • سختی تخمین آماری احتمالات عدم تشابه برچسب‌ها
      • الگوریتم تخمین و ساخت گراف
      • پیچیدگی زمانی الگوریتم
    • روش دوم : یادگیری متریک برای ساخت گراف
      • اهمیت رتبه‌بندی احتمالات در ساخت گراف
      • یادگیری متریک پیشنهادی
      • الگوریتم ساخت گراف
      • سازگار بودن روش یادگیری متریک
      • پیچیدگی زمانی الگوریتم
    • نتیجه‌گیری
  • ارزیابی روش‌های پیشنهادی
    • معرفی مجموعه داده‌ها
    • بررسی برقراری فرض منیفلد
    • نتایج روش پیشنهادی اول
      • نرخ خطا در دسته‌بندی اتصالات
      • دقت برچسب‌زدن داده‌ها
    • نتایج روش پیشنهادی دوم
      • بحث روی نتایج
      • ‌ارزیابی در یک پیکربندی واقعی‌تر
      • ‌ارزیابی برای مسئله دسته‌بندی تصاویر
    • نتیجه‌گیری
  • جمع‌بندی و کارهای آتی
  • مراجع
  • ‌واژه‌نامه انگلیسی به فارسی
  • ‌واژه‌نامه فارسی به انگلیسی
  • اختصارات
...see more