Variational Dirichlet Process Gaussian Mixture Model

Description

This software implements several algorithms. (see References section below.)

My PhD thesis uses similar notation for the parameters of the code. Please find Eq 3.11. You might find the thesis more useful when you read the code.

Algorithms

  1. accelerated variational Dirichlet process Gaussian mixture model
  2. collapsed variational stick-breaking Dirichlet process Gaussian mixture model
  3. variational Gaussian mixture model with a collapsed Dirichlet prior.
  4. variational Dirichlet process Gaussian mixture model by Blei and Jordan.

Note more algorithms are actually available. They are specified by options.

Usage

This software requires Matlab.

The usage is like,

>> result = vdpgm(X, opts);

The first argument is data. Each data point is a column vector of X.

The second argument opts is the option of this program which determines an algorithm and hyperparameters. You can set opts as you want, or basic option generators are also available.

>> opts = mkopts_avdp; % for the algorithm 1
>> opts = mkopts_csb(10); % for the algorithm 2 with T=10 
>> opts = mkopts_cdp(10); % for the algorithm 3 with K=10 
>> opts = mkopts_bj(10); % for the algorithm 4 with T=10

opts accepts many options. Some options are exclusive.

Posterior Assignment Probability

The output result is a structure containing parameters for posteriors.

Maybe, the most useful result is result.q_of_z which is the posterior probability of assignments. q_of_z is a N by K (or T) matrix s.t. sum_c q_of_z(i,c) = 1 for any i.q_of_z is available only when opts.get_q_of_z is set to 1.

Predictive Density

Let Test is a test data set (a D by M matrix; M test points). The following commands give the predctive density at each data point in Test.

>> opts.get_q_of_z = 1;
>> results = vdpgm(X, opts);
>> results.test_data = Test;
>> results_predictive = vdpgm(X, results);

results_predictive.predictive_posterior will be a 1 by M matrix whereresults_predictive.predictive_posterior(i) is a log predictive density of data pointTest(:,i).

Weighted Leaning

Let Test is a test data set (a D by M matrix; M test points). The following commands give the predctive density at each data point in Test.

>> opts.get_q_of_z = 1;
>> results = vdpgm(X, opts);
>> results.test_data = Test;
>> results_predictive = vdpgm(X, results);

results_predictive.predictive_posterior will be a 1 by M matrix whereresults_predictive.predictive_posterior(i) is a log predictive density of data pointTest(:,i).

Download

License

This software is distributed under the BSD license.

Copyright (C) 2007-2008 Kenichi Kurihara

Source

vdpgm.tar.gz

ChangeLog

  • June. 2 2010: Fixed a bug in mk_log_lambda(). Thank Dan for finding the bug.
  • Mar. 24 2010: Fixed a bug in the predictive distribution computation. Thank Antti for the patch.
  • Jul. 4 2008 : add a function for the predictive density.
  • Aug. 24 2007 : kd-tree code had a bug; fixed.
  • Apr. 20 2007 : The previous package did not contain “power_method.m”.

References

Kenichi Kurihara, Structure Learning by Bayesian Inference, Jan, 2008 PDF, Supervisor Taisuke Sato

Kenichi Kurihara, Max Welling and Yee Whye Teh, Collapsed Variational Dirichlet Process Mixture Models, the Twentieth International Joint Conference on Artificial Intelligence (IJCAI 2007).PDF

Kenichi Kurihara, Max Welling and Nikos Vlassis, Accelerated Variational Dirichlet Mixture Models, Advances in Neural Information Processing Systems 19 (NIPS 2006). PDF

David M. Blei and Michael I. Jordan, Variational Inference for Dirichlet Process Mixtures, Bayesian Analysis, Vol.1, No.1, 2005.