MATLAB, Learning Deep Boltzmann Machines

원본 : http://web.mit.edu/~rsalakhu/www/DBM.html

Code provided by Ruslan Salakhutdinov 
Permission is granted for anyone to copy, use, modify, or distribute this program and accompanying programs and documents for any purpose, provided this copyright notice is retained and prominently displayed, along with a note saying that the original programs are available from our web page. The programs and documents are distributed without any warranty, express or implied. As the programs were written for research purposes only, they have not been tested to the degree that would be advisable in any important application. All use of these programs is entirely at the user's own risk.

How to make it work:

1. Create a separate directory and download all these files into the same directory
2. Download from http://yann.lecun.com/exdb/mnist the following 4 files:
   • train-images-idx3-ubyte.gz
   • train-labels-idx1-ubyte.gz
   • t10k-images-idx3-ubyte.gz
   • t10k-labels-idx1-ubyte.gz
3. Unzip these 4 files by executing:
   • gunzip train-images-idx3-ubyte.gz
   • gunzip train-labels-idx1-ubyte.gz
   • gunzip t10k-images-idx3-ubyte.gz
   • gunzip t10k-labels-idx1-ubyte.gz
   If unzipping with WinZip, make sure the file names have not been changed by WinZip.
4. Download code_DBM.tar which contains 16 files OR 
   download each of the following 16 files separately:
   • demo_small.m Main file for training and fine-tuning a toy DBM model.
   • demo.m Main file for training and fine-tuning a DBM model (reproduces results of the DBM paper).
   • converter.m Converts raw MNIST digits into matlab format.
   • rbm.m Training RBM with binary hidden and binary visible units.
   • rbm_l2.m Training 2nd layer RBM with binary hidden and visible units.
   • dbm_mf.m Joint training of all layers in a DBM.
   • mf.m Implements mean-field inference.
   • backprop.m Backpropagation for fine-tuning a DBM.
   • mf_class.m Helper function used by backprop.m
   • CG_MNIST.m Conjugate Gradient optimization for fine-tuning a DBM.
   • CG_MNIST_INIT.m Conjugate Gradient optimization for fine-tuning a DBM (training top-level parameters, while holding low-level parameters fixed).
   • makebatches.m Creates minibatches for DBM training.
   • testerr.m Computes misclassification error on the MNIST dataset.
   • dispims.m Displays progress during DBM training.
   • minimize.m Conjugate gradient code.
   • README.txt
5. For the toy experiment, run demo_small in matlab.
6. Run demo in Matlab. This is a long version of training a Deep Boltzmann Machine, which can take over 2 days. The code also uses sparsity to improve model performance.
7. Make sure you have enough space to store the entire MNIST dataset on your disk.

Deep Boltzmann Machine을 어떻게 구현했는지 보기위해 일단 MATLAB코드를 다운받아서 한번 돌려보려고 한다.

위 순서대로 수행하면 된다.

확인할 것은, MNIST 데이터를 다운로드 받고 gz압축을 해제된 파일명을 확인해보길 바란다. MATLAB코드에서 이 파일을 convert할때 fread부분을 보면 파일명이 조금 달라서 에러를 띄운다. 파일명에서 '.'를 '-'로 바꿔줘야 한다.

트레이닝은 pc성능에 따라 다르겠지만, 위 설명에서는 이틀정도 걸린다고 한다. 그래서 보통 deep learning을 할때는 gpu와 multi cpu를 쓰는것이 거의 필수적인것 같다. 이 기회에 gpu나 multi cpu를 사용하는 것을 배워놓는것이 좋을듯.

-------------------------------------------------

2013. 08. 18 수정.

위에 링크가 깨져서 파일 다운로드가 안됩니다.

다행히 제가 이전에 다운로드 받은 파일이 있어 올려둡니다.

code_DBM.tar