Mairal, Julien and Bach, Francis and Ponce, Jean and Sapiro, Guillermo
Sparse coding—that is, modelling data vectors as sparse linear combinations of basis elements—is widely used in machine learning, neuroscience, signal processing, and statistics. This paper focuses on learning the basis set, also called dictionary, to adapt it to specific data, an approach that has recently proven to be very effective for signal reconstruction and classification in the audio and image processing domains. This paper proposes a new online optimization algorithm for dictionary learning, based on stochastic approximations. Unlike classical stochastic gradient approaches, our algorithm does not require any parameter tuning, yet it scales up gracefully to large datasets with millions of training samples. A proof of its convergence is presented, along with experiments with natural images demonstrating that it leads to faster performance and better dictionaries than classical batch algorithms for both small and large datasets.
Discussion
Dear authors,
One of the auhtors published “Online dictionary learning for sparse coding” in ICML'09 and “Learning to sense sparse signals: Simultaneous sensing matrix and sparsifying dictionary optimization” in IEEE TIP, July 2009. Both seem to be rather fast in learning dictionaries and are faster than the the existing methods. I'm just wondering which one is faster.
Looking forward to hearing from you soon! Thank you!
Chun-Shien Lu
The second paper by Duarte-Carvajalino and Sapiro is about learning simultaneously a dictionary and a projection matrix adapted to compressed sensing. Their software was based on an implementation of mine of the algorithm K-SVD for learning dictionaries (Elad & Aharon, 2006). The algorithm we present in ICML'09 is much faster. I will release the code very soon and I hope it will be useful!
Best regards
Julien Mairal