Abstract
Statistical mechanics is applied to lossy compression using multilayer perceptrons for unbiased Boolean messages. We utilize a treelike committee machine (committee tree) and treelike parity machine (parity tree) whose transfer functions are monotonic. For compression using a committee tree, a lower bound of achievable distortion becomes small as the number of hidden units increases. However, it cannot reach the Shannon bound even where . For a compression using a parity tree with hidden units, the rate distortion function, which is known as the theoretical limit for compression, is derived where the code length becomes infinity.
- Received 25 August 2005
DOI:https://doi.org/10.1103/PhysRevE.74.026108
©2006 American Physical Society