Signal representations for image classification need to be invariant with respect to the transformations which do not affect our ability to recognize. An important instance of these transformations is due to physical transformations, such as translations, dilations or rotations. Besides invariance, signal representations also need to be continuous with respect to signal deformations, and should capture enough signal information so that they can discriminate between different signal classes.

Scattering transforms build invariant, stable and informative representations through a non-linear, unitary transform, which delocalizes signal information into scattering decomposition paths. They are computed with a cascade of wavelet modulus operators, and correspond to a convolutional network where filter coefficients are given by a wavelet operator.

Thanks to their invariance and stability properties, scattering operators linearize deformations. This linearization property can be exploited to build linear generative classifiers in the scattering domain, which are computed with simple class-conditional PCA. When applied to stationary textures, scattering transforms provide new texture descriptors, incorporating high order moments which can discriminate non-Gaussian properties. As a result, state-of-the-art classification results are obtained on hand-written digit recognition and texture classification.

The Scattering transform shares some properties with the Fourier modulus transform. It has good frequency localization, it is translation invariant and it is unitary:

f = indicator of a square

Modulus of the Fourier transform of f

Scattering transform of f

However, the Scattering transform is also stable with respect to small deformations. In this example, a Gabor atom is slightly deformed with a dilation and a rotation. Most of the Fourier energy is displaced to other frequencies, whereas most of the scattering energy remains stable:

f = Gabor atom of varying frequency and direction

Modulus of the Fourier transform of f

Scattering transform of f

Besides, scattering coefficients capture high-order moments of stationary processes, as opposed to the Fourier power spectrum:

X1, X2 = Two realizations of stationary textures. X2 is obtained by equalizing random white noise according to the spectrum of X1.	Power spectrum of X1, X2.	Scattering transform of X1, X2. High order scattering coefficients discriminate between the Gaussian process X2 and the non-gaussian X1.

Classification Results

MNIST Digit

MNIST Digit database consists in 60.000 training images of digits and 10.000 training. The following results are reported in the paper , , .

Training Size	Raw Pixels		Windowed Fourier		Scatt \( m_\max = 1\)		Scatt \( m_\max = 2\)		Conv. Net.
	PCA	SVM	PCA	SVM	PCA	SVM	PCA	SVM
300	14.5	15.4	7.35	7.4	5.7	8	\( \bf{4.7} \)	5.6	7.18
1000	7.2	8.2	3.74	3.74	2.35	4	\( \bf{2.3} \)	2.6	3.21
2000	5.8	6.5	2.99	2.9	1.7	2.6	\( \bf{1.3} \)	1.8	2.53
5000	4.9	4	2.34	2.2	1.6	1.6	\( \bf{1.03} \)	1.4	1.52
10000	4.55	3.11	2.24	1.65	1.5	1.23	0.88	1	\( \bf{0.85} \)
20000	4.25	2.2	1.92	1.15	1.4	0.96	0.79	\(\bf{0.58}\)	0.76
40000	4.1	1.7	1.85	0.9	1.36	0.75	0.74	\( \bf{0.53} \)	0.65
60000	4.3	1.4	1.80	0.8	1.34	0.62	0.7	\( \bf{0.43} \)	0.53

CUReT

CUReT is a standard database of texture images. The following results are reported in the paper , ,

Training Size	Raw Pixels	Fourier Spectrum	Scat. \(m_{\max}=1\)	Scat. \(m_{\max}=2\)	Textons	MRF
	PCA	PCA	PCA	PCA
46	17	1	0.5	\( \bf{0.2} \)	1.53	2.4

OUTex 10

OUTex TC10 is a texture database with no rotation in the training set but all possible rotation in the testing set and thus requires built-in rotation invariant. The following results are reported in the paper Combined Scattering for Rotation Invariant Texture Analysis, Sifre L. and Mallat S., Proceedings of the ESANN 2012 conference, Apr. 2012. (PDF)

Training set	\( \text{LBP}^{riu2} + \) \( \text{VAR}_{(8,1) + (16,2) + (24,3)} \)	LBP-HF	RI-LPQ	Combined scattering \( m_{\max} = 1 \) \( \widetilde{m}_{\max}=2 \)	Combined scattering \( m_{\max}= 2 \) \( \widetilde{m}_{\max}=0 \)	Combined scattering \( m_{\max}= 2 \) \( \widetilde{m}_{\max}=1 \)	Combined scattering \( m_{\max}= 2 \) \( \widetilde{m}_{\max}=2 \)
rotation	97.7	96.59	98.26	96.72	97.73	98.62	\(\textbf{98.75} \)
rotation + tilt	NC	67.50	78.02	81.61	89.38	92.89	\(\textbf{93.07}\)