Neural Networks are old algorithms dated back to 80s and early 90s. It recent resurgence follows the explosion of computational power.

• Can be applied to learning problem with large feature space (n > 1000)
• If the network has $S_j$ units in layer j, $S_j+1$ units in layer j+1, then matrix of weight controlling function $\theta$ is $S_{j+1} (S_j+1) $
• Generally the hidden layers have same unit number