Document

# Size, Depth and Energy of Threshold Circuits Computing Parity Function

## File

LIPIcs.ISAAC.2020.54.pdf
• Filesize: 459 kB
• 13 pages

## Acknowledgements

We thank the reviewers for their constructive criticisms and suggestions.

## Cite As

Kei Uchizawa. Size, Depth and Energy of Threshold Circuits Computing Parity Function. In 31st International Symposium on Algorithms and Computation (ISAAC 2020). Leibniz International Proceedings in Informatics (LIPIcs), Volume 181, pp. 54:1-54:13, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2020)
https://doi.org/10.4230/LIPIcs.ISAAC.2020.54

## Abstract

We investigate relations among the size, depth and energy of threshold circuits computing the n-variable parity function PAR_n, where the energy is a complexity measure for sparsity on computation of threshold circuits, and is defined to be the maximum number of gates outputting "1" over all the input assignments. We show that PAR_n is hard for threshold circuits of small size, depth and energy: - If a depth-2 threshold circuit C of size s and energy e computes PAR_n, it holds that 2^{n/(elog ^e n)} ≤ s; and - if a threshold circuit C of size s, depth d and energy e computes PAR_n, it holds that 2^{n/(e2^{e+d}log ^e n)} ≤ s. We then provide several upper bounds: - PAR_n is computable by a depth-2 threshold circuit of size O(2^{n-2e}) and energy e; - PAR_n is computable by a depth-3 threshold circuit of size O(2^{n/(e-1)} + 2^{e-2}) and energy e; and - PAR_n is computable by a threshold circuit of size O((e+d)2^{n-m}), depth d + O(1) and energy e + O(1), where m = max (((e-1)/(d-1))^{d-1}, ((d-1)/(e-1))^{e-1}). Our lower and upper bounds imply that threshold circuits need exponential size if both depth and energy are constant, which contrasts with the fact that PAR_n is computable by a threshold circuit of size O(n) and depth 2 if there is no restriction on the energy. Our results also suggest that any threshold circuit computing the parity function needs depth to be sparse if its size is bounded.

## Subject Classification

##### ACM Subject Classification
• Theory of computation → Models of computation
##### Keywords
• Circuit complexity
• neural networks
• threshold circuits
• sprase activity

## Metrics

• Access Statistics
• Total Accesses (updated on a weekly basis)
0

## References

1. K. Amano and A. Maruoka. On the complexity of depth-2 circuits with threshold gates. In Proc. of the 30th international conference on Mathematical Foundations of Computer Science, pages 107-118, 2005.
2. M. Anthony and P. L. Bartlett. Neural Network Learning: Theoretical Foundations. Cambridge University Press, 1999.
3. L. Chen and R. Tell. Bootstrapping results for threshold circuits "just beyond" known lower bounds. In Proc. of the 51st Annual ACM Symposium on Theory of Computing, pages 34-41, 2019.
4. R. Chen, R. Santhanam, and S. Srinivasan. Average-case lower bounds and satisfiability algorithms for small threshold circuits. Theory of Computing, 14(9):1-55, 2018.
5. K. Dinesh, S. Otiv, and J. Sarma. New bounds for energy complexity of boolean functions. In Proc. of the 24th International Computing and Combinatorics Conference, pages 738-750, 2018.
6. P. F̈oldíak. Sparse coding in the primate cortex. In M. A. Arbib, editor, The Handbook of Brain Theory and Neural Networks, pages 1064-1068. MIT Press, 1995.
7. J. Forster, M. Krause, S. V. Lokam, R. Mubarakzjanov, N. Schmitt, and H. U. Simon. Relations between communication complexity, linear arrangements, and computational complexity. In Proc. of the 21st International Conference on Foundations of Software Technology and Theoretical Computer Science, pages 171-182, 2001.
8. A. Hajnal, W. Maass, P. Pudĺak, M. Szegedy, and G. Tuŕan. Threshold circuits of bounded depth. Journal of Computer and System Sciences, 46:129-154, 1993.
9. J. Håstad. Computational Limitations of Small-depth Circuits. MIT Press, 1987.
10. J. Håstad and M. Goldmann. On the power of small-depth threshold circuits. Computational Complexity, 1(2):113-129, 1991.
11. Y. He, K. Kavukcuoglu, Y. Wang, A. Szlam, and Y. Qi. Unsupervised feature learning by deep sparse coding. In Proc. of SIAM International Conference on Data Mining, pages 902-910, 2014.
12. R. Impagliazzo, R. Paturi, and M. E. Saks. Size-depth tradeoffs for threshold circuits. SIAM Journal on Computing, 26(3):693-707, 1997.
13. S. Jukna. Extremal Combinatorics with Applications in Computer Science. Springer-Verlag Berlin Heidelberg, 2011.
14. D. M. Kane and R. Williams. Super-linear gate and super-quadratic wire lower bounds for depth-two and depth-three threshold circuits. In Proc. of the 48th Annual ACM Symposium on Theory of Computing, pages 633-643, 2016.
15. H. Lee, A. Battle, R. Raina, and A. Y. Ng. Efficient sparse coding algorithms. In Proc. of the 19th Advances in Neural Information Processing Systems, pages 801-808, 2006.
16. P. Lennie. The cost of cortical computation. Current Biology, 13:493-497, 2003.
17. M. Minsky and S. Papert. Perceptrons: An Introduction to Computational Geometry. MIT Press, 1988.
18. A. Y. Ng. Sparse autoencoder. CS294A Lecture notes, 2011.
19. B. Olshausen and D. J. Field. Sparse coding of sensory inputs. Current Opinion in Neurobiology, 14(4):481-487, 2004.
20. I. Parberry. Circuit Complexity and Neural Networks. MIT Press, 1994.
21. A. A. Razborov and A. A. Sherstov. The sign-rank of ac⁰. SIAM Journal on Computing, 39(5):1833-1855, 2010.
22. K. Y. Siu, V. Roychowdhury, and T. Kailath. Discrete Neural Computation: A Theoretical Foundation. Prentice Hall, 1995.
23. X. Sun, Y. Sun, K. Wu, and Z. Xia. On the relationship between energy complexity and other boolean function measures. In Proc. of the 25th International Computing and Combinatorics Conference, pages 516-528, 2019.
24. A. Suzuki, K. Uchizawa, and X. Zhou. Energy-efficient threshold circuits computing MOD functions. In Proc. of the 17th Computing: the Australasian Theory Symposium, pages 105-110, 2011.
25. A. Suzuki, K. Uchizawa, and X. Zhou. Energy-efficient threshold circuits computing MOD functions. International Journal of Foundations of Computer Science, 24(1):15-29, 2013.
26. K. Uchizawa. Lower bounds for threshold circuits of bounded energy. Interdisciplinary Information Sciences, 20(1):27-50, 2014.
27. K. Uchizawa, R. Douglas, and W. Maass. On the computational power of threshold circuits with sparse activity. Neural Computation, 18(12):2994-3008, 2008.
28. K. Uchizawa and E. Takimoto. Exponential lower bounds on the size of constant-depth threshold circuits with small energy complexity. Theoretical Computer Science, 407(1-3):474-487, 2008.
29. K. Uchizawa, E. Takimoto, and T. Nishizeki. Size-energy tradeoffs of unate circuits computing symmetric Boolean functions. Theoretical Computer Science, 412:773-782, 2011.
30. A. C-C. Yao. Separating the polynomial-time hierarchy by oracles. In Proc. of the 26th Annual Symposium on Foundations of Computer Science, pages 1-10, 1985.
X

Feedback for Dagstuhl Publishing