Document Open Access Logo

Size, Depth and Energy of Threshold Circuits Computing Parity Function

Author Kei Uchizawa

PDF

File

Thumbnail PDF
PDF
LIPIcs.ISAAC.2020.54.pdf
  • Filesize: 459 kB
  • 13 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Theory of computation → Models of computation
Keywords
  • Circuit complexity
  • neural networks
  • threshold circuits
  • sprase activity
  • tradeoffs

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

We investigate relations among the size, depth and energy of threshold circuits computing the n-variable parity function PAR_n, where the energy is a complexity measure for sparsity on computation of threshold circuits, and is defined to be the maximum number of gates outputting "1" over all the input assignments. We show that PAR_n is hard for threshold circuits of small size, depth and energy:  
- If a depth-2 threshold circuit C of size s and energy e computes PAR_n, it holds that 2^{n/(elog ^e n)} ≤ s; and 
- if a threshold circuit C of size s, depth d and energy e computes PAR_n, it holds that 2^{n/(e2^{e+d}log ^e n)} ≤ s.  We then provide several upper bounds:  
- PAR_n is computable by a depth-2 threshold circuit of size O(2^{n-2e}) and energy e; 
- PAR_n is computable by a depth-3 threshold circuit of size O(2^{n/(e-1)} + 2^{e-2}) and energy e; and 
- PAR_n is computable by a threshold circuit of size O((e+d)2^{n-m}), depth d + O(1) and energy e + O(1), where m = max (((e-1)/(d-1))^{d-1}, ((d-1)/(e-1))^{e-1}).  Our lower and upper bounds imply that threshold circuits need exponential size if both depth and energy are constant, which contrasts with the fact that PAR_n is computable by a threshold circuit of size O(n) and depth 2 if there is no restriction on the energy. Our results also suggest that any threshold circuit computing the parity function needs depth to be sparse if its size is bounded.

Cite As Get BibTex

Kei Uchizawa. Size, Depth and Energy of Threshold Circuits Computing Parity Function. In 31st International Symposium on Algorithms and Computation (ISAAC 2020). Leibniz International Proceedings in Informatics (LIPIcs), Volume 181, pp. 54:1-54:13, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2020) https://doi.org/10.4230/LIPIcs.ISAAC.2020.54

Author Details

Kei Uchizawa
  • Graduate School of Science and Engineering, Yamagata University, Japan

Funding

  • Uchizawa, Kei: This work was supported by JSPS KAKENHI Grant Number JP19K11817.

Acknowledgements

We thank the reviewers for their constructive criticisms and suggestions.

References

  1. K. Amano and A. Maruoka. On the complexity of depth-2 circuits with threshold gates. In Proc. of the 30th international conference on Mathematical Foundations of Computer Science, pages 107-118, 2005. Google Scholar
  2. M. Anthony and P. L. Bartlett. Neural Network Learning: Theoretical Foundations. Cambridge University Press, 1999. Google Scholar
  3. L. Chen and R. Tell. Bootstrapping results for threshold circuits "just beyond" known lower bounds. In Proc. of the 51st Annual ACM Symposium on Theory of Computing, pages 34-41, 2019. Google Scholar
  4. R. Chen, R. Santhanam, and S. Srinivasan. Average-case lower bounds and satisfiability algorithms for small threshold circuits. Theory of Computing, 14(9):1-55, 2018. Google Scholar
  5. K. Dinesh, S. Otiv, and J. Sarma. New bounds for energy complexity of boolean functions. In Proc. of the 24th International Computing and Combinatorics Conference, pages 738-750, 2018. Google Scholar
  6. P. F̈oldíak. Sparse coding in the primate cortex. In M. A. Arbib, editor, The Handbook of Brain Theory and Neural Networks, pages 1064-1068. MIT Press, 1995. Google Scholar
  7. J. Forster, M. Krause, S. V. Lokam, R. Mubarakzjanov, N. Schmitt, and H. U. Simon. Relations between communication complexity, linear arrangements, and computational complexity. In Proc. of the 21st International Conference on Foundations of Software Technology and Theoretical Computer Science, pages 171-182, 2001. Google Scholar
  8. A. Hajnal, W. Maass, P. Pudĺak, M. Szegedy, and G. Tuŕan. Threshold circuits of bounded depth. Journal of Computer and System Sciences, 46:129-154, 1993. Google Scholar
  9. J. Håstad. Computational Limitations of Small-depth Circuits. MIT Press, 1987. Google Scholar
  10. J. Håstad and M. Goldmann. On the power of small-depth threshold circuits. Computational Complexity, 1(2):113-129, 1991. Google Scholar
  11. Y. He, K. Kavukcuoglu, Y. Wang, A. Szlam, and Y. Qi. Unsupervised feature learning by deep sparse coding. In Proc. of SIAM International Conference on Data Mining, pages 902-910, 2014. Google Scholar
  12. R. Impagliazzo, R. Paturi, and M. E. Saks. Size-depth tradeoffs for threshold circuits. SIAM Journal on Computing, 26(3):693-707, 1997. Google Scholar
  13. S. Jukna. Extremal Combinatorics with Applications in Computer Science. Springer-Verlag Berlin Heidelberg, 2011. Google Scholar
  14. D. M. Kane and R. Williams. Super-linear gate and super-quadratic wire lower bounds for depth-two and depth-three threshold circuits. In Proc. of the 48th Annual ACM Symposium on Theory of Computing, pages 633-643, 2016. Google Scholar
  15. H. Lee, A. Battle, R. Raina, and A. Y. Ng. Efficient sparse coding algorithms. In Proc. of the 19th Advances in Neural Information Processing Systems, pages 801-808, 2006. Google Scholar
  16. P. Lennie. The cost of cortical computation. Current Biology, 13:493-497, 2003. Google Scholar
  17. M. Minsky and S. Papert. Perceptrons: An Introduction to Computational Geometry. MIT Press, 1988. Google Scholar
  18. A. Y. Ng. Sparse autoencoder. CS294A Lecture notes, 2011. Google Scholar
  19. B. Olshausen and D. J. Field. Sparse coding of sensory inputs. Current Opinion in Neurobiology, 14(4):481-487, 2004. Google Scholar
  20. I. Parberry. Circuit Complexity and Neural Networks. MIT Press, 1994. Google Scholar
  21. A. A. Razborov and A. A. Sherstov. The sign-rank of ac⁰. SIAM Journal on Computing, 39(5):1833-1855, 2010. Google Scholar
  22. K. Y. Siu, V. Roychowdhury, and T. Kailath. Discrete Neural Computation: A Theoretical Foundation. Prentice Hall, 1995. Google Scholar
  23. X. Sun, Y. Sun, K. Wu, and Z. Xia. On the relationship between energy complexity and other boolean function measures. In Proc. of the 25th International Computing and Combinatorics Conference, pages 516-528, 2019. Google Scholar
  24. A. Suzuki, K. Uchizawa, and X. Zhou. Energy-efficient threshold circuits computing MOD functions. In Proc. of the 17th Computing: the Australasian Theory Symposium, pages 105-110, 2011. Google Scholar
  25. A. Suzuki, K. Uchizawa, and X. Zhou. Energy-efficient threshold circuits computing MOD functions. International Journal of Foundations of Computer Science, 24(1):15-29, 2013. Google Scholar
  26. K. Uchizawa. Lower bounds for threshold circuits of bounded energy. Interdisciplinary Information Sciences, 20(1):27-50, 2014. Google Scholar
  27. K. Uchizawa, R. Douglas, and W. Maass. On the computational power of threshold circuits with sparse activity. Neural Computation, 18(12):2994-3008, 2008. Google Scholar
  28. K. Uchizawa and E. Takimoto. Exponential lower bounds on the size of constant-depth threshold circuits with small energy complexity. Theoretical Computer Science, 407(1-3):474-487, 2008. Google Scholar
  29. K. Uchizawa, E. Takimoto, and T. Nishizeki. Size-energy tradeoffs of unate circuits computing symmetric Boolean functions. Theoretical Computer Science, 412:773-782, 2011. Google Scholar
  30. A. C-C. Yao. Separating the polynomial-time hierarchy by oracles. In Proc. of the 26th Annual Symposium on Foundations of Computer Science, pages 1-10, 1985. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact