Robust Digital Molecular Design of Binarized Neural Networks

Authors Johannes Linder, Yuan-Jyue Chen, David Wong, Georg Seelig, Luis Ceze, Karin Strauss



PDF
Thumbnail PDF

File

LIPIcs.DNA.27.1.pdf
  • Filesize: 2.14 MB
  • 20 pages

Document Identifiers

Author Details

Johannes Linder
  • University of Washington, Paul G. Allen School of Computer Science and Engineering, Seattle, WA, USA
Yuan-Jyue Chen
  • Microsoft Research, Redmond, WA, USA
David Wong
  • University of Washington, Department of Bioengineering, Seattle, WA, USA
Georg Seelig
  • University of Washington, Paul G. Allen School of Computer Science and Engineering, Seattle, WA, USA
  • University of Washington, Department of Electrical and Computer Engineering, Seattle, WA, USA
Luis Ceze
  • University of Washington, Paul G. Allen School of Computer Science and Engineering, Seattle, WA, USA
Karin Strauss
  • Microsoft Research, Redmond, WA, USA
  • University of Washington, Paul G. Allen School of Computer Science and Engineering, Seattle, WA, USA

Cite As Get BibTex

Johannes Linder, Yuan-Jyue Chen, David Wong, Georg Seelig, Luis Ceze, and Karin Strauss. Robust Digital Molecular Design of Binarized Neural Networks. In 27th International Conference on DNA Computing and Molecular Programming (DNA 27). Leibniz International Proceedings in Informatics (LIPIcs), Volume 205, pp. 1:1-1:20, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2021) https://doi.org/10.4230/LIPIcs.DNA.27.1

Abstract

Molecular programming - a paradigm wherein molecules are engineered to perform computation - shows great potential for applications in nanotechnology, disease diagnostics and smart therapeutics. A key challenge is to identify systematic approaches for compiling abstract models of computation to molecules. Due to their wide applicability, one of the most useful abstractions to realize is neural networks. In prior work, real-valued weights were achieved by individually controlling the concentrations of the corresponding "weight" molecules. However, large-scale preparation of reactants with precise concentrations quickly becomes intractable. Here, we propose to bypass this fundamental problem using Binarized Neural Networks (BNNs), a model that is highly scalable in a molecular setting due to the small number of distinct weight values. We devise a noise-tolerant digital molecular circuit that compactly implements a majority voting operation on binary-valued inputs to compute the neuron output. The network is also rate-independent, meaning the speed at which individual reactions occur does not affect the computation, further increasing robustness to noise. We first demonstrate our design on the MNIST classification task by simulating the system as idealized chemical reactions. Next, we map the reactions to DNA strand displacement cascades, providing simulation results that demonstrate the practical feasibility of our approach. We perform extensive noise tolerance simulations, showing that digital molecular neurons are notably more robust to noise in the concentrations of chemical reactants compared to their analog counterparts. Finally, we provide initial experimental results of a single binarized neuron. Our work suggests a solid framework for building even more complex neural network computation.

Subject Classification

ACM Subject Classification
  • Theory of computation → Models of computation
  • Applied computing
Keywords
  • Molecular Computing
  • Neural Network
  • Binarized Neural Network
  • Digital Logic
  • DNA
  • Strand Displacement

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads

References

  1. Y. Benenson, B. Gil, U. Ben-Dor, R. Adar, and E. Shapiro. An autonomous molecular computer for logical control of gene expression. Nature, 429(6990):423-429, 2004. Google Scholar
  2. H. Bui, S. Shah, R. Mokhtar, T. Song, S. Garg, and J. Reif. Localized DNA hybridization chain reactions on DNA origami. ACS nano, 12(2):1146-1155, 2018. Google Scholar
  3. L. Cardelli. Two-domain DNA strand displacement. Mathematical Structures in Computer Science, 23(2):247-271, 2013. Google Scholar
  4. L. Ceze, J. Nivala, and K. Strauss. Molecular digital data storage using dna. Nature Reviews Genetics, 20(8):456-466, 2019. Google Scholar
  5. H.L. Chen, D. Doty, and D. Soloveichik. Rate-independent computation in continuous chemical reaction networks. In Proceedings of the 5th Conference on Innovations in Theoretical Computer Science, pages 313-326, 2014 January. Google Scholar
  6. S.X. Chen and G. Seelig. A DNA neural network constructed from molecular variable gain amplifiers. In International Conference on DNA-Based Computers, pages 110-121, 2017 September. Google Scholar
  7. Y.J. Chen, N. Dalchau, N. Srinivas, A. Phillips, L. Cardelli, D. Soloveichik, and G. Seelig. Programmable chemical controllers made from DNA. Nature nanotechnology, 8(10):755-762, 2013. Google Scholar
  8. K.M. Cherry and L. Qian. Scaling up molecular pattern recognition with DNA-based winner-take-all neural networks. Nature, 559(7714):370-376, 2018. Google Scholar
  9. G.M. Church, Y. Gao, and S. Kosuri. Next-generation digital information storage in DNA. Science, 337(6102):1628-1628, 2012. Google Scholar
  10. S. Darabi, M. Belbahri, M. Courbariaux, and V.P. Nia. BNN+: Improved binary network training. OpenReview, 2018. Google Scholar
  11. I.R. Epstein and J.A. Pojman. An introduction to nonlinear chemical dynamics: Oscillations, waves, patterns, and chaos. Oxford Univ Press London, page London, 1998. Google Scholar
  12. F. Fages, G. Le Guludec, O. Bournez, and A. Pouly. Strong turing completeness of continuous chemical reaction networks and compilation of mixed analog-digital programs. In International Conference on Computational Methods in Systems Biology, pages 108-127, 2017 September. Google Scholar
  13. A. Hjelmfelt, E.D. Weinberger, and J. Ross. Chemical implementation of neural networks and turing machines. Proceedings of the National Academy of Sciences, 88(24):10983-10987, 1991. Google Scholar
  14. I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio. Binarized neural networks. In Advances in Neural Information Processing Systems, pages 4107-4115, 2016. Google Scholar
  15. D.P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv, 2014. URL: http://arxiv.org/abs/1412.6980.
  16. M.R. Lakin and A. Phillips. Modelling, simulating and verifying turing-powerful strand displacement systems. In International Workshop on DNA-Based Computers, pages 130-144, 2011 September. Google Scholar
  17. M.R. Lakin, S. Youssef, F. Polo, S. Emmott, and A. Phillips. Visual DSD: a design and analysis tool for DNA strand displacement systems. Bioinformatics, 27(22):3211-3213, 2011. Google Scholar
  18. R. Lopez, R. Wang, and G. Seelig. A molecular multi-gene classifier for disease diagnostics. Nature chemistry, 10(7):746-754, 2018. Google Scholar
  19. M.O. Magnasco. Chemical kinetics is turing universal. Physical Review Letters, 78(6):1190, 1997. Google Scholar
  20. L. Organick, S.D. Ang, Y.J. Chen, R. Lopez, S. Yekhanin, K. Makarychev, M.Z. Racz, G. Kamath, P. Gopalan, B. Nguyen, and C.N.etal. Takahashi. Random access in large-scale DNA data storage. Nature biotechnology, 36(3):242, 2018. Google Scholar
  21. A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in pytorch, 2017. Google Scholar
  22. L. Qian and E. Winfree. Scaling up digital circuit computation with DNA strand displacement cascades. Science, 332(6034):1196-1201, 2011. Google Scholar
  23. L. Qian, E. Winfree, and J. Bruck. Neural network computation with DNA strand displacement cascades. Nature, 475(7356):368-372, 2011. Google Scholar
  24. P. Senum and M. Riedel. Rate-independent constructs for chemical computation. PloS one, 6(6):e21414, 2011. Google Scholar
  25. S. Shah, J. Wee, T. Song, L. Ceze, K. Strauss, Y.J. Chen, and J. Reif. Using strand displacing polymerase to program chemical reaction networks. Journal of the American Chemical Society, 142(21):9587-9593, 2020. Google Scholar
  26. T. Simons and D.J. Lee. A review of binarized neural networks. Electronics, 8(6):661, 2019. Google Scholar
  27. D. Soloveichik, M. Cook, E. Winfree, and J. Bruck. Computation with finite stochastic chemical reaction networks. Natural Computing, 7(4):615-633, 2008. Google Scholar
  28. D. Soloveichik, G. Seelig, and E. Winfree. DNA as a universal substrate for chemical kinetics. Proceedings of the National Academy of Sciences, 107(12):5393-5398, 2010. Google Scholar
  29. T. Song, A. Eshra, S. Shah, H. Bui, D. Fu, M. Yang, R. Mokhtar, and J. Reif. Fast and compact DNA logic circuits based on single-stranded gates using strand-displacing polymerase. Nature nanotechnology, 14(11):1075-1081, 2019. Google Scholar
  30. N. Srinivas, J. Parkin, G. Seelig, E. Winfree, and D. Soloveichik. Enzyme-free nucleic acid dynamical systems. Science, 358(6369), 2017. Google Scholar
  31. A. Stephenson, M. Willsey, J. McBride, S. Newman, B. Nguyen, C. Takahashi, K. Strauss, and L. Ceze. PurpleDrop: A digital microfluidics-based platform for hybrid molecular-electronics applications. IEEE Micro, 40(5):76-86, 2020. Google Scholar
  32. M. Vasic, C. Chalk, S. Khurshid, and D. Soloveichik. Deep molecular programming: A natural implementation of binary-weight ReLU neural networks. arXiv, 2020. URL: http://arxiv.org/abs/2003.13720.
  33. M. Vasic, D. Soloveichik, and S. Khurshid. CRN++: Molecular programming language. In International Conference on DNA Computing and Molecular Programming, pages 1-18, 2018 October. Google Scholar
  34. D.Y. Zhang and G. Seelig. DNA-based fixed gain amplifiers and linear classifier circuits. In International Workshop on DNA-Based Computers, pages 176-186, 2010 June. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail