News Reader.info

ABSTRACT Learning-to-learn, a progressive speedup of learning while solving a series of similar problems, represents a core process of knowledge acquisition that draws attention in both

neuroscience and artificial intelligence. To investigate its underlying brain mechanism, we trained a recurrent neural network model on arbitrary sensorimotor mappings known to depend on the

prefrontal cortex. The network displayed an exponential time course of accelerated learning. The neural substrate of a schema emerges within a low-dimensional subspace of population

activity; its reuse in new problems facilitates learning by limiting connection weight changes. Our work highlights the weight-driven modifications of the vector field, which determines the

population trajectory of a recurrent network and behavior. Such plasticity is especially important for preserving and reusing the learned schema in spite of undesirable changes of the vector

field due to the transition to learning a new problem; the accumulated changes across problems account for the learning-to-learn dynamics. Access through your institution Buy or subscribe

This is a preview of subscription content, access via your institution ACCESS OPTIONS Access through your institution Access Nature and 54 other Nature Portfolio journals Get Nature+, our

best-value online-access subscription $32.99 / 30 days cancel any time Learn more Subscribe to this journal Receive 12 print issues and online access $209.00 per year only $17.42 per issue

Learn more Buy this article * Purchase on SpringerLink * Instant access to full article PDF Buy now Prices may be subject to local taxes which are calculated during checkout ADDITIONAL

ACCESS OPTIONS: * Log in * Learn about institutional subscriptions * Read our FAQs * Contact customer support SIMILAR CONTENT BEING VIEWED BY OTHERS FLEXIBLE MULTITASK COMPUTATION IN

RECURRENT NETWORKS UTILIZES SHARED DYNAMICAL MOTIFS Article Open access 09 July 2024 FLEXIBLE GATING BETWEEN SUBSPACES IN A NEURAL NETWORK MODEL OF INTERNALLY GUIDED TASK SWITCHING Article

Open access 01 August 2024 NEURAL REPRESENTATIONAL GEOMETRIES REFLECT BEHAVIORAL DIFFERENCES IN MONKEYS AND RECURRENT NEURAL NETWORKS Article Open access 01 August 2024 DATA AVAILABILITY

Data files, including pre-trained networks, are available for further analyses on GitHub (https://github.com/xjwanglab/learning-2-learn) in Python and MATLAB readable formats. CODE

AVAILABILITY All training and analysis codes are available on GitHub (https://github.com/xjwanglab/learning-2-learn). REFERENCES * Piaget, J. _The Language and Thought of the Child_

(Harcourt Brace, 1926). * Bartlett, F.C. _Remembering: A Study in Experimental and Social Psychology_ (Cambridge University Press, 1932). * Rumelhart, D. E. Schemata: the building blocks of

cognition. in _Theoretical Issues in Reading Comprehension_ 33–58 (Erlbaum Associates, 1980). * Gilboa, A. & Marlatte, H. Neurobiology of schemas and schema-mediated memory. _Trends

Cogn. Sci._ 21, 618–631 (2017). Article  PubMed  Google Scholar  * Chi, M. T., Glaser, R. & Rees, E. Expertise in problem solving.

https://www.public.asu.edu/~mtchi/papers/ChiGlaserRees.pdf (1982). * Harlow, H. F. The formation of learning sets. _Psychological Review_ 56, 51–65 (1949). Article  CAS  PubMed  Google

Scholar  * Lewis, P. A. & Durrant, S. J. Overlapping memory replay during sleep builds cognitive schemata. _Trends Cogn. Sci._ 15, 343–351 (2011). Article  PubMed  Google Scholar  *

Behrens, T. E. J. et al. What is a cognitive map? Organizing knowledge for flexible behavior. _Neuron_ 100, 490–509 (2018). Article  CAS  PubMed  Google Scholar  * Preston, A. R. &

Eichenbaum, H. Interplay of hippocampus and prefrontal cortex in memory. _Curr. Biol._ 23, R764–R773. (2013). Article  CAS  PubMed  PubMed Central  Google Scholar  * Wang, S.-H. &

Morris, R. G. Hippocampal–neocortical interactions in memory formation, consolidation, and reconsolidation. _Annu. Rev. Psychol._ 61, 49–79 (2010). Article  PubMed  Google Scholar  *

McKenzie, S. et al. Hippocampal representation of related and opposing memories develop within distinct, hierarchically organized neural schemas. _Neuron_ 83, 202–215 (2014). Article  CAS 

PubMed  PubMed Central  Google Scholar  * Bernardi, S. et al. The geometry of abstraction in the hippocampus and prefrontal cortex. _Cell_ 183, 954–967 (2020). Article  CAS  PubMed  PubMed

Central  Google Scholar  * Braun, D. A., Mehring, C. & Wolpert, D. M. Structure learning in action. _Behav. Brain Res._ 206, 157–165 (2010). Article  PubMed  PubMed Central  Google

Scholar  * Finn, C., Abbeel, P. & Levine, S. Model-agnostic meta-learning for fast adaptation of deep networks. In _Proceedings of the 34th International Conference on Machine Learning_

1126–1135 (PMLR, 2017). * Wang, J. X. et al. Prefrontal cortex as a meta-reinforcement learning system. _Nat. Neurosci._ 21, 860–868 (2018). Article  CAS  PubMed  Google Scholar  *

Passingham, R. _The Frontal Lobes and Voluntary Action_ (Oxford University Press, 1995). * Asaad, W. F., Rainer, G. & Miller, E. K. Neural activity in the primate prefrontal cortex

during associative learning. _Neuron_ 21, 1399–1407 (1998). Article  CAS  PubMed  Google Scholar  * Fusi, S., Asaad, W. F., Miller, E. K. & Wang, X.-J. A neural circuit model of flexible

sensorimotor mapping: learning and forgetting on multiple timescales. _Neuron_ 54, 319–333 (2007). Article  CAS  PubMed  PubMed Central  Google Scholar  * Cromer, J. A., Machon, M. &

Miller, E. K. Rapid association learning in the primate prefrontal cortex in the absence of behavioral reversals. _J. Cogn. Neurosci._ 23, 1823–1828 (2011). Article  PubMed  Google Scholar 

* Bussey, T. J., Wise, S. P. & Murray, E. A. Interaction of ventral and orbital prefrontal cortex with inferotemporal cortex in conditional visuomotor learning. _Behav. Neurosci._ 116,

703–715 (2002). Article  PubMed  Google Scholar  * Petrides, M. Deficits on conditional associative-learning tasks after frontal- and temporal-lobe lesions in man. _Neuropsychologia_ 23,

601–614 (1985). Article  CAS  PubMed  Google Scholar  * Stringer, C. et al. Spontaneous behaviors drive multidimensional, brainwide activity. _Science_ 364, 255 (2019). Article  PubMed 

PubMed Central  Google Scholar  * Zhou, J. et al. Evolving schema representations in orbitofrontal ensembles during learning. _Nature_ 590, 606–611 (2021). * Sadtler, P. T. et al. Neural

constraints on learning. _Nature_ 512, 423–426 (2014). Article  CAS  PubMed  PubMed Central  Google Scholar  * Bao, P., She, L., McGill, M. & Tsao, D. Y. A map of object space in primate

inferotemporal cortex. _Nature_ 583, 103–108 (2020). * Eacott, M. & Gaffan, D. Inferotemporal–frontal disconnection: the uncinate fascicle and visual associative learning in monkeys.

_Eur. J. Neurosci._ 4, 1320–1332 (1992). Article  PubMed  Google Scholar  * Kobak, D. et al. Demixed principal component analysis of neural population data. _eLife_ 5, e10989 (2016). Article

  PubMed  PubMed Central  Google Scholar  * Anderson, R. C., Spiro, R. J. & Anderson, M. C. Schemata as scaffolding for the representation of information in connected discourse. _Am.

Educ. Res. J._ 15, 433–440 (1978). Article  Google Scholar  * Rumelhart, D. E. & Norman, D. A. Accretion, tuning and restructuring: three modes of learning.

https://www.dsoergel.com/UBLIS571DS-06.1a-1Reading10RumelhartAccretionTuningAndRestructuring.pdf (1978). * Thorndyke, P. W. & Hayes-Roth, B. The use of schemata in the acquisition and

transfer of knowledge. _Cogn. Psychol._ 11, 82–106 (1979). Article  Google Scholar  * Kaufman, M. T., Churchland, M. M., Ryu, S. I. & Shenoy, K. V. Cortical activity in the null space:

permitting preparation without movement. _Nat. Neurosci._ 17, 440–448 (2014). Article  CAS  PubMed  PubMed Central  Google Scholar  * Strogatz, S. H. _Nonlinear Dynamics and Chaos: With

Applications to Physics, Biology, Chemistry and Engineering_ 2nd edn (Taylor & Francis, 2016). * Vyas, S., Golub, M. D., Sussillo, D. & Shenoy, K. V. Computation through neural

population dynamics. _Annu. Rev. Neurosci._ 43, 249–275 (2020). Article  CAS  PubMed  PubMed Central  Google Scholar  * Long, P. M. & Sedghi, H. Generalization bounds for deep

convolutional neural networks. In _International Conference on Learning Representations_ (ICLR, 2020). * Gouk, H., Hospedales, T. M. & Pontil, M. Distance-based regularisation of deep

networks for fine-tuning. In _International Conference on Learning Representations_ (ICLR, 2021). * Kaufman, M. T. et al. The largest response component in the motor cortex reflects movement

timing but not movement type. _eNeuro_ 3, ENEURO.0085-16.2016 (2016). * Tenenbaum, J. B., Kemp, C., Griffiths, T. L. & Goodman, N. D. How to grow a mind: statistics, structure, and

abstraction. _Science_ 331, 1279–1285 (2011). Article  CAS  PubMed  Google Scholar  * Sussillo, D. & Barak, O. Opening the black box: low-dimensional dynamics in high-dimensional

recurrent neural networks. _Neural Comput._ 25, 626–649 (2013). Article  PubMed  Google Scholar  * Dubreuil, A., Valente, A., Beiran, M., Mastrogiuseppe, F. & Ostojic, S. The role of

population structure in computations through neural dynamics. _Nat. Neurosci._ 25, 783–794 (2022). * Wang, S.-H., Tse, D. & Morris, R. G. Anterior cingulate cortex in schema assimilation

and expression. _Learn. Mem._ 19, 315–318 (2012). Article  PubMed  PubMed Central  Google Scholar  * Sehgal, M., Song, C., Ehlers, V. L. & Moyer, J. R. Jr. Learning to learn—intrinsic

plasticity as a metaplasticity mechanism for memory formation. _Neurobiol. Learn. Mem._ 105, 186–199 (2013). * Bittner, K. C., Milstein, A. D., Grienberger, C., Romani, S. & Magee, J. C.

Behavioral time scale synaptic plasticity underlies CA1 place fields. _Science_ 357, 1033–1036 (2017). Article  CAS  PubMed  PubMed Central  Google Scholar  * Wang, X.-J. Probabilistic

decision making by slow reverberation in cortical circuits. _Neuron_ 36, 955–968 (2002). Article  CAS  PubMed  Google Scholar  * Kingma, D. P. & Ba, J. Adam: a method for stochastic

optimization. In _International Conference on Learning Representations_ (ICLR, 2015). * Abadi, M. et al. TensorFlow: a system for large-scale machine learning. _USENIX Symposium on Operating

Systems Design and Implementation_ 16, 265–283 (2016). * Stewart, G. W. The efficient generation of random orthogonal matrices with an application to condition estimators. _SIAM J. Numer.

Anal._ 17, 403–409 (1980). Article  Google Scholar  * Krogh, A. & Hertz, J. A. A simple weight decay can improve generalization. In _Advances in Neural Information Processing Systems_

950–957 (NeurIPS, 1991). * Merity, S., McCann, B. & Socher, R. Revisiting activation regularization for language RNNs. In _International Conference on Machine Learning’s Workshop on

Learning to Generate Natural Language_ (ICML, 2017). * Gao, P. et al. A theory of multineuronal dimensionality, dynamics and measurement. Preprint at

https://www.biorxiv.org/content/10.1101/214262v2 (2017). Download references ACKNOWLEDGEMENTS We thank A. L. Fairhall, I. Skelin, J. J. Lin, B. Doiron, G. R. Yang, N. Y. Masse, U. P.

Obilinovic, L. Y. Tian, D. V. Buonomano, J. Jaramillo, J. E. Fitzgerald and H. Sompolinksy for fruitful discussions and Y. Liu, K. Berlemont, A. Battista and P. Theodoni for critical

comments on the manuscript. This work was supported by National Institute of Health U-19 program grant 5U19NS107609-03 and Office of Naval Research grant N00014-23-1-2040. AUTHOR INFORMATION

AUTHORS AND AFFILIATIONS * Center for Neural Science, New York University, New York, NY, USA Vishwa Goudar & Xiao-Jing Wang * Department of Neurobiology, University of Chicago, Chicago,

IL, USA Barbara Peysakhovich & David J. Freedman * Department of Physiology and Biophysics, University of Washington School of Medicine, Seattle, WA, USA Elizabeth A. Buffalo *

Washington National Primate Research Center, Seattle, WA, USA Elizabeth A. Buffalo Authors * Vishwa Goudar View author publications You can also search for this author inPubMed Google

Scholar * Barbara Peysakhovich View author publications You can also search for this author inPubMed Google Scholar * David J. Freedman View author publications You can also search for this

author inPubMed Google Scholar * Elizabeth A. Buffalo View author publications You can also search for this author inPubMed Google Scholar * Xiao-Jing Wang View author publications You can

also search for this author inPubMed Google Scholar CONTRIBUTIONS B.P., D.J.F., E.A.B. and X.-J.W. designed the study. V.G. performed the research. V.G. and X.-J.W. wrote the manuscript.

CORRESPONDING AUTHOR Correspondence to Xiao-Jing Wang. ETHICS DECLARATIONS COMPETING INTERESTS The authors declare no competing interests. PEER REVIEW PEER REVIEW INFORMATION _Nature

Neuroscience_ thanks the anonymous reviewers for their contribution to the peer review of this work. ADDITIONAL INFORMATION PUBLISHER’S NOTE Springer Nature remains neutral with regard to

jurisdictional claims in published maps and institutional affiliations. SUPPLEMENTARY INFORMATION SUPPLEMENTARY INFORMATION Supplementary Notes, Supplementary Methods, Supplementary

Discussion and Supplementary Figs. 1–10. REPORTING SUMMARY RIGHTS AND PERMISSIONS Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article

under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such

publishing agreement and applicable law. Reprints and permissions ABOUT THIS ARTICLE CITE THIS ARTICLE Goudar, V., Peysakhovich, B., Freedman, D.J. _et al._ Schema formation in a neural

population subspace underlies learning-to-learn in flexible sensorimotor problem-solving. _Nat Neurosci_ 26, 879–890 (2023). https://doi.org/10.1038/s41593-023-01293-9 Download citation *

Received: 02 September 2021 * Accepted: 27 February 2023 * Published: 06 April 2023 * Issue Date: May 2023 * DOI: https://doi.org/10.1038/s41593-023-01293-9 SHARE THIS ARTICLE Anyone you

share the following link with will be able to read this content: Get shareable link Sorry, a shareable link is not currently available for this article. Copy to clipboard Provided by the

Springer Nature SharedIt content-sharing initiative