An instantaneous voice-synthesis neuroprosthesis | Nature


  • Card, N. S. et al. An correct and quickly calibrating speech neuroprosthesis. N. Engl. J. Med. 391, 609–618 (2024).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Willett, F. R. et al. A high-performance speech neuroprosthesis. Nature 620, 1031–1036 (2023).

    Article 
    ADS 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Metzger, S. L. et al. A high-performance neuroprosthesis for speech decoding and avatar management. Nature 620, 1037–1046 (2023).

    Article 
    ADS 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Silva, A. B., Littlejohn, Ok. T., Liu, J. R., Moses, D. A. & Chang, E. F. The speech neuroprosthesis. Nat. Rev. Neurosci. 25, 473–492 (2024).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Herff, C. et al. Producing pure, intelligible speech from mind exercise in motor, premotor, and inferior frontal cortices. Entrance. Neurosci. 13, 1267 (2019).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Angrick, M. et al. Speech synthesis from ECoG utilizing densely related 3D convolutional neural networks. J. Neural Eng. 16, 036019 (2019).

    Article 
    ADS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Anumanchipalli, G. Ok., Chartier, J. & Chang, E. F. Speech synthesis from neural decoding of spoken sentences. Nature 568, 493–498 (2019).

    Article 
    ADS 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Meng, Ok. et al. Steady synthesis of synthetic speech sounds from human cortical floor recordings throughout silent speech manufacturing. J. Neural Eng. 20, 046019 (2023).

    Article 
    ADS 

    Google Scholar
     

  • Le Godais, G. et al. Overt speech decoding from cortical exercise: a comparability of various linear strategies. Entrance. Hum. Neurosci. 17, 1124065 (2023).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Liu, Y. et al. Decoding and synthesizing tonal language speech from mind exercise. Sci. Adv. 9, eadh0478 (2023).

    Article 
    ADS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Berezutskaya, J. et al. Direct speech reconstruction from sensorimotor mind exercise with optimized deep studying fashions. J. Neural Eng. 20, 056010 (2023).

    Article 
    ADS 
    PubMed Central 

    Google Scholar
     

  • Shigemi, Ok. et al. Synthesizing speech from ECoG with a mixture of transformer-based encoder and neural vocoder. In ICASSP 2023 – 2023 IEEE Int. Conf. Acoust. Speech Sign Course of. 1–5 (IEEE, 2023).

  • Chen, X. et al. A neural speech decoding framework leveraging deep studying and speech synthesis. Nat. Mach. Intell. 6, 467–480 (2024).

    Article 

    Google Scholar
     

  • Wilson, G. H. et al. Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus. J. Neural Eng. 17, 066007 (2020).

    Article 
    ADS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Wairagkar, M., Hochberg, L. R., Brandman, D. M. & Stavisky, S. D. Synthesizing speech by decoding intracortical neural exercise from dorsal motor cortex. In 2023 eleventh Int. IEEE/EMBS Conf. on Neural Eng. (NER) 1–4 (IEEE, 2023).

  • Angrick, M. et al. Actual-time synthesis of imagined speech processes from minimally invasive recordings of neural exercise. Commun. Biol. 4, 1055 (2021).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Wu, X., Wellington, S., Fu, Z. & Zhang, D. Speech decoding from stereo-electroencephalography (sEEG) indicators utilizing superior deep studying strategies. J. Neural Eng. 21, 036055 (2024).

    Article 

    Google Scholar
     

  • Angrick, M. et al. On-line speech synthesis utilizing a chronically implanted mind–pc interface in a person with ALS. Sci. Rep. 14, 9617 (2024).

    Article 
    ADS 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Glasser, M. F. et al. A multi-modal parcellation of human cerebral cortex. Nature 536, 171–178 (2016).

    Article 
    ADS 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Vaswani, A. et al. Consideration is all you want. In Advances in Neural Data Processing Programs 30 (NIPS, 2017).

  • Downey, J. E., Schwed, N., Chase, S. M., Schwartz, A. B. & Collinger, J. L. Intracortical recording stability in human mind–pc interface customers. J. Neural Eng. 15, 046016 (2018).

    Article 
    ADS 
    PubMed 

    Google Scholar
     

  • Valin, J.-M. & Skoglund, J. LPCNET: enhancing neural speech synthesis by means of linear prediction. In ICASSP 2019 – 2019 IEEE Int. Conf. on Acoust. Speech Sign Course of. 5891–5895 (IEEE, 2019).

  • Li, Y. A., Han, C., Raghavan, V. S., Mischler, G. & Mesgarani, N. StyleTTS 2: in the direction of human-level text-to-speech by means of fashion diffusion and adversarial coaching with giant speech language fashions. Adv. Neural Inf. Course of. Syst. 36, 19594–19621 (2023).

  • Dichter, B. Ok., Breshears, J. D., Leonard, M. Ok. & Chang, E. F. The management of vocal pitch in human laryngeal motor cortex. Cell 174, 21–31 (2018).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Kaufman, M. T., Churchland, M. M., Ryu, S. I. & Shenoy, Ok. V. Cortical exercise within the null area: allowing preparation with out motion. Nat. Neurosci. 17, 440–448 (2014).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Stavisky, S. D., Kao, J. C., Ryu, S. I. & Shenoy, Ok. V. Motor cortical visuomotor suggestions exercise is initially remoted from downstream targets in output-null neural state area dimensions. Neuron 95, 195–208 (2017).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Churchland, M. M. & Shenoy, Ok. V. Preparatory exercise and the expansive null-space. Nat. Rev. Neurosci. 25, 213–236 (2024).

    Article 
    CAS 
    PubMed 

    Google Scholar
     

  • Moses, D. A. et al. Neuroprosthesis for decoding speech in a paralyzed individual with anarthria. N. Engl. J. Med. 385, 217–227 (2021).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Kunz, E. M. et al. Illustration of verbal thought in motor cortex and implications for speech neuroprostheses. Preprint at bioRxiv https://doi.org/10.1101/2024.10.04.616375 (2024).

  • Bouchard, Ok. E., Mesgarani, N., Johnson, Ok. & Chang, E. F. Practical group of human sensorimotor cortex for speech articulation. Nature 495, 327–332 (2013).

    Article 
    ADS 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Chartier, J., Anumanchipalli, G. Ok., Johnson, Ok. & Chang, E. F. Encoding of articulatory kinematic trajectories in human speech sensorimotor cortex. Neuron 98, 1042–1054 (2018).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Lu, J. et al. Neural management of lexical tone manufacturing in human laryngeal motor cortex. Nat. Commun. 14, 6917 (2023).

    Article 
    ADS 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Breshears, J. D., Molinaro, A. M. & Chang, E. F. A probabilistic map of the human ventral sensorimotor cortex utilizing electrical stimulation. J. Neurosurg. 123, 340–349 (2015).

    Article 
    PubMed 

    Google Scholar
     

  • Ammanuel, S. G. et al. Intraoperative cortical stimulation mapping with laryngeal electromyography for the localization of human laryngeal motor cortex. J. Neurosurg. 141, 268–277 (2024).

    Article 
    PubMed 

    Google Scholar
     

  • Pandarinath, C. et al. Neural inhabitants dynamics in human motor cortex throughout actions in folks with ALS. eLife 4, e07436 (2015).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Stavisky, S. D. et al. Neural ensemble dynamics in dorsal motor cortex throughout speech in folks with paralysis. eLife 8, e46015 (2019).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Willett, F. R. et al. Hand knob space of premotor cortex represents the entire physique in a compositional approach. Cell 181, 396–409 (2020).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Ali, Y. H. et al. BRAND: a platform for closed-loop experiments with deep community fashions. J. Neural Eng. 21, 026046 (2024).

    Article 
    ADS 
    PubMed Central 

    Google Scholar
     

  • Younger, D. et al. Sign processing strategies for decreasing artifacts in microelectrode mind recordings brought on by practical electrical stimulation. J. Neural Eng. 15, 026014 (2018).

    Article 
    ADS 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Levelt, W. J., Roelofs, A. & Meyer, A. S. A principle of lexical entry in speech manufacturing. Behav. Mind Sci. 22, 1–38 (1999).

    Article 
    CAS 
    PubMed 

    Google Scholar
     

  • Räsänen, O., Doyle, G. & Frank, M. C. Unsupervised phrase discovery from speech utilizing computerized segmentation into syllable-like items. Proc. Interspeech 2015, 3204–3208 (2015).


    Google Scholar
     

  • Williams, A. H. et al. Discovering exact temporal patterns in large-scale neural recordings by means of sturdy and interpretable time warping. Neuron 105, 246–259 (2020).

    Article 
    CAS 
    PubMed 

    Google Scholar
     

  • Roussel, P. et al. Remark and evaluation of acoustic contamination of electrophysiological mind indicators throughout speech manufacturing and sound notion. J. Neural Eng. 17, 056028 (2020).

    Article 
    ADS 
    PubMed 

    Google Scholar
     

  • Shah, N., Sahipjohn, N., Tambrahalli, V., Subramanian, R. & Gandhi, V. StethoSpeech: speech technology by means of a medical stethoscope connected to the pores and skin. Proc. ACM Work together. Mob. Wearable Ubiquitous Technol. 8, 123 (2024).

    Article 

    Google Scholar
     

  • Wairagkar, M. et al. Information for an instantaneous voice synthesis neuroprosthesis. Dryad https://doi.org/10.5061/dryad.2280gb64f (2025).

  • Leave a Reply

    Your email address will not be published. Required fields are marked *