AIELSON: A neural spoken-word poetry generator with a distinct South American voice | Intellect Skip to content
1981
  1. Home
  2. A-Z Publications
  3. Journal of Interdisciplinary Voice Studies
  4. Volume 7, Issue 1
  5. Article
Posthuman Voices: Channels across Time and Shared Memories
  • ISSN: 2057-0341
  • E-ISSN: 2057-035X

Abstract

Human–computer interaction will soon be framed as a dialogue in-between two agents, rather than the imposition of the needs and desires of the human entity over the inert machine. As the latter become seemingly more intelligent, we will witness how they reshape art, knowledge and society in general even more in the not-so-distant future. In this framework, decolonization of their algorithms becomes imperative so as not to reproduce the ethnic and cultural biases that prevail in contemporary human society. By using a pre-trained transformer-based language model (GPT-2) (Radford et al. 2019a), retrained with poetry in Spanish, fine-tuned on examples of South American poetry recited by two different text-to-speech synthesis systems – the Tacotron 2 (Radford et al. 2019b) + Waveglow (Prenger et al. 2018) – coupled posteriorly using the ESPnet-TTS toolkit (Hayashi et al. 2020), trained on an Argentinean voice dataset fine-tuned on voice snippets of Peruvian poet Jorge Eduardo Eielson, I came up with a selection of spoken-word poems in a distinctly Latin American voice that ended up presented as the (‘The Time of Man’) album, printed on a set of four 7-inch lathe-cut stereo vinyl discs. This process turns into a self-reflecting gesture when the dataset used for training is based on South American Artistic Traditions of both the present and the past.

© 2022 Intellect Ltd
Loading

Article metrics loading...

/content/journals/10.1386/jivs_00052_1
2022-08-01
2024-04-19
Loading full text...

Full text loading...

References

  1. Agüera y Arcas, B.. ( 2017;), ‘ Art in the age of machine intelligence. ’, Artists + Machine Intelligence blog, 24 February, https://medium.com/artists-and-machine-intelligence/what-is-ami-ccd936394a83. Accessed 21 December 2021.
     [Google Scholar]
  2. AIELSON ( 2020;), ‘ Ciencia. ’, El Tiempo del Hombre, digital record , USA:: Bandcamp;, https://khipumancer.bandcamp.com/album/el-tiempo-del-hombre. Accessed 10 January 2022.
     [Google Scholar]
  3. Alex, Zabjek. ( 2018;), ‘ How artificial intelligence is reshaping our lives. ’, ScienceX , 17 April, https://phys.org/news/2018-04-artificial-intelligence-reshaping.html. Accessed 21 December 2021.
  4. Alvarado, Luis. ( 2020;), ‘ Colores. ’, Audiopinturas: Estructuras verbales para voz (1972) de Jorge Eduardo Eielson, digital record , Perú:: Buh Records;, https://buhrecords.bandcamp.com/track/colores-1972. Accessed 22 December 2021.
     [Google Scholar]
  5. Barthes, Roland. ( 1977), Images Music Text, London:: Fontana Press;.
     [Google Scholar]
  6. Biggs, Tim, and Moran, Robert. ( 2021;), ‘ What is a deepfake?. ’, The Sydney Morning Herald, 2 June, https://www.smh.com.au/technology/what-is-the-difference-between-a-fake-and-a-deepfake-20200729-p55ghi.html. Accessed 19 December 2021.
     [Google Scholar]
  7. Brock, David. ( 2017;), ‘ Software as hardware: Apollo’s rope memory. ’, IEEE Spectrum, 29 September, https://spectrum.ieee.org/software-as-hardware-apollos-rope-memory. Accessed 21 December 2021.
     [Google Scholar]
  8. Brokaw, Galen. ( 2003), The Poetics of Khipu Historiography: Felipe Guaman Poma de Ayala’s ‘Nueva Crónica’ and the ‘Relación de los Quipucamayos’, Pittsburgh, PA:: The Latin American Studies Association;.
     [Google Scholar]
  9. Brophy, Jessica. ( 2010;), ‘ Developing a corporeal cyberfeminism: Beyond cyberutopia. ’, New Media & Society, 12:6, pp. 929–45.
     [Google Scholar]
  10. Brownlee, Jason. ( 2019;), ‘ A gentle introduction to generative adversarial networks (GANs). ’, Machine Learning Mastery, 17 June, https://machinelearningmastery.com/what-are-generative-adversarial-networks-gans/. Accessed 22 December 2021.
     [Google Scholar]
  11. DeepDream ( 2022), https://deepdreamgenerator.com/. Accessed 20 May 2022.
  12. Derrida, Jacques. ( 1973), Speech and Phenomena, Evanston, IL:: Northwestern University Press;.
     [Google Scholar]
  13. Eidshem, Nina S.. ( 2019;), ‘ Formal and informal pedagogies: Believing in race, teaching race, hearing race. ’, in The Race of Sound: Listening, Timbre, and Vocality in African American Music, Durham, NC, and London:: Duke University Press;, pp. 39–60.
     [Google Scholar]
  14. Eielson, Jorge Eduardo. ( 1964), White Quipus, Manhattan, NY:: MoMa Collection;, https://www.moma.org/collection/works/78814. Accessed 6 December 2021.
     [Google Scholar]
  15. Eielson, Jorge Eduardo. ( 1971), El cuerpo de Giulia-no, México City:: Joaquín Mortiz Editores;.
     [Google Scholar]
  16. Eielson, Jorge Eduardo. ( 1972;), ‘ Escultura horripilante. ’, in Creación y crítica 12, Perú:: Editorial Jurídica;, pp. 8–9.
     [Google Scholar]
  17. Eielson, Jorge Eduardo. ( 2021;), ‘ Biography. ’, Archivio Eielson, http://www.jorgeeielson.org/english-biography.html. Accessed 21 December 2021.
     [Google Scholar]
  18. El Comercio ( 2015), ¿Cuánto pesa cada sector en el PBI del Perú?, Perú:: Editora El Comercio S. A;., https://elcomercio.pe/economia/peru/grafico-dia-pesa-sector-pbi-peru-194520-noticia/?ref=ecr. Accessed 10 April 2022.
     [Google Scholar]
  19. FakeYou ( 2022), https://fakeyou.com/. Accessed 20 May 2022.
  20. Foley, Joseph. ( 2022;), ‘ 14 deepfake examples that terrified and amused the internet. ’, Creative Bloq, 13 April, https://www.creativebloq.com/features/deepfake-examples. Accessed 20 May 2022.
     [Google Scholar]
  21. Gacea, Alexandru-Ovidiu. ( 2019;), ‘ Plato and the “internal dialogue”: An ancient answer for a new model of the self. ’, in L. Pitteloud, and E. Keeling. (eds), Philosophical Studies Series, 139, Cham:: Springer;, pp. 33–54, https://doi.org/10.1007/978-3-030-04654-5_4. Accessed 17 June 2022.
     [Google Scholar]
  22. Hamilton, Andrew. ( 2020;), ‘ Ana De Orbegoso’s Neo-Huaco #3. ’, The Art Institute of Chicago, 1 April, https://www.artic.edu/articles/802/ana-de-orbegosos-neo-huaco-3. Accessed 21 December 2021.
     [Google Scholar]
  23. Hao, Karen. ( 2020;), ‘ We read the paper that forced Timnit Gebru out of Google: Here’s what it says. ’, MIT Technology Review, 4 December, https://www.technologyreview.com/2020/12/04/1013294/google-ai-ethics-research-paper-forced-out-timnit-gebru/. Accessed 7 December 2021.
     [Google Scholar]
  24. Hayashi, Tomoki,, Yamamoto, Ryuichi,, Inoue, Katsuki,, Yoshimura, Takenori,, Watanabe, Shinji,, Toda, Tomoki,, Takeda, K.,, Zhang, Yu, and Tan, Xu. ( 2020;), ‘ Espnet-TTS: Unified, reproducible, and integratable open source end-to-end text-to-speech toolkit. ’, 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, Barcelona, 4–8 May, IEEE Xplore;, pp. 7654–58.
     [Google Scholar]
  25. Hodassman, Shiri,, Vardi, Roni,, Tugendhaft, Yael,, Goldental, Amir, and Kanter, Ido. ( 2022;), ‘ Efficient dendritic learning as an alternative to synaptic plasticity hypothesis. ’, Scientific Reports, 12, p. 6571, https://doi.org/10.1038/s41598-022-10466-8. Accessed 17 June 2022.
     [Google Scholar]
  26. Hodge, Susie. ( 2017), Short Story of Art, London:: Laurence King Publishing;.
     [Google Scholar]
  27. Horta, Moisés. ( 2020;), ‘ Mix for AMBIX 09. ’, Internet Public Radio, 11 August, https://soundcloud.com/h-e-x-o-r-c-i-s-m-o-s/ambix09-w. Accessed 7 December 2021.
     [Google Scholar]
  28. IBM Cloud Education ( 2020a;), ‘ Deep learning. ’, 1 May, https://www.ibm.com/cloud/learn/deep-learning. Accessed 17 June 2022.
  29. IBM Cloud Education ( 2020b;), ‘ Neural networks. ’, 17 August, https://www.ibm.com/cloud/learn/neural-networks. Accessed 17 June 2022.
  30. Karras, Tero,, Laine, Samuli, and Aila, Timo. ( 2019;), ‘ A style-based generator architecture for generative adversarial networks. ’, arXiv, 19 March, https://arxiv.org/abs/1812.04948. Accessed 20 May 2022.
     [Google Scholar]
  31. Kirn, Peter. ( 2020;), ‘ Transfiguración: Decolonizing AI, in Hexorcismos’ shamanistic music and art. ’, Create Digital Music, 6 July, https://cdm.link/2020/07/transfiguracion-decolonizing-ai-in-hexorcismos-shamanistic-music-and-art/. Accessed 6 December 2021.
     [Google Scholar]
  32. Lewis, Jason Edward,, Arista, Noelani,, Pechawis, Archer, and Kite, Suzanne. ( 2020;), ‘ Making kin with the machines. ’, in B. Vickers, and K. Allado-McDowell. (eds), Atlas of Anomalous AI, London:: Ignota Books;, pp. 40–53.
     [Google Scholar]
  33. Lugones, Maria. ( 2008;), ‘ The coloniality of gender. ’, Worlds & Knowledges Otherwise, 2, Spring, pp. 1–17.
     [Google Scholar]
  34. Medrano, Manuel. ( 2021;), ‘ What do we know about Khipus?. ’, Google Arts & Culture , https://artsandculture.google.com/story/what-do-we-know-about-khipus/9AXRnol-w-3crQ. Accessed 19 July 2022.
  35. Mignolo, Walter D.. ( 2014;), ‘ Looking for the meaning of decolonial gesture. ’, Hemispheric Institute, https://hemisphericinstitute.org/en/emisferica-11-1-decolonial-gesture/11-1-essays/looking-for-the-meaning-of-decolonial-gesture.html. Accessed 19 December, 2021.
     [Google Scholar]
  36. Murray, Freya, and Allado-McDowell, K.. ( 2021;), ‘ When artists and machine intelligence work together. ’, Artists + Machine Intelligence blog, 30 April, https://medium.com/artists-and-machine-intelligence/artistsmeetai-230c65cae093. Accessed 21 February 2022.
     [Google Scholar]
  37. Nelson, Robin. ( 2013), Practice as Research in the Arts: Principles, Protocols, Pedagogies, Resistances, Basingstoke:: Palgrave Macmillan;.
     [Google Scholar]
  38. O’Neil, Cathy. ( 2016), Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, New York:: Crown Books;.
     [Google Scholar]
  39. Onorato, Rina S., and Turner, John C.. ( 2004;), ‘ Fluidity in the self-concept: The shift from personal to social identity. ’, European Journal of Social Psychology, 34:3, pp. 257–78.
     [Google Scholar]
  40. Overdub ( 2022), https://www.descript.com/overdub. Accessed 20 May 2022.
  41. Payá, Begoña. ( 2009;), ‘ Voice and identity: A contrastive study of identity perception in voice. ’, Ph.D. dissertation, Munich:: Ludwig-Maximilians-Universität;.
     [Google Scholar]
  42. Pereyra, Patricia. ( 2014), Eielson desnudo, Spain:: Quechua Films;.
     [Google Scholar]
  43. Peru Travel ( 2022), https://www.peru.travel/pe. Accessed 10 April 2022.
  44. Petropolous, Georgios. ( 2018;), ‘ The impact of artificial intelligence on employment. ’, Bruegel, https://www.bruegel.org/wp-content/uploads/2018/07/Impact-of-AI-Petroupoulos.pdf. Accessed 6 April 2022.
     [Google Scholar]
  45. Prenger, Ryan Rafael Valle, and Catanzaro, Bryan. ( 2018;), ‘ WaveGlow: A flow-based generative network for speech synthesis. ’, arXiv, 31 October, https://arxiv.org/pdf/1811.00002.pdf. Accessed 19 December 2021.
     [Google Scholar]
  46. Quijano, Rodrigo. ( 2018;), ‘ Juan Javier Salazar, La Realidad Entera Está en Llamas. ’, Artishock Revista, 20 January, https://artishockrevista.com/2018/01/20/juan-javier-salazar/. Accessed 21 December 2021.
     [Google Scholar]
  47. Radford, Alec,, Wu, Jeff,, Amodei, Dario,, Amodei, Daniela,, Clark, Jack,, Brundage, Mike, and Sutskever, Ilya. ( 2019a;), ‘ Better language models and their implications. ’, OpenAI, 14 February, https://openai.com/blog/better-language-models/. Accessed 18 December 2021.
     [Google Scholar]
  48. Radford, Alec,, Wu, Jeff,, Child, Rewon,, Luan, David,, Amodei Dario, and Sutskever, Ilya. ( 2019b;), ‘ Language models are unsupervised multitask learners. ’, OpenAI, https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf. Accessed 21 May 2022.
     [Google Scholar]
  49. Rayner, Alex. ( 2016;), ‘ Can Google’s deep dream become an art machine?. ’, The Guardian, 28 March, https://www.theguardian.com/artanddesign/2016/mar/28/google-deep-dream-art. Accessed 8 January 2022.
     [Google Scholar]
  50. Rege, Manjeet, and Yarmolouk, Dan. ( 2020;), ‘ Artificial intelligence and its impact on jobs. ’, St. Thomas University News, 19 November, https://news.stthomas.edu/artificial-intelligence-and-its-impact-on-jobs/. Accessed 6 April 2022.
     [Google Scholar]
  51. ResembleAI ( 2022), https://www.resemble.ai/. Accessed 20 May 2022.
  52. Roberts, Leland., ( 2020;), ‘ Understanding the Mel spectrogram. ’, Analytics Vidhya , 6 March, https://medium.com/analytics-vidhya/understanding-the-mel-spectrogram-fca2afa2ce53. Accessed 20 May 2022.
  53. Roxanne, Tiara. ( 2021;), ‘ About. ’, Tiara Roxanne , https://www.tiararoxanne.com/about.html. Accessed 21 December 2021.
  54. Salomon, Frank. ( 2004), The Cord Keepers: Khipus and Cultural Life in a Peruvian Village, Durham, NC, and London:: Duke University Press;.
     [Google Scholar]
  55. Santos, Boavenutra de Sousa. ( 2014), Epistemologies of the South, New York:: Routledge;.
     [Google Scholar]
  56. Schmidt, Peter R.. ( 2019;), ‘ Ontology unveiled, serpents remembered, time reconfigured. ’, in E. Baysal,, S. Souvatzi, and A. Baysal. (eds), Time and History in Prehistory, Abingdon and New York:: Routledge;, pp. 58–76.
     [Google Scholar]
  57. Silverman, Kaja. ( 1988), The Acoustic Mirror: The Female Voice in Psychoanalysis and Cinema, Bloomington, IN:: Indiana University Press;.
     [Google Scholar]
  58. Solak, Imdat. ( 2019;), ‘ The M-AILABS speech dataset. ’, Caito , 3 January, https://www.caito.de/2019/01/the-m-ailabs-speech-dataset/. Accessed 21 December 2021.
  59. Tello, Mario D.. ( 2014), ¿Podemos hablar de una maldición de los recursos naturales en el Perú?, Perú:: Consorcio de Investigación Económica y Social;, https://cies.org.pe/sites/default/files/files/articulos/economiaysociedad/06-tello.pdf. Accessed 10 January 2022.
     [Google Scholar]
  60. TensorFlow ( 2019;), ‘ Transfer learning and fine-tuning. ’, https://www.tensorflow.org/tutorials/images/transfer_learning. Accessed 20 May 2022.
  61. Torres Núñez del Prado, Paola. ( 2003;), ‘ Quipu performance in Central Park. ’, Github , https://autodios.github.io/info/NYQuipu.html. Accessed 21 December 2021.
  62. Torres Nuñez del Prado, Paola. ( 2020a), The Quipus of Tupicocha, self-published documentary , https://khipucamayoc.github.io/documentary.html. Accessed 8 December 2021.
     [Google Scholar]
  63. Torres Nuñez del Prado, Paola. ( 2020b), El Tiempo del Hombre, USA:: Bandcamp;, https://khipumancer.bandcamp.com/album/el-tiempo-del-hombre. Accessed 21 December 2021.
     [Google Scholar]
  64. Torres Nuñez del Prado, Paola. ( 2021a;), ‘ Knots of code. ’, Github , https://khipucamayoc.github.io/. Accessed 21 December 2021.
  65. Torres Nuñez del Prado, Paola. ( 2021b;), ‘ From Quipucamayocs to Neoquipucamayocs. ’, Github , https://khipucamayoc.github.io/AboutProject.html. Accessed 17 June 2022.
  66. Torres Nuñez del Prado, Paola. ( 2021c;), ‘ The Neokhipukamayoqs. ’, Github , https://khipumantes.github.io/. Accessed 21 December 2021.
  67. Torres Nuñez del Prado, Paola. ( 2021d;), voxINformatio. , Vimeo , https://vimeo.com/523759790. Accessed 21 December 2021.
  68. Turkle, Sherry. ( 2005), The Second Self: Computers and the Human Spirit, Cambridge, MA:: MIT Press;.
     [Google Scholar]
  69. Urco, Jaime, and Cisneros Cox, Aalfonso. ( 1988;), ‘ Jorge Eduardo Eielson: El Creador como transgresor. ’, Lienzo, 8, April, pp. 189–205.
     [Google Scholar]
  70. Urton, Gary. ( 2003), Signs of the Inka Khipu: Binary Coding in the Andean Knotted-String Records, Austin, TX:: University of Texas Press;.
     [Google Scholar]
  71. Vera Cubas, Rodrigo. ( 2017), Un lugar para ningún objeto: las esculturas subterráneas de J. E. Eielson, Perú:: Meier Ramirez, Publicaciones Independientes;.
     [Google Scholar]
  72. Vera Cubas, Rodrigo. ( 2018;), ‘ Planos, diagramas e instrucciones en el arte no objetual de J.E Eielson, Teresa Burga y Emilio Rodríguez Larraín. ’, in Pontificia Universidad Católica del Perú (ed.), Investigaciones en arte y diseño, Tomo II, Perú:: Pontificia Universidad Católica del Perú;, pp. 5–7.
     [Google Scholar]
  73. Vera Cubas, Rodrigo. ( 2020), Un lugar para ningún objeto: emplazamientos subterráneos y utopías de papel en la práctica artística de Jorge Eduardo Eielson, Perú:: Pontificia Universidad Católica del Perú;, https://tesis.pucp.edu.pe/repositorio/bitstream/handle/20.500.12404/19768/VERA_CUBAS_RODRIGO.pdf?sequence=1&isAllowed=y. Accessed 10 January 2022.
     [Google Scholar]
  74. Zhang, Jianlei,, Zeng, Yukun, and Starly, Binil. ( 2021;), ‘ Recurrent neural networks with long term temporal dependencies in machine tool wear diagnosis and prognosis. ’, SN Applied Sciences, 3, p. 442.
     [Google Scholar]
  75. Torres Núñez del Prado, Paola. ( 2022;), ‘ AIELSON: A neural spoken-word poetry generator with a distinct South American voice. ’, Journal of Interdisciplinary Voice Studies, 7:1, pp. 11–33, https://doi.org/10.1386/jivs_00052_1
     [Google Scholar]
http://instance.metastore.ingenta.com/content/journals/10.1386/jivs_00052_1
Loading
AIELSON: A neural spoken-word poetry generator with a distinct South American voice
Journal of Interdisciplinary Voice Studies 7, 11 (2022); https://doi.org/10.1386/jivs_00052_1
/content/journals/10.1386/jivs_00052_1
/content/journals/10.1386/jivs_00052_1
Loading

Data & Media loading...

  • Article Type: Article
Keyword(s): art; history; machine learning; natural language processing; South America; text to speech synthesis

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a success
Invalid data
An error occurred
Approval was partially successful, following selected items could not be processed due to error