Skip to content
1981

oa Hyperformalism: Notes on Machine Vision and Art Historical Method

image of Hyperformalism: Notes on Machine Vision and Art Historical Method
file format pdf download PDF

This chapter is a critical examination of the relationship between machine vision and art historical method. It looks at many of the assumptions and resulting problems when art historical methods, namely the formalistic approaches of Giovanni Morelli and Heinrich Wolfflin, are used in computer vision. It then posits an alternative approach (called hyperformalism) based upon the Viennese art historian Alois Riegl's concept of ‘stilfragen’ and the history of ornament, arguing that it holds untapped potential for the application of computer vision for the history of art.

Keywords: Aby Warburg ; Alois Riegl ; Art History ; Artificial Intelligence ; Digital Art History ; Digital Humanities ; Giovanni Morelli ; Heinrich Wolfflin ; Historiography ; Vienna School

Loading full text...

Full text loading...

/deliver/fulltext/9781789389760/9781789389746-c08.html?itemId=/content/books/9781789389760.c08&mimeType=html&fmt=ahah
Hyperformalism: Notes on Machine Vision and Art Historical Method
Intellect , 146 (2024); https://doi.org/10.1386/9781789389760_9
/content/books/9781789389760.c08
/content/books/9781789389760.c08
Loading

Data & Media loading...

References

  1. Bell, Peter and Offert, Fabian (2021), ‘Reflections on connoisseurship and computer vision’, Journal of Art Historiography, 24, pp. 1–9.
     [Google Scholar]
  2. Bell, Peter and Ommer, Björn (2016), ‘Digital connoisseur? How computer vision supports art history’, in A. Aggujaro and S. Albl (eds), Connoisseurship nel XXI secolo. Approcci, Limiti, Prospettive, Rome: Artemide, pp. 187–200.
     [Google Scholar]
  3. Bell, Peter , Schlecht, Joseph and Ommer, Björn (2013), ‘Nonverbal communication in medieval illustrations revisited by computer vision and art history’, Visual Resources, 29:1–2, pp. 26–37.
     [Google Scholar]
  4. Bishop, Claire (2018), ‘Against digital art history’, International Journal for Digital Art History, 3, pp. 122–31, https://doi.org/10.11588/dah.2018.3.49915.
     [Google Scholar]
  5. Caraffa, Costanza , Pugh, Emily , Stuber, Tracy and Ruby, Louisa Wood (2020), ‘PHAROS: A digital research space for photo archives’, Art Libraries Journal, 45:1, pp. 2–11.
     [Google Scholar]
  6. Deng, Jia , Dong, Wei, Socher, Richard, Li, Li-Jia, Li, Kai and Fei-Fei, Li (n.d.), ‘ImageNet: A large-scale hierarchical image database’, https://image-net.org/static_files/papers/imagenet_cvpr09.pdf. Accessed 5 March 2023 .
  7. Drimmer, Sonja (2021), ‘How AI is hijacking art history’, The Conversation, 21 November, https://theconversation.com/how-ai-is-hijacking-art-history-170691. Accessed 9 October 2023 .164
  8. Elgammal, Ahmed (2014), ‘Computer science can only help – not hurt – art historians’, The Conversation, 4 December, https://theconversation.com/computer-science-can-only-help-not-hurt-art-historians-33780. Accessed 4 October 2023 .
  9. Elgammal, Ahmed , Liu, Bingchen , Kim, Diana , Elhoseiny, Mohamed and Mazzone, Marian (2018), ‘The shape of art history in the eyes of the machine’, in Thirty-Second AAAI Conference on Artificial Intelligence, Cambridge: AAAI Press, pp. 2183–91.
     [Google Scholar]
  10. Elsner, Jas (2006), ‘From empirical evidence to the big picture: Some reflections on Riegl's concept of Kunstwollen’, Critical Inquiry, 32:4, pp. 741–66.
     [Google Scholar]
  11. Fernie, Eric (1995), Art History and Its Methods, London: Phaidon.
    [Google Scholar]
  12. Greenberger, Alex (2023), ‘A painting attributed to Raphael by AI is questioned by experts as contradictory study emerges’, ARTnews, 11 September, https://www.artnews.com/art-news/news/ai-art-artificial-intelligence-attributed-raphael-painting-questioned-experts-contradictory-study-1234679282/. Accessed 13 October 2023 .
  13. Herlihy, David (1992), ‘Computer-assisted analysis of the statistical documents of medieval society’, in J. Powell (ed.), Medieval Studies: An Introduction, 2nd ed., Syracuse: Syracuse University Press, pp. 227–51.
     [Google Scholar]
  14. Hinojosa, Lynne Walhout (2009), ‘The connoisseur and the spiritual history of art: Morelli and Berenson’, in L. Hinojosa (ed.), The Renaissance, English Cultural Nationalism, and Modernism, 1860–1920, New York: Palgrave Macmillan, pp. 89–111.
     [Google Scholar]
  15. Impett, Leonardo (2020), ‘Analyzing gesture in digital art history’, in K. Brown (ed.), Routledge Handbook of Digital Humanities and Art History, New York: Routledge, pp. 386–407.
     [Google Scholar]
  16. Impett, Leonardo and Offert, Fabian (2023), ‘There is a digital art history’, arXiv preprint, https://doi.org/10.48550/arXiv.2308.07464.
     [Google Scholar]
  17. Iversen, Margaret (1993), Alois Riegl: Art History and Theory, Cambridge: MIT Press.
     [Google Scholar]
  18. Jones, Owen (2016), Grammar of Ornament, Princeton: Princeton University Press.
     [Google Scholar]
  19. Lang, Sabine and Ommer, Björn (2018), ‘Attesting similarity: Supporting the organization and study of art image collections with computer vision’, Digital Scholarship in the Humanities, 33:4, pp. 845–56.
     [Google Scholar]
  20. Lang, Sabine and Ommer, Björn (2019), ‘Reflecting on how artworks are processed and analyzed by computer vision’, in L. Leal-Taixé and S. Roth (eds), Computer Vision – ECCV 2018 Workshops. ECCV 2018. Lecture Notes in Computer Science, Berlin: Springer Verlag, pp. 647–52.
     [Google Scholar]
  21. Lang, Sabine and Ommer, Björn (2021), ‘Transforming information into knowledge: How computational methods reshape art history’, Digital Humanities Quarterly, 15:3, http://digitalhumanities.org/dhq/vol/15/3/000560/000560.html. Accessed 29 March 2024.
     [Google Scholar]
  22. Langmead, Alison , Nygren, Christopher , Rodriguez, Paul and Craig, Alan (2021), ‘Leonardo, Morelli, and the computational mirror’, Digital Humanities Quarterly, 15:1, http://www.digitalhumanities.org/dhq/vol/15/1/000540/000540.html. Accessed 29 March 2024.
     [Google Scholar]
  23. Mansfield, Elizabeth , Zhang, Zhuomin , Li, Jia , Russell, John , Young, George S. , Adams, Catherine and Wang, James Z. (2022), ‘Techniques of the art historical observer’, Nineteenth-Century Art Worldwide, 21:1, pp. 137–49, https://doi.org/10.29411/ncaw.2022.21.1.5.165
     [Google Scholar]
  24. Manovich, Lev (2011), ‘Style space: How to compare image sets and follow their evolution’, http://manovich.net/content/04-projects/073-style-space/70_article_2011.pdf. Accessed 19 February 2023 .
  25. Mercuriali, Giacomo (2018), ‘Computational imagination and digital art history’, International Journal for Digital Art History, 3, pp. 140–51, https://doi.org/10.11588/dah.2018.3.47287.
     [Google Scholar]
  26. Näslund Dahlgren, Anna and Wasielewski, Amanda (2021), ‘The digital U-turn in art history’, Konsthistorisk Tidskrift, 90:4, pp. 249–66.
  27. Necipoğlu, Gülru and Payne, Alina (eds) (2016), Histories of Ornament: From Global to Local, Princeton: Princeton University Press.
    [Google Scholar]
  28. Nygren, Christopher and Drimmer, Sonja (2023), ‘AI and art history: Ten axioms’, International Journal for Digital Art History, 9, pp. 5.02–5.13, https://doi.org/10.11588/dah.2023.9.90400.
     [Google Scholar]
  29. Ommer, Björn , Bell, Peter and Arnold, Michael (n.d.), COMPOSITO, https://hci.iwr.uni-heidelberg.de/content/composito-arthistorical-analysis-architecture-computer-vision. Accessed 5 March 2023 .
  30. Podro, Michael (1982), The Critical Historians of Art, New Haven: Yale University Press.
    [Google Scholar]
  31. Pollock, Griselda (2014), ‘Computers can find similarities between paintings – but art history is about so much more’, The Conversation, 22 August, https://theconversation.com/computers-can-find-similarities-between-paintings-but-art-history-is-about-so-much-more-30752. Accessed 4 October 2023 .
  32. Riegl, Riegl ([1893] 1992), Problems of Style: Foundations for the History of Ornament (ed. D. Castriota, trans. E. Kain), Princeton: Princeton University Press.
    [Google Scholar]
  33. Rodriguez, Paul , Craig, Alan , Langmead, Alison and Nygren, Christopher (2020), ‘Extracting and analyzing deep learning features for discriminating historical art’, in Practice and Experience in Advanced Research Computing (PEARC '20), 26–30 July, New York: ACM, pp. 358–63.
     [Google Scholar]
  34. Rodríguez-Ortega, Nuria (2020), ‘Image processing and computer vision in the field of art history: Overview, challenges, and critical inquiries’, in K. Brown (ed.), The Routledge Companion to Digital Humanities and Art History, New York: Routledge, pp. 338–57.
     [Google Scholar]
  35. Ruggles, Seven and Magnuson, Diana L. (2019), ‘The history of quantification in history: The JIH as a case study’, The Journal of Interdisciplinary History, 50:3, pp. 363–81.
     [Google Scholar]
  36. Stork, David (2009), ‘Computer vision and computer graphics analysis of paintings and drawings: An introduction to the literature’, in CAIP ‘09: Proceedings of the 13th International Conference on Computer Analysis of Images and Patterns, Berlin: Springer Verlag, pp. 9–24.
     [Google Scholar]
  37. Ufer, Nikolai , Simon, Max , Lang, Sabine and Ommer, Björn (2021), ‘Large-scale interactive retrieval in art collections using multi-style feature aggregation’, PLoS ONE, 16:11, pp. 1–38, https://doi.org/10.1371/journal.pone.0259718.
     [Google Scholar]
  38. Vaughan, William (1987), ‘The automated connoisseur: Image analysis and art history’, in P. Denley and D. Hopkin (eds), History and Computing, Manchester: Manchester University Press, pp. 215–21.166
     [Google Scholar]
  39. Vaughan, William (1992), ‘Automated picture referencing: A further look at “Morelli”’, Computers and the History of Art, 2:2, pp. 7–18.
     [Google Scholar]
  40. Wang, James Z , Kandemir, Baris and Li, Jai (2020), ‘Computerized analysis of paintings’, in K. Brown (ed.), The Routledge Handbook of Digital Humanities and Art History, New York: Routledge, pp. 299–312 .
    [Google Scholar]
  41. Wasielewski, Amanda (2023), Computational Formalism: Art History and Machine Learning, Cambridge: MIT University Press.
    [Google Scholar]
  42. Wood, Christopher S. (ed.) (2003), The Vienna School Reader: Politics and Art Historical Method in the 1930s, New York: Zone Books.
    [Google Scholar]
  43. X Degrees of Separation, https://artsexperiments.withgoogle.com/xdegrees/. Accessed 7 April 2023.167

References

  1. Bell, Peter and Offert, Fabian (2021), ‘Reflections on connoisseurship and computer vision’, Journal of Art Historiography, 24, pp. 1–9.
     [Google Scholar]
  2. Bell, Peter and Ommer, Björn (2016), ‘Digital connoisseur? How computer vision supports art history’, in A. Aggujaro and S. Albl (eds), Connoisseurship nel XXI secolo. Approcci, Limiti, Prospettive, Rome: Artemide, pp. 187–200.
     [Google Scholar]
  3. Bell, Peter , Schlecht, Joseph and Ommer, Björn (2013), ‘Nonverbal communication in medieval illustrations revisited by computer vision and art history’, Visual Resources, 29:1–2, pp. 26–37.
     [Google Scholar]
  4. Bishop, Claire (2018), ‘Against digital art history’, International Journal for Digital Art History, 3, pp. 122–31, https://doi.org/10.11588/dah.2018.3.49915.
     [Google Scholar]
  5. Caraffa, Costanza , Pugh, Emily , Stuber, Tracy and Ruby, Louisa Wood (2020), ‘PHAROS: A digital research space for photo archives’, Art Libraries Journal, 45:1, pp. 2–11.
     [Google Scholar]
  6. Deng, Jia , Dong, Wei, Socher, Richard, Li, Li-Jia, Li, Kai and Fei-Fei, Li (n.d.), ‘ImageNet: A large-scale hierarchical image database’, https://image-net.org/static_files/papers/imagenet_cvpr09.pdf. Accessed 5 March 2023 .
  7. Drimmer, Sonja (2021), ‘How AI is hijacking art history’, The Conversation, 21 November, https://theconversation.com/how-ai-is-hijacking-art-history-170691. Accessed 9 October 2023 .164
  8. Elgammal, Ahmed (2014), ‘Computer science can only help – not hurt – art historians’, The Conversation, 4 December, https://theconversation.com/computer-science-can-only-help-not-hurt-art-historians-33780. Accessed 4 October 2023 .
  9. Elgammal, Ahmed , Liu, Bingchen , Kim, Diana , Elhoseiny, Mohamed and Mazzone, Marian (2018), ‘The shape of art history in the eyes of the machine’, in Thirty-Second AAAI Conference on Artificial Intelligence, Cambridge: AAAI Press, pp. 2183–91.
     [Google Scholar]
  10. Elsner, Jas (2006), ‘From empirical evidence to the big picture: Some reflections on Riegl's concept of Kunstwollen’, Critical Inquiry, 32:4, pp. 741–66.
     [Google Scholar]
  11. Fernie, Eric (1995), Art History and Its Methods, London: Phaidon.
    [Google Scholar]
  12. Greenberger, Alex (2023), ‘A painting attributed to Raphael by AI is questioned by experts as contradictory study emerges’, ARTnews, 11 September, https://www.artnews.com/art-news/news/ai-art-artificial-intelligence-attributed-raphael-painting-questioned-experts-contradictory-study-1234679282/. Accessed 13 October 2023 .
  13. Herlihy, David (1992), ‘Computer-assisted analysis of the statistical documents of medieval society’, in J. Powell (ed.), Medieval Studies: An Introduction, 2nd ed., Syracuse: Syracuse University Press, pp. 227–51.
     [Google Scholar]
  14. Hinojosa, Lynne Walhout (2009), ‘The connoisseur and the spiritual history of art: Morelli and Berenson’, in L. Hinojosa (ed.), The Renaissance, English Cultural Nationalism, and Modernism, 1860–1920, New York: Palgrave Macmillan, pp. 89–111.
     [Google Scholar]
  15. Impett, Leonardo (2020), ‘Analyzing gesture in digital art history’, in K. Brown (ed.), Routledge Handbook of Digital Humanities and Art History, New York: Routledge, pp. 386–407.
     [Google Scholar]
  16. Impett, Leonardo and Offert, Fabian (2023), ‘There is a digital art history’, arXiv preprint, https://doi.org/10.48550/arXiv.2308.07464.
     [Google Scholar]
  17. Iversen, Margaret (1993), Alois Riegl: Art History and Theory, Cambridge: MIT Press.
     [Google Scholar]
  18. Jones, Owen (2016), Grammar of Ornament, Princeton: Princeton University Press.
     [Google Scholar]
  19. Lang, Sabine and Ommer, Björn (2018), ‘Attesting similarity: Supporting the organization and study of art image collections with computer vision’, Digital Scholarship in the Humanities, 33:4, pp. 845–56.
     [Google Scholar]
  20. Lang, Sabine and Ommer, Björn (2019), ‘Reflecting on how artworks are processed and analyzed by computer vision’, in L. Leal-Taixé and S. Roth (eds), Computer Vision – ECCV 2018 Workshops. ECCV 2018. Lecture Notes in Computer Science, Berlin: Springer Verlag, pp. 647–52.
     [Google Scholar]
  21. Lang, Sabine and Ommer, Björn (2021), ‘Transforming information into knowledge: How computational methods reshape art history’, Digital Humanities Quarterly, 15:3, http://digitalhumanities.org/dhq/vol/15/3/000560/000560.html. Accessed 29 March 2024.
     [Google Scholar]
  22. Langmead, Alison , Nygren, Christopher , Rodriguez, Paul and Craig, Alan (2021), ‘Leonardo, Morelli, and the computational mirror’, Digital Humanities Quarterly, 15:1, http://www.digitalhumanities.org/dhq/vol/15/1/000540/000540.html. Accessed 29 March 2024.
     [Google Scholar]
  23. Mansfield, Elizabeth , Zhang, Zhuomin , Li, Jia , Russell, John , Young, George S. , Adams, Catherine and Wang, James Z. (2022), ‘Techniques of the art historical observer’, Nineteenth-Century Art Worldwide, 21:1, pp. 137–49, https://doi.org/10.29411/ncaw.2022.21.1.5.165
     [Google Scholar]
  24. Manovich, Lev (2011), ‘Style space: How to compare image sets and follow their evolution’, http://manovich.net/content/04-projects/073-style-space/70_article_2011.pdf. Accessed 19 February 2023 .
  25. Mercuriali, Giacomo (2018), ‘Computational imagination and digital art history’, International Journal for Digital Art History, 3, pp. 140–51, https://doi.org/10.11588/dah.2018.3.47287.
     [Google Scholar]
  26. Näslund Dahlgren, Anna and Wasielewski, Amanda (2021), ‘The digital U-turn in art history’, Konsthistorisk Tidskrift, 90:4, pp. 249–66.
  27. Necipoğlu, Gülru and Payne, Alina (eds) (2016), Histories of Ornament: From Global to Local, Princeton: Princeton University Press.
    [Google Scholar]
  28. Nygren, Christopher and Drimmer, Sonja (2023), ‘AI and art history: Ten axioms’, International Journal for Digital Art History, 9, pp. 5.02–5.13, https://doi.org/10.11588/dah.2023.9.90400.
     [Google Scholar]
  29. Ommer, Björn , Bell, Peter and Arnold, Michael (n.d.), COMPOSITO, https://hci.iwr.uni-heidelberg.de/content/composito-arthistorical-analysis-architecture-computer-vision. Accessed 5 March 2023 .
  30. Podro, Michael (1982), The Critical Historians of Art, New Haven: Yale University Press.
    [Google Scholar]
  31. Pollock, Griselda (2014), ‘Computers can find similarities between paintings – but art history is about so much more’, The Conversation, 22 August, https://theconversation.com/computers-can-find-similarities-between-paintings-but-art-history-is-about-so-much-more-30752. Accessed 4 October 2023 .
  32. Riegl, Riegl ([1893] 1992), Problems of Style: Foundations for the History of Ornament (ed. D. Castriota, trans. E. Kain), Princeton: Princeton University Press.
    [Google Scholar]
  33. Rodriguez, Paul , Craig, Alan , Langmead, Alison and Nygren, Christopher (2020), ‘Extracting and analyzing deep learning features for discriminating historical art’, in Practice and Experience in Advanced Research Computing (PEARC '20), 26–30 July, New York: ACM, pp. 358–63.
     [Google Scholar]
  34. Rodríguez-Ortega, Nuria (2020), ‘Image processing and computer vision in the field of art history: Overview, challenges, and critical inquiries’, in K. Brown (ed.), The Routledge Companion to Digital Humanities and Art History, New York: Routledge, pp. 338–57.
     [Google Scholar]
  35. Ruggles, Seven and Magnuson, Diana L. (2019), ‘The history of quantification in history: The JIH as a case study’, The Journal of Interdisciplinary History, 50:3, pp. 363–81.
     [Google Scholar]
  36. Stork, David (2009), ‘Computer vision and computer graphics analysis of paintings and drawings: An introduction to the literature’, in CAIP ‘09: Proceedings of the 13th International Conference on Computer Analysis of Images and Patterns, Berlin: Springer Verlag, pp. 9–24.
     [Google Scholar]
  37. Ufer, Nikolai , Simon, Max , Lang, Sabine and Ommer, Björn (2021), ‘Large-scale interactive retrieval in art collections using multi-style feature aggregation’, PLoS ONE, 16:11, pp. 1–38, https://doi.org/10.1371/journal.pone.0259718.
     [Google Scholar]
  38. Vaughan, William (1987), ‘The automated connoisseur: Image analysis and art history’, in P. Denley and D. Hopkin (eds), History and Computing, Manchester: Manchester University Press, pp. 215–21.166
     [Google Scholar]
  39. Vaughan, William (1992), ‘Automated picture referencing: A further look at “Morelli”’, Computers and the History of Art, 2:2, pp. 7–18.
     [Google Scholar]
  40. Wang, James Z , Kandemir, Baris and Li, Jai (2020), ‘Computerized analysis of paintings’, in K. Brown (ed.), The Routledge Handbook of Digital Humanities and Art History, New York: Routledge, pp. 299–312 .
    [Google Scholar]
  41. Wasielewski, Amanda (2023), Computational Formalism: Art History and Machine Learning, Cambridge: MIT University Press.
    [Google Scholar]
  42. Wood, Christopher S. (ed.) (2003), The Vienna School Reader: Politics and Art Historical Method in the 1930s, New York: Zone Books.
    [Google Scholar]
  43. X Degrees of Separation, https://artsexperiments.withgoogle.com/xdegrees/. Accessed 7 April 2023.167

  • From This Site

    /content/books/9781789389760.c08
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
/content/books/9781789389760.c08
dcterms_title,dcterms_subject,pub_keyword
-contentType:Contributor -contentType:Concept -contentType:Institution
10
5
Chapter
content/books/9781789389760
Book
false
en
Loading
This is a required field
Please enter a valid email address
Approval was a success
Invalid data
An error occurred
Approval was partially successful, following selected items could not be processed due to error
Please enter a valid_number test