Assessing the impact of the S+T+ARTS framework. Main evidence and reflections to inform the community and policymakers for supporting artistic research

Artistic research holds significant potential by addressing contemporary challenges at societal, environmental, and technological levels. The aim of this paper is to contribute to the understanding of the impact of artistic research by assessing a framework funded since 2016 by the European Commission called Science, Technology and Arts (=S+T+ARTS). The discussion assesses the impact of the S+T+ ARTS framework by focusing on its ability to bridge science, technology, and artistic practice, reaching significant impacts at different levels. Through a comprehensive evaluation methodology, the study examines various dimensions of impact, including social, economic, technological, political, and environmental aspects. Findings suggest a pronounced social impact, evidenced by enhanced networking, interdisciplinary collaboration, and engagement with vulnerable groups. While economic impacts are less tangible, the framework underscores the need for innovative indicators to capture the economic value of artistic research accurately. Overall, the study advocates for (i) continued support and exploration of artistic research frameworks to address contemporary societal challenges effectively and (ii) the development of ad-hoc methodologies to map the assessment of such initiatives.

Introduction

Over the last three decades there has been a development of highly active artistic research contexts at a European and international level. Specific initiatives are changing the landscape of traditional research practice and are emphasising the dynamic character, potential and usefulness of artistic research¹ (Tytgat and Larsen, 2024). Nonetheless, the impact of artistic research remains underexplored and its potential untapped. Artistic research (De Assis, 2018; De Assis and D' Errico, 2019) differs from other types of research in that it requires planning, support, and administration diverse from other more traditional research areas. There is indeed an urgent need to promote, support and empower artistic research as it can be instrumental in addressing the complexities of our times at a climate, digital, social level and beyond.² Within this context, reducing the gap between scientific research and artistic practice is one of the objectives of the European Commission (EC) through the funding of the Science, Technology and Arts (=S+T+ARTS) framework,³ an initiative to foster alliances between science, technology and artistic practice.

Since 2016 the S+T+ARTS Prize has been awarded to the most pioneering collaborations and results in the field of creativity and innovation at the intersection of science and technology with the arts.⁴ Science, technology, and the arts form a nexus with an extraordinarily high potential for creative innovation. As reported on the S+T+ARTS website and on the open calls, the role of artists is no longer seen to be only about propagating scientific and technological knowledge and skills among the general public but rather as a kind of catalyst that can inspire and trigger innovative processes.

The innovative process implemented by the S+T+ARTS Prize⁵ has been very relevant to boost discussion and awareness of the role of the arts fostering scientific and technological outcomes. However, even if the importance of the S+T+ARTS Prize is well known and intuitive, no concrete quantification of the impacts of the Prize have been calculated so far. This makes all the more urgent the need to analyse the impact of the S+T+ARTS framework. Accordingly, during the S+T+ARTS Prize 2021–2023,⁶ a specific task has been dedicated to designing a framework for the impact assessment (De Rosa et al., 2022) and to validate the methodology through its testing (Diakou et al., 2023). The research question driving the impact assessment has been the following: Which is the impact of an EU funding programme aimed at stimulating the connection between arts, science, and technology? Which are the main results and main barriers?⁷

Theoretical background

In a paper published in 2019, Bublitz et al. state: “we identify a purposeful benefit of aesthetics and art: it is a solution to a problem, specifically, the vehicle through which community engagement and transformation may occur” (Bublitz et al., 2019). In the case of the S+T+ARTS, the arts are intended as a tool to foster certain topics; however, there is an additional element to consider, namely technology. The combination of both science and research through the use and application of technologies makes a critical difference thanks to the unique contribution of the arts. But what is the impact of this powerful combination, specifically supported by the S+T+ARTS initiative?

In the specific case of artistic research, even if impacts can be difficult to measure, “the value and importance of arts activities should not be negated simply because they are difficult to express using conventional systems” (Reeves, 2002: 37). There are, indeed, problems in trying to measure the impacts of the arts. Among others, measurements are often viewed as a validating experience even when aspects of artistic practice and research are often difficult to quantify (Moriarty, 1997; Williams, 1997). In this sense, the definition of a proper and balanced approach is crucial to be able to measure all dimensions. In the case of the S+T+ARTS Prize project, the scope of the evaluation framework for impact assessment was to measure the impact of the S+T+ARTS Prize, providing structured information on how the S+T+ARTS Prize impacts narratives and actions related to science, technology, and the arts.

The impact assessment framework for the S+T+ARTS Prize 2021–2023

To achieve the above-mentioned objectives, a flexible, adaptable, and iterative methodology has been developed to match the specificities of the S+T+ARTS ecosystem using a co-design approach engaging the project partners in the definition of dimensions and indicators to be assessed⁸. The International Association for Impact Assessment defines impact assessment as “the process of identifying the future consequences of a current or proposed action.” In other words, the “impact is the difference between what would happen with the action and what would happen without it” (IAIA, 2009: 1). Looking at the impact assessment of the S+T+ARTS Prize, the aim of the analysis was to explore the type and magnitude of difference that the S+T+ARTS Prize makes and the types of stakeholders that it affects. To map the impact of the project, the adopted framework is based on the impact value chain approach (Figure 1).

Figure 1

Figure 1. Impact assessment framework: the value-chain approach (elaboration on SIIT, 2014:6).

Following the impact value chain approach, the framework has been adapted to reflect the specificities of the S+T+ARTS Prize and of the S+T+ARTS ecosystem in general (Figure 2).

Figure 2

Figure 2. Impact assessment framework for the S+T+ARTS Prize.

The impact assessment of the S+T+ARTS Prize 2021–2023 project

The S+T+ARTS impact assessment framework considers 5 areas of impact - social, economic, technological, political, and environmental - and reflects on different typologies of stakeholders. Each impact area corresponds to several dimensions and indicators that were translated into questions. Data was collected through a combination of desk research, questionnaires and interviews with the winners and honorary mentions, in order to ensure the gathering of both qualitative and quantitative data. Following the data collection, each project was analysed aggregating the interview transcription and the questionnaire with the notes from desk research. Each winner and honorary mention have been investigated following a case study approach, allowing us to identify the most relevant impacts for each project. Only after having completed the analysis of the individual case studies, have we been able to aggregate the main dimensions of impact mapping and to provide a broader comprehension of the impact of the S+T+ARTS Prize⁹.

According to the data analysis, the majority of the projects (35%) believe that the social dimension is the most important one in terms of impact, followed by the environmental dimension (26%) and less so by the scientific, technological (16% each) and economic dimensions (7%) (Figure 3). The emphasis on social impact emerged clearly in the interviews as participants expect that their projects will have an impact on societal and environmental challenges through raising awareness and educating, in addition to involvement and empowerment. The low importance of the economic impact of the S+T+ARTS Prize on the projects was also noted, as most participants do not view their outcomes as products to be commercially exploited. While this information must be considered in its self-assessment value, it is highly significant as it indicates what is perceived as the most relevant impact areas from the perspective of the participants themselves.

Figure 3

Figure 3. Pie chart with the most important impact dimensions according to the S+T+ARTS Prize projects 2021–2023.

One of the most relevant social impact dimensions of the S+T+ARTS Prize on the projects is on bridging and bonding social capital, i.e. the capability of the teams to create connections and collaborations with different typologies of stakeholders, both within and outside their sector. Seventy percent of the interviewed teams observed a positive correlation between receiving the Prize and networking with stakeholders from the same sector, while a smaller percentage referred to networking with stakeholders from different sectors and in particular the civic, public, industry and business sector. The ability to create such collaborations was connected by the teams to the enhanced visibility and recognition offered by the S+T+ARTS Prize, specifically through participation in the Ars Electronica festival¹⁰.

Highly relevant to this is the interdisciplinarity that characterises all the projects and goes beyond work at the intersection of science, technology, and art. The discussion with the teams highlighted how receiving the S+T+ARTS Prize, an award that celebrates interdisciplinarity at its core, has an impact on the artists as it validates and enhances their work that is more fluid between sectors. Interdisciplinarity, especially in the way that it is promoted by the S+T+ARTS Prize, is essential for innovation and for pushing the boundaries of scientific and technological research.

Another important aspect that emerged is the impact on social inclusion and the role of citizens in the projects. While social inclusion was not understood in a uniform way by all projects, a significant number of the projects (9 projects, while an additional 3 teams would like to establish such strategies in the future) targeted vulnerable groups such as minorities, women, people with disabilities, people with different ethnic and cultural backgrounds, LGBTQ+ community, immigrants, etc.

The majority of the projects honoured by the S+T+ARTS Prize employed a range of strategies for citizen engagement. The nature and degree of involvement of citizens varied depending on the target of each project and its outcome. Some projects worked closely with local communities and involved them in the collection of information; other projects targeted the issues of data dignity, data sovereignty and data accessibility, while others engaged students and researchers through the dissemination of their scientific outcomes and the public through various modes of presenting their work. The overall goals of citizen engagement were to inform, involve, empower, and collaborate with the public. While the intention of the projects to have an impact on citizens is there, the lack of clear examples and quantitative data illustrating the impact on citizens indicates that more should be done in order to support these kinds of projects in developing a social inclusion strategy and a more structured way of gathering feedback from users.

The economic impact of the S+T+ARTS Prize on the projects and its subdimensions has been more difficult to explore and is often considered a more sensitive topic to discuss. The most relevant dimension that emerged clearly in the interviews, from the perspective of the projects is the impact on visibility and recognition. This would potentially translate into an economic impact since it could lead to more connections and opportunities. In fact, this has been the most recognisable impact of the S+T+ARTS Prize on the projects, observed at various degrees of intensity, but commented on by almost all the projects that we interviewed. Sixteen projects commented positively that the award of the S+T+ARTS Prize, either the Grand Prize or the Honorary Mention has had an impact on their visibility and recognition. The S+T+ARTS Prize functions as a “seal of approval” for the projects and offers recognition, validated by the EC, for projects that are at the intersection of science, technology, and art. Consequently, the Prize validates the strong interdisciplinary nature of the projects and the ability of the creators to produce innovative work at the intersection of science, technology, and the arts. Participants noted that with the Prize it is easier to form collaborations with potential partners and institutions and to move forward their projects.

Another interesting aspect of the economic dimension is the impact on business and commercial exploitation. For the majority of the projects this was not a priority and for some, not even a possibility. A potential reason for this, lies in the open-source character of many projects and their objective of contributing to digital commons rather than private driven profits. In addition, there is the perception that commercial exploitation may compromise the integrity of the artist and the project. It is important to stress that the S+T+ARTS Prize does not entail specific activities that would help the development of this aspect, something that was commented on by the participants. While some teams would welcome the opportunity to explore this aspect further, especially as it could lead to further income and help with the sustainability of the projects and the continuation of the relevant research, more support would be needed from the S+T+ARTS ecosystem.

Environmental impact is at the core of interest in many of the S+T+ARTS Prize projects. However, sometimes in artistic projects the object addressed on an aesthetic-artistic level does not always reflect an ecological practice within the teams. Thus, the environmental impact of the S+T+ARTS Prize on the projects has been explored from the point of view of raising awareness to citizens and potentially leading to behavioural changes towards the establishment of sustainable practices. The majority of the projects deal with various iterations of environmental issues and target human-made destruction of ecosystems, energy crisis, biobased materials, recycling food systems, biodiversity, data warming and climate change. It was possible to observe that projects indeed targeted environmental challenges and aimed to inform and educate the public on these issues, leading to policy change. However, despite the fact that a number of projects showcased sustainable practices in terms of energy production, food systems, material production, etc., it is not possible to determine whether the honoured projects had an impact on the use of sustainable practices by citizens.

The technological impact of the S+T+ARTS Prize has been explored in view of the range of innovative technologies used by the projects honoured by the Prize. This is related to project conceptualization, as the S+T+ARTS Prize is a recognition programme of existing projects rather than an innovation programme to deliver technological developments. In other words, the award of the S+T+ARTS Prize did not facilitate the production of new scientific knowledge or innovative technologies but rather it recognized projects that are producing cutting-edge advancements in this regard. In order to explore this aspect from the perspective of the projects, we identified the most innovative aspects in terms of technology employed by the projects, such as biotechnology, machine learning and blockchain and new algorithms. It should be noted however that in the interviews, the majority of the teams did not view the technological dimension as a significant impact area for their project. Rather, teams used innovative technologies in order to promote environmental, social, and political issues.

Conclusion

To conclude, even if the methodology and data collection used has shortcomings that should be addressed in future implementation, the analysis allowed us to better understand the limits and barriers of such a framework and provide some suggestions on actions to be undertaken to improve it. The results reported in this paper can be useful for institutions that intend to promote specific frameworks on artistic research; additionally, they provide insights on which is the potentiality of the projects analysed, and which are the difficulties encountered by the stakeholders engaged in artistic research.

The analysis of the data shows that the most readily observed and identifiable impact dimension is the social one. The award of the S+T+ARTS Prize has a stimulating effect on increasing the social capital of the teams and results in new connections and collaborations with various stakeholders both within and outside the participants’ areas of expertise. The S+T+ARTS Prize, an award that celebrates interdisciplinarity at its core, has a great impact on the artists as it validates and enhances their work that is more fluid between sectors and their role in producing innovation. Within the social dimension we should also emphasise that many projects target underrepresented and vulnerable groups at risk of social exclusion. In addition, projects employ diverse strategies for social engagement and involve citizens at various stages of the projects from collecting information to being educated about various issues. The overwhelming majority of the projects target societal challenges including environmental and human rights issues. Indeed, the social dimension of the projects has been mentioned by the majority of the participants to the interviews as the most significant area of impact. Education and awareness on social issues ranging from social exclusion to data ethics and the effects of the environmental crisis on our society (e.g., energy and food crisis) are two of the most commonly mentioned goals brought up by the respondents, in addition to involvement and empowerment. Artistic research, especially within a supported framework, such as the one offered by the S+T+ARTS Prize is indeed a powerful tool to explore innovative ways to deal with the current societal challenges that the world is facing.

The economic impact of the S+T+ARTS Prize emerges as less relevant and as a dimension that is more complex to track and quantify. Indeed, we are dealing with an epistemological gap since on the one hand the EC and policymakers emphasise the need to map the economic impact of artistic practices while on the other artists and creatives are seemingly not interested or able in measuring the economic impact of their work. To this we should also add the fact that the impact of artistic research is difficult to be assessed through traditional economic indicators. The work carried out as part of the S+T+ARTS Prize, both within the context of the impact assessment as well as through policy work (De Rosa and Liebl, 2023) highlights the need to rethink the economic impact of the S+T+ARTS framework and the urgent necessity to explore new indicators that can better capture the potential economic results of the work carried out. The translation of aspects of economic impact that better reflect the essence of artistic research into concrete impacts for the EC and policymakers will ensure that frameworks such as the S+T+ARTS will continue to receive the financial support that is necessary for their growth.

Data availability statement

The datasets presented in this article are not readily available because Data are anonymously reported but the dataset contains personal data which is why it is not available for sharing. Requests to access the datasets should be directed to the corresponding author.

Author contributions

All authors participated in the design, interpretation of the studies and analysis of the data and review of the manuscript. In particular, SiD worked on the definition of the methodological framework; StD focused on the data collection and performed the analysis. The authors worked together on data interpretations deriving main results and conclusions. All authors contributed to the article and approved the submitted version.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. The authors declare that this study received funding from the European Union’s Horizon 2020 research and innovation programme under GA No 956603 for the project S+T+ARTS Prize 2021–2023. This publication reflects the views only of the authors, and the European Commission cannot be held responsible for any use which may be made of the information contained therein. The funder was not involved in the study design, collection, analysis, interpretation of data, the writing of this article or the decision to submit it for publication.

Conflict of interest

Authors StD and SiD are employed by T6 Ecosystems.

Footnotes

¹For a discussion on the definition of artistic research see Klein 2010.

²For the call for papers for this special issue see: https://www.frontierspartnerships.org/research-topics/80/unlocking-artistic-research---how-to-reframe-and-transform-the-current-challenges-of-managing-and-supporting-artistic-research-by-discovering-and-imagining-new-forms-of-impact-and-value

³For the S+T+ARTS framework see https://culture.ec.europa.eu/node/1186. The website of the S+T+ARTS initiative can be accessed at https://starts.eu/about/

⁴For an archive of all projects honored with the S+T+ARTS Prize and supported by the S+T+ARTS initiative see https://archive.aec.at/startsprize/

⁵For the S+T+ARTS Prize see https://ars.electronica.art/starts-prize/en/

⁶https://cordis.europa.eu/project/id/956603

⁷The task has been carried out by T6 Ecosystems, a consortium partner in the S+T+ARTS Prize 2021–2023. The developed methodology has been applied during the 3 years of project lifetime and tested over the 6 winners and 30 honorary mentions that have been honoured with the S+T+ARTS Prize between 2021 and 2023.

⁸The theoretical and methodological aspects of the framework have been inspired by previous work on European projects such as SEQUOIA (Passani et al., 2014), MAXICULTURE, IA4SI (Bellini et al., 2014; 2016; Passani et al., 2016), ACTION (Passani et al., 2020) and MediaFutures (Passani and De Rosa, 2021). The selection of dimensions and indicators has been led by an extensive literature review and several rounds of experimentation and refinement (De Rosa et al., 2022).

⁹According to the privacy agreements with the teams, all data are presented here in an aggregated and anonymous manner so as to ensure the privacy of the participants.

¹⁰At the time of data collection many teams were in the process of forming collaborations without concrete results yet; indeed, the impact of the S+T+ARTS Prize is expected to increase in the months following the end of the project.

References

Bellini, F., Passani, A., Klitsi, M., and Vanobbergher, W. (2016). Exploring the impact of collective awareness platforms for sustainability and social innovation. Rome: Eurokleis press.

Google Scholar

Bellini, F., Passani, A., Spagnoli, F., Crombie, D., and Ioannidis, G. (2014). MAXICULTURE: assessing the impact of EU projects in the digital cultural heritage domain. In digital heritage. Progress in cultural heritage: documentation, preservation, and protection. Lect. Notes Comput. Sci. 8740, 364–373. doi:10.1007/978-3-319-13695-0_35

CrossRef Full Text | Google Scholar

Bublitz, M. G., Rank-Christman, T., Cian, L., Cortada, X., Madzharov, A., Patrick, V. M., et al. (2019). Collaborative art: a transformational force within communities. J. Assoc. Consumer Res. 4 (4), 313–331. doi:10.1086/705023

CrossRef Full Text | Google Scholar

De Assis, P. (2018). “Experimental systems and artistic research,” in Logic of experimentation (Leuven: Leuven University Press), 107–122.

Google Scholar

De Assis, P., and D’ Errico, L. (2019). Artistic Research: charting a field in expansion. Lanham: Rowman & Littlefield International.

Google Scholar

De Rosa, S., and Liebl, V. S+T+ARTS Prize policy recommendations: for a systematic ecosystem’s growth (2023).

Google Scholar

De Rosa, S., Saracco, F., and Carradore, R. D5.1 Evaluation Framework with indicators for impact assessment of the S+T+ARTS Prize projects (2022).

Google Scholar

Diakou, S., De Rosa, S., and Carradore, R. D5.5 Impact assessment of the S+T+ARTS Prize projects (2023).

Google Scholar

IAIA (2009). What is impact assessment? Fargo, ND: International Association for Impact Assessment.

Google Scholar

Klein, J. (2010). “Was ist künstlerische Forschung?,” in Gegenworte 23, Berlin-Brandenburgische Akademie der Wissenschaften, 24–27.

Google Scholar

Moriarty, G. (1997). Taliruni’s Travellers: an arts worker’s view of evaluation, the Social Impact of the Arts. Comedia, Stroud. Working Paper 7.

Google Scholar

Passani, A., and De Rosa, S. D6.3 MediaFutures impact assessment framework (2021).

Google Scholar

Passani, A., Janssen, A. L., and Hoelscher, K. (2020). Impact assessment methodological framework v1. Deliverable of the ACTION project. Available at: https://www.zenodo.org/record/4432132#.YLDJdOvONp8.

Google Scholar

Passani, A., Monacciani, F., Van Der Graaf, S., Spagnoli, F., Bellini, F., Debicki, M., et al. (2014). SEQUOIA: a methodology for the socio-economic impact assessment of Software-as-a-Service and Internet of Services research projects. Res. Eval. 23 (2), 133–149. doi:10.1093/reseval/rvu004

CrossRef Full Text | Google Scholar

Passani, A., Spagnoli, F., Bellini, F., Prampolini, A., and Firus, K. (2016). “Collective awareness platform for sustainability and social innovation (CAPS),” in Organizational innovation and change. Lecture notes in information systems and organisation. Editors C. Rossignoli, M. Gatti, and R. Agrifoglio, 13, 103–114. doi:10.1007/978-3-319-22921-8_9

CrossRef Full Text | Google Scholar

Reeves, M. (2002). Measuring the economic and social impact of the arts: a review. London: Arts Council of England.

Google Scholar

Social Impact Investment Taskforce (SIIT). Measuring impact. Subject paper of the impact measurement working group. (2014).

Google Scholar

Tytgat, R., and Larsen, L. B. (2024). On artistic research. Available at: https://starts.eu/article/detail/on-artistic-research/(Accessed March 28, 2024).

Google Scholar

Williams, D. How the arts measure up: Australian research into the social impact of the arts, the social impact of the arts, working paper 8, comedia, stroud (1997).

Google Scholar