Evaluating the Impact of Europlanet 2024 RI
Project Evaluator, Jennifer DeWitt, and Communications Manager, Anita Heward, report on outcomes of the first review of Europlanet’s flagship research infrastructure.
Read article in the fully formatted PDF of the Europlanet Magazine.
On the cusp of the pandemic, over two days in February 2020, 69 members of the planetary community met in the historic town of Windsor, UK, for the kick-off of the Europlanet 2024 Research Infrastructure (RI) project. Largely unaware of the global disruption and challenges that were to come, we discussed our plans for the project and the outcomes that we hoped we would achieve.
An important part of the conversation was how we could measure success. Europlanet 2024 RI is the third in a series of projects to provide access to research facilities and field sites, virtual services, tools and networking, in support of the planetary community. To sustain these programmes beyond July 2024, when the current €10 million grant from the European Commission’s Horizon 2020 programme finishes, we will need to be able to show the impact that we have achieved to date, demonstrate that we have a wide-reaching user-base, and make a solid case for future funding.
Building on the consultations in Windsor, we put in place an evaluation framework through which we could examine the extent to which the project’s various activities reach their intended goals, and understand what causes barriers to their success. Our framework is aligned with the Organisation for Economic Co-operation and Development (OECD) strategic objectives for assessing research infrastructures, which include a range of core indicators for scientific, education, training and societal impacts. The framework also incorporates categories of information requested by the European Commission’s reporting portal.
The project management and activity coordination teams identified metrics and indicators of impact, such as publications and conference presentations, new collaborations, numbers of users or participants in workshops and training sessions, and media coverage. We also defined the user groups that we wanted to engage, such as researchers, industry, policy makers, early careers and under-represented communities.
Through the Europlanet website and online tools, we set up a series of forms and databases to track this information on a regular basis, and designed feedback surveys for users and visitors to facilities to provide input on their experiences. To help understand the experience of TA visits in more depth, we included questions in the feedback forms that ask respondents if they would be willing to be contacted for a short interview to assist with evaluation.
On 4 November 2021, the European Commission held its first interim review of the Europlanet 2024 RI project and we presented the progress made to date. Many of the activities planned in Windsor have been delayed or implemented in a radically different way. However, despite all the challenges and restrictions brought by the pandemic on a personal, professional and societal level, the project evaluation shows some early signs of significant impact.
Out of 171 Transnational Access (TA) applications received to visit laboratories and field sites, 117 have been approved for funding and 18 teams of researchers have visited facilities and submitted their reports. All seven of the field trips and four of the lab visits involved people travelling to the sites, with the remainder taking place as virtual visits (including one hybrid of in-person and virtual participation). While the virtual trips have been innovations in response to travel restrictions from the pandemic, feedback from both in-person and online participants has reflected very positive experiences.
Evaluation from the visitors highlights that the trips are extremely productive scientifically. Three quarters of visitors are considering new scientific activities that they would not have envisaged without Europlanet support. Many are planning future collaborations with the host facility, and are also planning outreach activities connected with their visit. While delays due to the pandemic mean that TA visits have only started relatively recently, nearly 30 publications are expected from the visits that have taken place to date. One conference paper has been presented so far, with nearly 40 anticipated in the coming years.
Several of the TA facilities will be upgraded or extended through Europlanet 2024 RI funding, and all works will be complete by the end of 2022, in time for the next full call for TA applications. The new Ice Chamber for Astrophysics-Astrochemistry (ICA) at the ATOMKI Institute for Nuclear Research in Debrecen, Hungary, has already hosted four TA visits, and has been featured in a review paper on sulfur ice astrochemistry. Collaborations that originated in TA visits have also led to research beyond planetary science, e.g. art conservators have used facilities at the Vrije Universiteit Amsterdam to identify the geological origins of pigments in Vermeer’s Girl with a Pearl Earring. Wider society has been engaged with the TAs through social media and press coverage of visits and emerging results, and through a set of teaching resources linking the planetary field sites to core topics in schools’ curricula.
Overall, the visits are enabling impacts, particularly on a scientific basis, that would simply not be possible without access to the Europlanet 2024 RI’s facilities and expertise. Going forward, the inclusion of additional facilities in China and Korea in the TA programme promises impact (scientific and otherwise) at an even broader global scale.
Examples of impact are also emerging from Europlanet 2024 RI’s suite of four Virtual Access (VA) services. The VESPA virtual observatory for planetary data has been in development since 2015.
At the start of the Europlanet 2024 RI project, 52 planetary and heliophysical data services were installed in VESPA from 18 different institutes worldwide. The European Space Agency (ESA) has implemented EPN-TAP, a protocol to describe and access data, on the Planetary Science Archive (PSA), which now includes over 20 million files from 30 years of space missions in Europe.
Since February 2020, VESPA has built closer links with space agencies, through representation in the International Planetary Data Alliance (IPDA), participation in the NASA Planetary Data Ecosystem independent review board, and through additional projects with ESA and NASA. The VESPA management team has also been closely involved in international consortia such as the International Virtual Observatory Alliance (IVOA) in astronomy, the Research Data Alliance, and bodies related to the European Open Science Cloud (EOSC).
Although planned workshops to bring new data services into VESPA are only now taking place in late 2021, many of the existing services have been updated and upgraded.
Also building on heritage from the previous Europlanet 2020 RI project, SPIDER provides novel services to model planetary environments and solar wind interactions through a dedicated run-on-request infrastructure and associated databases. Since February 2020, services have been prototyped and launched to support studies of Mercury’s exosphere, as well as plasma instrument background counts at Venus and Mars. SPIDER publications relating to simulations of Mercury’s magnetosphere, and were used by the community for the ESA/JAXA Bepicolombo mission’s flybys of Venus and Mercury. Opportunities identified through SPIDER for synergistic observations by BepiColombo during its cruise phase with the Solar Orbiter and Solar Parker Probe missions have been implemented by ESA and JAXA.
Two new virtual access services have been introduced for Europlanet 2024 RI. GMAP is building infrastructure for Europe in geological mapping of planetary bodies, a critical requirement to support future robotic and human exploration. A virtual Planetary Mapping Winter School in February 2021 was highly successful, attracting 200 participants from around the world, and was a major first step in starting to build a user-community of planetary geological mappers. A follow-up school is planned in February 2022.
Europlanet 2024 RI also aims to foster wider use of Machine Learning (ML) technologies in planetary research, and to develop open-source ML tools for seven science cases. A portal has been developed to provide access to services, and codes and scripts are now available through a GitHub. To start building the user community, two completed ML pipelines were presented at EPSC2021, and 100 participants attended a workshop on how ML can support fireball observations. A further workshop is planned in early 2022.
Restrictions on travel and face-to-face contact have meant that Europlanet 2024 RI’s networking activities have mainly taken place virtually. Lessons learned from the 2020-2021 experiences show that online meetings can bring in additional speakers and participants from industry, as well as people with caring responsibilities, or from under-represented groups that have limitations on travel, time or financial support.
However, in-depth and one-on-one conversations are less likely to take place in a purely virtual environment, and this has, to some extent, limited the impact of networking and opportunities for building new collaborations.
Nonetheless, virtual training sessions focused on supporting early careers and new users in under-represented states have been over-subscribed and generated positive feedback.
A hybrid workshop on supporting the development of a planetary community in Africa has taken place successfully, implementing the first step in Europlanet’s Global Collaboration and Development Strategy, published in December 2020. Europe’s main annual gathering of planetary scientists, the Europlanet Science Congress (EPSC), held virtually in 2020 and 2021, has developed a successful and robust model for holding online conferences using a combination of asynchronous scientific presentations and discussion supported by a programme of live events. Multiple presentations and splinter sessions relating to Europlanet 2024 RI were held at the EPSC virtual meetings, and high-level sessions were attended by Members of the European Parliament, representatives of space agencies and the European Commission, and (importantly) industry.
Fostering links between industry and academia is a priority for Europlanet 2024 RI and has proved particularly challenging without opportunities for face-to-face networking. However, the Europlanet Society’s ten Regional Hubs have supported development of a database of industry with an interest in planetary science, and this has laid solid foundations for industry engagement as restrictions begin to ease.
The wider societal pressures of the pandemic, from financial restrictions to mental health, mean that early career researchers are currently in particular need of support. The bursaries, training schools, internships and mentoring services provided by Europlanet 2024 RI are more important than ever in alerting early careers to opportunities, and helping them develop skills and build their network of contacts. A mentorship programme, launched in August 2020, now involves 28 mentor-mentee pairs and has already resulted in a paper and two mentees finding new positions.
Europlanet 2024 RI’s other main innovation, a network of 16 small-scale telescopes in Europe and beyond, was established in June 2020. Through an open call, amateur and professional astronomers can apply for time to carry out observational campaigns, either by visiting the telescope or (for 14 facilities) through remote access. Eight projects have taken place to date, with two thirds led by women from under-represented countries.
All data collected using the Europlanet Telescope Network will be open access and two scientific articles resulting from the network have already been published. An observational alert system has also been implemented for planetary targets requiring a rapid response, enabling amateur astronomers to contribute scientifically valuable data. The alerts, which are followed by over 200 active observers, have resulted in publications, including on storm activity in Saturn’s north polar region and in Jupiter’s south temperate belt. More widely, collaborations stimulated through Europlanet 2024 RI, and involving its beneficiaries, have led to a number of successful proposals for new projects funded through the European Commission and national agencies. From the development of ML tools to mobilising researchers in Africa, these external projects act as multipliers for the impact of Europlanet 2024 RI in many different communities.
Overall, Europlanet 2024 RI is largely on track to meet its expected impacts. The world has changed since we met in Windsor; the pandemic has led to innovations that support widening participation and reduced carbon footprints, but has also highlighted areas where face-to-face contact remains important. As we approach the halfway point of the project, with more time elapsing for case studies to emerge, we look forward to developing a more robust understanding of these impacts, and using lessons learned to build a sustainable research infrastructure for planetary science.
Europlanet 2024 RI has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 871149.
