Indicators card game

What is the indicators card game?

The indicators card game is a simple way to inspire researchers to reflect on what they consider to be key indicators for their research and discuss these with colleagues.

Discussions about indicators do not always go smoothly, and there are several reasons for this. One reason is the prevailing view that indicators must always be quantitative, which evokes resistance (‘my research cannot be captured in numbers!’). But such concerns are not necessary – indicators also show that you have met a certain criterion. Demonstrating that can be done in various ways, including qualitative ones. Another reason is that often only a limited number of indicators are brought into play, such as number of publications and grants secured. This is an unnecessary curtailment, as many more indicators can be formulated, such as those involving networking, use of research results and interactions with practitioners. Yet another reason is that indicators are often used for accountability purposes, to ‘sign off’ the research. This makes the discussion unnecessarily charged. By discussing indicators, people can learn from each other what is considered important in research. This will undoubtedly vary from one researcher to another and across research fields, but recognising and expressing that is often the first step towards greater insight and understanding in terms of how researchers see, conduct and position their research. The indicators card game, through its game format, attempts to remove these barriers and facilitate substantive discussion.

How do you use the indicators card game?
Several game forms can be devised to use the card game; that is up to the creativity of the workshop facilitator. Participants could, for instance, be asked to sort the different indicators in order of priority, classify the indicators into ‘must have’ and ‘nice to have’, discuss them based on a selection of indicators or brainstorm about which indicators are still missing, and so on. An interactive format that provides mutual interaction and discussion is the following:

1. Form groups of around 5 participants.
2. For each group, divide the game cards face down among the participants (you will need several sets of game cards for multiple groups).
3. Each participant selects, from the cards received, 2 indicators which the participant finds most relevant. Pass the remaining cards face down to the person sitting to your left.
4. From the cards passed on, choose 1 more indicator you think is relevant.
5. Put the 3 chosen cards (2 from step 3, 1 from step 4) on the table in front of you.
6. Discuss among yourselves the considerations and choices made by each participant.
7. As a group, select up to 10 indicators by the end of the game. In doing so, consider whether the distinctiveness of research in higher professional education is accurately reflected? Are the different aspects of research in higher professional education well represented? Does the discipline/theme of the research come into its own?
8. As a group, you can use 2 ‘wildcards’ if necessary; these are indicators you may not have seen on the cards but believe are important enough to add. (Or add 2 new indicators yourself by using the ‘blank’ cards).
9. Explore whether the chosen indicators can be grouped into specific categories, such as indicators about cooperation or about quality.
10. Have the different groups present the results and discussion points to each other.

The indicators in the card game have not been formulated according to target groups. The aspect of target groups (education, professional practice/society, science) can be included in the following two ways:

• Play the game three times, with the following instructions: Choose indicators that you think best demonstrate the relevance of the research for education/professional practice/science (or any other target groups).
• Add an 11th step to ask whether the different target groups are adequately represented with the chosen indicators, and why this is or is not the case.

What is the origin of the indicators card game?
The indicators card game was developed at Amsterdam University of Applied Sciences (AUAS). The purpose of the game is to make better statements about the impact and continuous effects of research at AUAS. A working group was set up in 2019 to develop a number of tools for this purpose through literature research, working sessions with professors and analyses of critical self-reflections of the CoEs. These tools are underpinned by an analysis of the discussion around impact, continuous effects and the use of indicators (see van Vliet et al., 2020). The tools were then used in preparation for external assessments of the knowledge centres at AUAS in the autumn of 2021. Experiences gained will be incorporated into a future update of the tool.

Variant
A variant of the original indicators game was developed at Hanze University of Applied Sciences Groningen (Hanze UAS) in 2022. Hanze UAS uses so-called ‘Indicards’ to facilitate the conversation about impact with researchers and administrators. This variant distinguishes between the criterion (what do you want to substantiate?) and the indicator (how do you substantiate it?). The criteria are related to the sample criteria from the new Sector Protocol for Quality Assurance in Research (Brancheprotocol Kwaliteitszorg Onderzoek, BKO). The background colour of the cards indicates the area in which the continuous effects take place (green = practice, yellow = education, blue = knowledge development/science). This variant is also available for download.