Portal de Periódicos da CAPES

Reflecting on factors influencing long‐lasting organisational effects of group model‐building interventions

2022; Wiley; Volume: 38; Issue: 2 Linguagem: Inglês

10.1002/sdr.1705

1099-1727

Hugo Herrera, Nuno Videira, Hubert Korzilius, Kathya Cordova-Pozo, M.H.F. McCardle-Keurentjes,

Information Systems Theories and Implementation

Researchers and practitioners recognise that many organisations, to different extents, deal with nonroutine, complex problems that hinder their performance and constrain them from fully delivering their mission. These situations are often characterised by lack of agreement about the scope and depth of the problem at hand, as well as conflicting views and agendas regarding the desired course of action (Eden and Ackermann, 2001; Franco and Montibeller, 2010; Howick and Ackermann, 2011; Huz, 1999; Mingers and Rosenhead, 2004). Decision support approaches like Group Model Building (GMB) have been widely used to aid on solving these complex problems (Franco and Montibeller, 2010). The usefulness of GMB, and other facilitated modelling approaches, in complex and ambiguous situations resides in using a transitional object to improve problem understanding and communication among the parties involved (Rouwette et al., 2002). This transitional object is a representation of a problem built by stakeholders with help of a facilitator that can be used to update and align stakeholders' mental models (Black and Andersen, 2012). In GMB interventions, the transitional object is either a diagram or a simulation model that is mostly built and discussed during participatory workshops involving those with a stake in the problem. Many field studies have shown the advantages of GMB interventions (e.g. Rouwette and Vennix, 2006; Videira et al., 2012). So far, researchers have mainly focused on assessing immediate effects at the individual (e.g. mental model refinement) and the group level (e.g. consensus and commitment) (Rouwette et al., 2002; Scott et al., 2016). The assessment of long-lasting effects of GMB interventions, and participatory modelling methods in general, on organisations has remained largely unexplored (Howick and Ackermann, 2011; Luoma, 2016; Schilling et al., 2007). This gap in the literature was recently highlighted by Randers (2019) while reflecting on the 60 years of the system dynamics field. In his reflection, Randers pointed out that, while there has been a lot of progress, the issue of how to achieve organisational change through modelling insights remain largely puzzling (Randers, 2019). Ultimately, system dynamics interventions, such as GMB, add greater value if they influence behaviour and enhance organisational performance (Luoma, 2016). It thus seems a matter of good practice to reflect on assumptions underlying design and process choices and to examine the long-lasting organisational effects of GMB interventions (cf. Ormerod and Ulrich, 2013). In this Notes and Insights essay, we reflect on how system dynamics interventions, and GMB in particular, could support organisational change by looking at the long-lasting organisational effects of boundary objects created in GMB interventions. In this regard, Black (2013) offered us a useful framework to reflect about why the objects we use and the processes that we follow are not always effective in developing shared understandings and “actionable,” practical guidelines. We built on this work by deliberating about how we can manage boundary objects produced during the GMB process. In 2014, we conducted an experiment involving two client organisations based in Lisbon, Portugal. The first (Organisation 1) is a small private enterprise focused on hospitality services to international students. The second (Organisation 2) is a medium-sized public body responsible for managing domestic waste across the country. The objective of the experiment was to compare real-world differences between two variations of a GMB script. In each of the two organisations, we worked with two comparable groups addressing the same problem. This was a rare opportunity since, like Rouwette et al. pointed out, “It is unlikely (…) that a management team confronted with a real-life problem would voluntarily split up in two or more (comparable) groups which then each use a specific method to work on the problem” (2011, p. 788). Developing such kind of experimental conditions in a real-life context allowed us to limit the number of independent variables to consider in the analysis and to make relatively objective comparisons. Figure 1 offers an overview of the process followed in the interventions. It is worth noting that for ‘Workshop 1: structure elicitation’, we used different scripts for each of the two groups in the same client organisation. In each case, there was a group following a ‘traditional’ GMB approach using causal loop diagramming (CLDs) as the main elicitation technique and another group using Strategic Options Development Analysis (SODA) with the same purpose as proposed by Ackerman et al. (2011). The participants in the experiment (a) did not have previous experience working with facilitated modelling approaches; (b) had adequate language skills to understand and participate in the workshops conducted in English; and (c) were not aware which specific strategic issue would be analysed before the experiment started to avoid negotiations between participants before the workshops. As in other case studies described in the literature (e.g. Huz et al., 1997; McCardle-Keurentjes et al., 2018 l Rouwette et al., 2009; Rouwette et al., 2011; Scott et al., 2013), our assessment focused mainly on evaluating the process and outputs of the workshops. Namely, we evaluated quality of communication, perception of consensus, and commitment. The communication among participants was assessed by analysing the conversations happening during the workshops. Next, the perception of consensus regarding potential solutions and participants commitment to take these solutions forward was measured through questionnaires (before and after each workshop). Finally, the overall perception on the usefulness of the intervention was captured through interviews with a sample of participants (eight out of 17) at the end of the intervention. We recognise that the controlled environment set up for an experiment bounds the insights that can be gained regarding ownership of the model and power conflict between stakeholders because the impact of political agendas is to some extent limited by such environment. However, the same environment allowed us, to the extent possible, to separate the effect of the method on the final outcomes from the effects of other factors – for instance, by controlling for group composition. Recognising that the long-lasting effectiveness of the interventions remains largely unexplored, we contacted the case-study participants 5 years after the workshops with the aim of assessing what happened after our engagement. Overall, six participants (out of the 17 that originally participated in the intervention), including the two gatekeepers, agreed to participate in this recent round of interviews (see Table 1). The follow-up interviews were conducted through video calls in two sessions. The first session focused on eliciting the outcomes of the intervention in terms of solutions implemented and behavioural changes seen in the organisation. Namely, we asked participants about three concrete potential impacts of the intervention: (a) implementation of solutions proposed during the workshop; (b) changes in behaviour following the workshops; and (c) lessons learned from the workshops. The second round of interviews was designed after revising the answers from the first round of interviews and aimed at understanding what elements in the process could have increased the impact of the intervention. We did this by directly asking participants what they thought that could have helped to increase the workshops' impact. Moreover, we asked them to reflect on their expectations and understanding of what the workshop were supposed to achieve. In the next section, we present the highlights from these interviews to later reflect on the factors that may influence long-lasting effectiveness of GMB interventions (e.g. the occurrence or absence of long-term organisational effects). During the first round of interviews, participants stated that the solutions proposed in the workshop were not implemented (see Table 2). Two participants did not have explicit recollection of the solutions proposed, with one of them not recalling at all the solutions that were discussed during the intervention (see answers in Table 2). Those that remembered that solutions were proposed pointed out that suggested strategies to solve the problem were not taken forward because there was not a concrete action plan from those responsible for implementing the strategies after the workshops (see 1G and 2G responses in Table 2). Likewise, when asked if they saw changes in the way meetings were conducted in the organisation, participants also indicated that there were not visible changes. However, all interviewed participants recognised having incorporated some elements of the GMB workshops, for example, using diagrams to facilitate the discussions in other meetings, mainly for planning and collaboration (see answer from 1G in Table 2). They also provided some hints on the justifications for such effects, by pointing out the need of having a trained facilitator in order to be able to incorporate some of the scripts used in the workshops in their regular meetings. Finally, when asked about organisational learning outcomes, participants indicated that their organisations did not learn from the experience (see responses in Table 3). This is particularly interesting when we compare their latest answers to those they provided immediately after the intervention. Immediately after the intervention 12 out of 18 participants agreed or strongly agreed that they “learn more about the issues surrounding the initial question of the workshop.” You mean, like practical things? Mmmm….probably an introductory talk to system dynamics would have been helpful. I think some in the team were not sure what the workshop was about. It probably would have been good to spend additional time talking about ‘so what’. Like what are we going to do know with the things we have learned and the model you prepared. Maybe two workshops were not enough. As I say before, probably spend more time discussing the model. I thought it was good but I did not understand it properly so I could not use it later. I think it would have been good to get a report about the workshop, you know with the diagrams and the summary of the model… (“I think I sent you a report”) Ohh… maybe you did…I do not remember. Maybe then just having…. like a wrap up session to discuss the report and how we could used the model and diagrams later. During the second round of interviews participants provided their views about things that could have helped us to increase the impact of the GMB intervention. The suggestions proposed by participants included: hosting more workshops, agreeing on concrete actions and next steps during the workshops, and having some prior training on the modelling and diagramming methods. The latter suggestion also surfaced several times when we asked participants how they would have seen the workshops making a difference in their organisation. In terms of expectations, most participants remembered they were interested in learning about system dynamics and GMB, with some interviewees stating they did not have any expectations for the intervention. Learning was also mentioned when asked about what they wanted to get out of the workshop. When answering this question some participants also mentioned more concrete outcomes, such as eliciting new ideas, next steps for tackling the problem at hand, and a simulation model. There are many examples in the literature valuating the impact of GMB interventions based on participants feedback immediately or a short term after it took place. Like many of these cases, we also got positive feedback and concluded that the GMB interventions have been, to some degree, successful in improving communication, reaching consensus, and raising commitment among participants (see Herrera et al., 2016). These results contrast with the feedback provided 5 years after the GMB sessions. For the evaluation of our initial intervention, we need to refer to our original study done by Herrera (2014). In our case, participants mostly agreed that the workshops helped them to learn or gain insights about the situation presented. For example, 11 out of the 17 participants agreed that the workshop helped them to change their mind about the problem at hand, 12 out of the 17 agreed or strongly agreed they learned more about the issues discussed, and 14 out of the 17 agreed the workshop challenged their initial mental models. All of these are often seen as signs of mental model refinement and considered as a positive sign of the outcome of GMB interventions. Even more important, 14 out of 17 agreed or strongly agree that they felt committed with the implementation of the solution proposed during the workshop. When we interviewed them a couple of days after the workshop, six out of the 10 participants interviewed agreed they thought the solution proposed during the workshop was the best alternative to tackle the issue discussed, and five out of 10 were willing to present the proposed solution to the decision makers in their organisation. What happened after the intervention that diminished its practical impact? What could we have done differently? We framed our analysis on the work of Black (2013) and Black and Andersen (2012) regarding using visual representations as boundary objects. Many authors have proposed that the outcomes of GMB and other facilitated modelling methods derive from using diagrams or models as boundary objects that facilitate the cognitive process that enables learning and facilitates consensus. According to Black, “boundary objects are visual representations that synchronously span a boundary of expertise or objectives among multiple individuals” (2013, p. 80). As discussed by Black (2013), GMB workshops provide the conditions for using system dynamics models and diagrams as boundary objects by facilitating effective multidirectional communication (Rouwette et al., 2011) among multidisciplinary groups (see Figure 2). In our case study, multidisciplinary teams used diagrams and simulation models to diagnose the problem, propose solutions, and prompt those in the workshops to act on the proposed solution. At the end of the intervention, participants' responses to the postquestionnaires suggested that we created boundary objects that improved communication and helped to reach consensus. However, Black (2013) proposes that an important and often neglected characteristic of boundary objects is iterate-ability or the extent to which the objects remain easily transformable by the users. We hypothesise that this characteristic might be particularly important to understand long-lasting effects of GMB interventions. If boundary objects are not easily transformable after the workshops to be used and updated by the organisation, they stop their functionality for the long-lasting effectiveness to the organisation. If this happens, the potential boundary object becomes a unidirectional object that will not have any effect in the long-term. From the case study presented above, as well as other experiences in which we have collaborated, we reckon that in the majority of GMB interventions there was no one in the organisation who was able to use the diagrams and models produced to engage with other stakeholders that were not part of the workshops. Although we often communicate our results through reports and even share the model with the participants, these representations are not easy to transform and may regress into unidirectional forms of communication. In fact, during the interviews, participants indicated that this lack of iterate-ability was indeed a problem. For instance, participants revealed that they would have needed further training on the method and would have benefitted understanding the model in more detail: “We probably could have done more with the model. You know like really understand it and how did it work” (1G Table 3). Using our experience and participants' feedback, we are proposing a framework for investigating when and why completed boundary objects created during GMB interventions transform into unidirectional ones. For simplicity, we hypothesise that this transition happens at two points in time (see Figure 3). In the short-term, e.g. a couple of weeks after the intervention, this may happen when participants realise that, although they grasped the model during the GMB session, they are not comfortable enough to explain the model to other stakeholders that were not present at the workshop. Moreover, participants realise that even if they share the model with others, they are not capable of changing it to incorporate missing elements or to test hypotheses provided by these new stakeholders. Therefore, models and diagrams quickly stop being used by the organisation and the impact of the intervention fades away. Furthermore, even if participants are comfortable enough to use the model and communicate its results, these objects still risk becoming unidirectional objects if the process for transforming the object is not institutionalised and, for example, newcomers are not trained and involved in continued use of the boundary object initially produced. Institutionalisation of the process is important because organisations are not static. As the time passes, new staff joins the team backfilling for those who left, priorities change, and constraints emerge and disappear. This is particularly important for interventions addressing long-term problems as the completed boundary objects, the solutions proposed, and the insights gained will need to be constantly updated before fully implementing a solution. In practice, institutionalisation of the process requires the facilitating team members think about what will happen after the workshops have been delivered. Based on the feedback we received, there are at least two elements that need to be considered while planning the workshops. First, the team needs to support the organisation to move from forecasting and testing proposals into ‘action planning’. This need for thinking on implementation has been already identified by Größler (2007) when reviewing system dynamics projects that failed to have an impact. In his reflections Größler (2007, p. 447) concluded that missing this step was, as in our case study, “the reason why many good potential solutions were never implemented.” Second, the facilitating team also needs to plan for developing organisational capabilities that will enable stakeholders in the organisation to fully understand the objects created (diagrams and simulation models) so they can update and modify them (Hovmand, 2014). Building organisational capability has been an aim in system dynamics projects since Senge (1990). However, this step in the intervention process is often neglected due to time and resource constraints (both among organisations and facilitating teams). As shown in the feedback we got from participants in our intervention (see Table 3), without an explicit effort form the facilitating team, participants might not feel comfortable explaining the model to others not to say making changes to it. Considering the effort and time we often dedicate to creating boundary objects and its widely accepted importance towards achieving mental model refinement, consensus, and commitment, it is paramount to understand how we can manage the completed boundary objects so that they remain transformable by and accessible to everyone in the organisation. Using Figure 3 as a framework, we propose a road map with eight milestones towards achieving long-lasting impact that happen along the way of the intervention. Milestone 1 – Setting the stage for success: The facilitation team needs to understand and manage the expectations that the client organisation has for the interventions. Individual agendas need to be accounted for, including the agenda of the gatekeeper(s). The facilitation team should carefully select the stakeholders who will participate using the goal of the intervention as reference point. Scripts available in ´Scriptapedia´ (Hovmand et al., 2011), like ‘Creating a Shared Vision of Modelling Project’ might be a useful starting point in this process. Milestone 2 – Link the intervention to a wider purpose: One-off facilitated modelling interventions are unlikely to have high impact unless they are part of the organisation's ongoing strategic management processes. Milestone 3 – Self-awareness: While planning the intervention, it is important to make a critical assessment of the process bias the facilitator brings to the discussion. For instance, in our case study, the facilitator was focused on keeping a similar pace in all the groups so that the results between groups could be comparable. However, this meant that the time spent discussing the diagrams and the model itself was insufficient for some participants. Milestone 4 – Plan beyond the workshops: The targeted impact of the intervention needs to be clear, and the planning should account for medium and long-term activities to achieve, and for risks to mitigate, from the very onset of the process. While it is likely that facilitating the team will move on after the workshops have been completed, it is important that the team considers what would happen after – for instance, who will own the model going forward? What actions will need to be taken after the workshop? Milestone 5 – Introduce the stakeholders to a new language: The method should be made accessible (easy to understand) to all the participants (Hovmand, 2014). In some cases, it might be useful to introduce/train the group on system dynamics modelling/diagramming methods during the planning stage, as well as providing training on model use afterwards. This is paramount not only to improve understanding and participation during the workshops, but, maybe even more, to keep the transitional object accessible to the client organisation. This does not mean that stakeholders can necessarily build a similar diagram on their own, but at least some of them should understand it to the extent they can explain it and make small adjustments. Milestone 6 – Keep an eye on the process: A member of the facilitation team, for example, the process coach (see Richardson and Andersen, 1995), could help the facilitator to adjust the process depending on the observed group dynamics. This is particularly relevant when there is a tension between formal and informal procedures of GMB defended by a majority of participants. This can be seen as the natural follow up to Milestone 2. Milestone 7 – Hand over of the model: Insights gained during the workshop(s), especially those of dynamic nature, should be captured and shared with the group if they are to endure. Preparing a brief or a short report after the workshop(s), a workbook (Vennix, 1996), or having a ‘wrap-up’ session could be good ways to increase the chance that long-lasting impact will be realised. Handing over a simulation model might be more difficult than handing over a diagram, and the facilitation team needs to spend enough time making sure the model remains understandable and accessible for the organisation to keep using and adapting it. Milestone 8 – Develop an implementation and follow-up plan: Regardless the purpose of the intervention, follow-up activities should be included as part of the modelling programme (see Größler, 2007; Videira et al., 2017). These activities might include, for example, helping the group to draw concrete plans or protocols for the solutions proposed, using the simulation model, and/or interviewing participants. In Table 4 we complement these milestones with some suggestions for practitioners to consider as part of good practice. While further research and reflection will test whether any of these milestones could substantially improve long-lasting organisational effects of GMB interventions, in this note and insights essay we provide food for thought to both researchers and practitioners who are aiming to advance our field. We see our work as a contribution to an ongoing debate aiming to inspire our community to plan and review their projects with a different lens. The authors would like to thank and acknowledge the collaboration of Dr. Marleen H. F. McCardle-Keurentjes in the conceptualization of the ideas elaborated in this essay, as well as for the collaboration in the broader research process and case studies described in the article. We also appreciate the effort and support of the two organisations participating in the case studies and the follow-up interviewees. The original project was carried out by the first author as part of the European Master in System Dynamics programme hosted by Bergen University, Universidade Nova de Lisboa, Radboud University, and Universidad de Palermo. CENSE is supported by the Portuguese Foundation for Science and Technology (FCT) through the strategic project UIDB/04085/2020. The authors declare that they have no conflict of interest. Hugo Herrera is Erasmus Master in System Dynamics alumnus and has a joint PhD in model-based public planning, policy design and management from the University of Bergen and University of Palermo. Hugo is Postdoctoral Research Fellow at the Department of Geography, University of Bergen. His goal is to help create sustainable and resilient systems by supporting the decision process using stakeholders' involvement, computer simulation models and a wide range of analytical tools. He has spent the last years working on a broad scope of projects including management of food systems, waste management, energy transition and ecosystem service management. Nuno Videira is Associate Professor at NOVA School of Science and Technology, NOVA University Lisbon, Portugal, and researcher at CENSE, Center for Environmental and Sustainability Research. His research work focuses on participatory system dynamics modelling approaches for stakeholder engagement in environmental and sustainability decisions, with applications to human-wildlife coexistence, ecosystem services, water resource management, sustainable consumption and production, degrowth pathways and sustainability transitions topics. Dr. Hubert P. L. M. Korzilius is associate professor of Research Methodology at the Institute for Management Research of Radboud University, Nijmegen, the Netherlands. His current research interests are: research methodology, method development, cross-cultural management, dynamic decision making, system dynamics and group mental models. He has published extensively, among which books, book chapters and papers in scientific journals, such as European Journal of Operational Research, Food Policy, Frontiers in Psychology, International Journal of Intercultural Relations, Social Networks, System Dynamics Review, and Systems Research and Behavioral Science. Kathya Lorena Cordova-Pozo is Master in International Politics and economics and PhD. in Economics (Université Grenoble Alpes, France). She is associate professor of Research Methodology at the Institute for Management Research of Radboud University, Nijmegen, the Netherlands. Kathya is permanently involved in collaborative research projects related with health economics in Latin America. In the last 13+ years has been a guest researcher in several countries in Europe and South America and a guest lecturer in Bolivia. Her focus of expertise is statistics, economic analysis in sexual and reproductive health for adolescents and nutrition and lifestyle for children and adolescents in Latin America. Marleen H. F. McCardle-Keurentjes was assistant professor of Research and Intervention Methodology at the Institute for Management Research at Radboud University, The Netherlands. She retired in May 2021. Her research interests included information sharing in groups and effects of facilitated modelling on decision quality, mental model alignment, and commitment to decisions.