Using Compendium as a tool to support the design of learning activities1
The Institute of Educational Technology, The Open University, UK
This chapter describes how a mind mapping tool, Compendium, is being used to help designers and teachers create and share learning activities. Initial evaluation of the use of the tool for learning design has been positive; users report that it is easy to use and helps them organise and articulate their learning designs. Importantly the tool also enables them to share and discuss their design strategies. The chapter will ground this work within the wider literature on learning design, focusing in particular on how learning activities can be represented and mechanisms for supporting decision making in creating new learning activities.
Technologies are now beginning to be used in a rich range of ways to support learning; beyond the simple didactic instructional approaches which dominated the early use of technologies in education. In particular social networking tools offer exciting possibilities in terms of supporting more distributed and collaborative learning activities (Alexander, 2006; Downes, 2006). Recent research on students’ experience of using technologies shows that many are comfortable in this technology-enriched environment (Conole et al., 2006; Conole et al. forthcoming; Creanor et al., 2006). ‘Google’, ‘Wikipedia’, ‘Email’, and ‘chat’ emerge as core tools to support students’ learning. They are sophisticated users who appropriate the technologies to their own needs. Coupled with this, current thinking in terms of effective learning, promotes active, engaging learning, where students construct knowledge, building on prior experience, often through collaboration with peers (Dyke et al., 2007). However despite these exciting possibilities examples of truly innovative forms of learning maximising the potential affordances new technologies seem to offer, are still rare. Indeed recent research with practitioners on the creation of learning activities revealed that the most common design strategy was to mirror existing practice rather than exploit the opportunities and affordances of new technologies (Falconer and Conole, 2006, ; Falconer et al., 2007).
We have argued that there is a gap between the potential of technologies to support learning and the reality of how they are actually used and that this is due to a lack of understanding about how technologies can be used to afford specific learning advantages and to a lack of appropriate guidance at the design stage (Conole et al. 2007a). Its cause is due to a range of inter-connected issues: technological (immature tools, lack of interoperability etc.), organisational (barriers and enablers to uptake, cultural barriers) as well as pedagogical issues.
This chapter describes a project which is exploring the design for learning issues within a distance learning institutional context, the UK Open University. The initial focus of the work is reported elsewhere (Conole et al., 2007b), this chapter focuses on how we are using Compendium as a tool for aiding the design process. It will describe the rationale behind the work and initial findings from the evaluation of eight faculty-based workshops run using the software.
Our goal is to build on recent research on learning design to develop a tool that provides support in the course design process with an emphasis on the use of technology-enhanced learning. Users of the system might include individual teachers or course teams, as well as others involved in the design process such as learning technologists or those in our Learning and Teaching Solutions department tasked with helping course teams translate their ideas into technical solutions. The learning design tool will act as a bridge between good pedagogic practice and effective use of new technologies.
Design is a core part of any teaching or training role; i.e. how concepts can be presented to students to enable them to achieve a set of required learning outcomes. Educational text books might give the impression that there is a simple linear basis to the design process; starting with a set of learning outcomes, based on a particular pedagogical approach, appropriate resources, tools and activities are identified and linked together, assessment acting as the ultimate arbitrator in terms of success or failure. However in reality the design process is rarely so simple. In our previous research we observed a series of Geographers over a semester, noting their approaches to design and including any critical decision making points (Fill et al., forthcoming). More recently we have collated forty-four case studies through interviews with teachers across different subject disciplines within the Open University (Wilson et al., 2007). We focused on how they were using technologies in their courses and interrogated them on how they designed the courses and what support mechanisms (if any) they used. Both the Geography studies and the OU studies revealed that the design process is messy. Designers juggle a range of questions, focusing on different aspects of the design process at different points in time: ‘What do I want the students to be able to do having completed this learning activity (a focus on learning outcomes)?’ ‘What tools and resources do I want to incorporate?’ ‘What are the particular characteristics of this group of learners?’ ‘How am I going to assess the activities’? ‘What specific discipline issues or problem does this address?’ ‘How can I design the activity to promote: reflection, collaboration, application?’ Therefore any form of support or tool for the design process needs to be cognisant of this messy, multifaceted and iterative approach.
‘Learning design’ is a methodology that has emerged in recent years as a semi-formal process for support the curriculum design process. The term ‘learning design’ came into common usage with the development of the IMS Learning Design specification, which sought to provide a means of formally representing (and thus reusing) learning sequences. Since then the term has gained a broader usage, and is often synonymous with ‘course design.’ Learning design has seen increased activity in the past few years, as researchers and developers have moved beyond a focus on creation and presentation of content (and hence associated concern with the management of ‘learning objects’) to consideration of learning activities. Beetham and Sharpe (2007) provide a valuable overview of current work in learning design and provide a ‘critical discussion of the issues surrounding the design, sharing and reuse of learning activities, and tools that practitioners can apply to their own concerns and contexts’. Learning design provides a formal methodology for describing learning activities and for formally representing (and hence potentially reusing) learning activities. Crucially it is seen as providing a way of representing learning activities so that they can be shared between tutors and designers and a scaffold to the process of creating new learning activities.
We have identified six main reasons why adopting a learning design approach is beneficial (Conole et al., 2007b):
There are essentially two approaches to the design process: starting from existing practice or through a process of scaffolding the design process through a series of prompts and issues to be considered. Therefore the key research issues are:
The Mod4L project2 identified a range of representations that practitioners use to present practice. These included taxonomies and matrices, visual presentations (flow diagrams, mind maps), case studies, patterns and lesson plans. The project used these with practitioners in a series of workshops to identify their usage and perceived value. They concluded that use is complex and contextualised and that no one presentation is adequate (Falconer et al., 2007).
One of the most popular approaches to abstracting existing practice is in the form of a narrative-based case study. The Joint Information System Committee (JISC) in the UK gathered a range of effective and innovative practice case studies. Each case study was described in terms of the learning outcomes and problem being addressed and was aligned to a particular pedagogical approach (associative, cognitive or situative). In addition to the narrative description case studies included, where appropriate, additional resources such as video clips. The case studies are available as downloadable pdfs.3 A similar exercise was carried out in Australia through the AUTC Learning Design project.4 In addition to the case study narrative, the project developed a specific approach to presenting the core essence of the learning activities being described. In their approach learning activities are broken down into a series of tasks which students undertake, alongside these associated resources and support are illustrated. The project was a large-scale initiative which captured a wide range of learning activities and associated information. In addition to the visual ‘temporal sequences’ for each learning activity there is a rich range of additional information about the design process.
An alternative to the descriptive case study approach is the application of the concepts of patterns derived from Alexander’s work in Architecture (see for example Goodyear 2005). This provides a more structured approach which starts with an intended pedagogical problem being addressed and moves on to provide a potential solution. The patterns approach is built on an underlying philosophy that there are a set of inherent ‘patterns’ which, if identified, can be reused in a multitude of different ways. In addition these patterns combine to form a pattern language (see the Pedagogical Patterns project5 and the EU-funded TELL pattern book (TELL, 2005) for examples).
The alternative to presenting case studies or patterns is to provide some form of guided support or scaffold to the learning design process. A number of toolkits and pedagogical planners have been developed in recent years which adopt different approaches to aiding the design process. The DialogPlus toolkit6 guides users through the process of developing pedagogically informed learning activities (Conole and Fill, 2005). It is underpinned by a pedagogical taxonomy for learning activities (Conole, 2007). This includes a description of the types of tasks students might do as part of the learning activity; assimilative (attending and understanding content), information handling (e.g. gathering and classifying resources or manipulating data), adaptive (use of modelling or simulation software), communicative (dialogic activities, e.g. pair dialogues or group-based discussions), productive (construction of an artefact such as a written essay, new chemical compound or a sculpture) and experiential (practising skills in a particular context or undertaking an investigation). Other examples of support for learning design include the pedagogic planner project7 and the Phoebe project8. Phoebe adopts a similar approach to DialogPlus by attempting to provide a comprehensive online resource of tips and hints to support decision making. However it doesn’t provide any directed guidance, acting more as a set of resources which users can work through. The pedagogic planner instead adopts more of a modelling perspective through mapping tasks to resources and attempting to align the design with specific pedagogical approaches. It is attempting to adopt a user-orientated approach and plans to integrate the tool with LAMS9 a tool for managing and delivering learning activities.
Both from the experience of the Mod4l project and our own work with teachers and designers, it is evident that no one approach meets all needs. Case studies can provide useful ideas, but do not specifically guide users through the decision making process of their own design. Toolkits and planners on the other hand do provide this guidance but can be prescriptive in the approach adopted. With this in mind we decided to adopt a multi-faceted approach; by gathering case studies of good practice and using these as a basis for populating a learning design tool. Our approach was to enable users to be able to use the online tool in as flexible a means as possible, enabling multiple entry points and forms of guidance and support, trying as best as possible to mirror the real, messy process of design we identified by working with practitioners. The next section describes this work and progress to date.
Conole (2007 and forthcoming) argues that practitioners use a wide range of processes and tools (‘mediating artefacts’) to support and guide decision-making in creating learning activities (figure one). These are needed to guide various aspects of learning design: the context of a learning activity, the choice of pedagogy, the creation of associated learner tasks or any combination of these. They range from contextually rich illustrative examples of good practice (case studies, guidelines, narratives, etc.) to more abstract forms of representation that distil out the ‘essences’ of good practice (such as vocabularies or educational models). Each mediating artefact abstracts different aspects of the existing learning activity. Individual mediating artefacts can then be grouped in a variety of different ways for example as a repository of case studies or a set of overarching tips and hints or they can be used as the basis for a more systematic tool such as a toolkit or planner which can then be used to guide the user through the design process.
The OU Learning Design project
The OU is currently undertaking a cross-institutional Learning Design project. We are adopting an iterative methodology focusing on two areas of activity in parallel: a) capturing and representing practice – through user consultation and case studies and b) supporting learning design – through the development of an online tool and associated workshops.
Initial user requirements gathering
The initial phase was carried out as part of a broader programme of work to introduce a MOODLE-based VLE environment.10 During 2006, a series of user consultation exercises were undertaken to gather requirements for a learning design tool specification. These also highlighted a range of perceived barriers and enablers to adopting a learning design approach and to more effective use of technologies to support learning. From this a series of overarching factors emerged; designers and teachers wanted:
A number of possible scenarios for use of a Learning Design tool emerged: by an individual to find examples of how different tools or pedagogical approaches can be used to undertake different tasks, to give them ideas, by a course team as part of the team design process, in discussions between an individual teacher and developer or as the basis for staff development workshops on effective use of the VLE. Following on from the user consultation exercise it was decided that it would be useful to explore some of the emergent issues in more detail and also to gather existing discipline specific examples of how the tools were being used. The focus was on examples which include some form of innovative use of technologies either to support a single learning activity within a course or to provide a scaffold or support across the course in relation to the development of a particular skill or towards a specified set of learning outcomes. The intention is that the tool will act both as a repository of existing learning activities (such as the case studies) and as a design support tool for creating new learning activities.
Forty-four case studies were captured through in-depth interviews with course leaders (table one). The focus was on the pedagogies used to achieve specific learning outcomes and the use of tools (blogs, wikis, e-assessment, etc.) to support learning activities. Interviews were semi-structured around a number of core themes: contextual data (level, subject, etc.), details about the learning activity being described and the sub-tasks involved, pedagogical approaches adopted, and barriers and enablers to the creation of the activity (both technical and organisational). Each interview lasted ca. one hour and was recorded, transcribed, and content checked for accuracy with the interviewee.
The case studies are already highlighting a number of overarching themes (Wilson, 2007). Disciplinary differences are evident – the reasons why and how tools are being used is often aligned with specific discipline needs. For example one case study focuses on the use of an e-Portfolio for a vocational practice-based course where it is a professional requirement to provide evidence of skills development. Some courses are using tools to mimic current practices which are known to be successful, for example a post-graduate course which has created a virtual ‘summer school’. Comparative studies are also proving useful in terms of highlighting the way particular tools are used in different contexts. For example a number of courses are exploring the collaborative potential of wikis but the ways in which they are doing this are tied into the pedagogical needs and context of the course. In the Open University traditionally the main resource load is focused on the production aspects of course development, rather than during presentation (i.e. when courses are being delivered). However the case studies have revealed that this appears to be shifting, as new technologies enable teams to adapt and change course content and activities on a much shorter time frame. Use of technologies is also impacting on assessment methods and the forms of support and communication which are provided.
From our previous work, we were aware that representation of practice is notoriously difficult for a number of reasons. Firstly the degree or level of detail provided – too much is overwhelming, too little not informative enough. Secondly, the degree to which a case study is specific and contextualised. Thirdly the way in which a case study is presented (for example as a textual narrative, diagrammatically or through use of multi-media such as videos or interactive screen shots) has an impact on how much it is valued. We decided to adopt a multifaceted approach to presenting the case studies which would include the following elements: a clear and informative title, a short description of the learning activity and associated salient features, a detailed case study description, visual mapping using notational software and additional views, audio or videos, etc. as appropriate (figure two).
Figure two: Learning Activity template
In choosing a visual representation we adopted a similar column or ‘swim lane’ approach to that used in UML modelling and the AUTC project, but with a central focus on tasks. We choose to distinguish between the different roles and the things associated with each task (tools, resources, etc.) by using different iconic representation. The diagram was built using a mind mapping tool Compendium11 which enables you to provide hyperlinks between different parts of the diagram. It also enabled us to tag icons with appropriate metadata (such as roles, tools, tasks, resources, etc.) and to layer additional information about each element so that when the user hovers over an icon additional information appears. By clicking on an icon the user can either be linked to a specific URL, resource or tool, or to a sequence of layered additional information. Our development of the use of Compendium for learning design is described in more detail in this section.
In addition to gathering the institutional case studies, the other aspect of our learning design project is to develop an online learning design tool. This will be populated with both the information derived from the case studies, as well as selected resources and expertise drawn from our own experience in the field and the wider research literature. As discussed earlier the design process is messy and no one approach is likely to meet the needs of all users. Therefore part of our philosophy in terms of developing a specification for the online learning design tool is that it needs to accommodate a range of different ways in which it might be used. Our initial discussions included the development of a use case scenario of how such a tool might be used. Users could either begin by searching the database of case studies or start from a set of predefined templates. From their preferred starting point users could drag elements onto their workspace and start to build up their learning activity. Elements would relate to the different aspects of the learning activity (such as tools, resources, and roles of those involved). The system would provide adaptive help for each of the elements, for example, if the user has selected a collaborative activity, then tools such as asynchronous conferencing, wikis etc, would be shown, along with additional advice and examples. The user would then build up an activity sequence, adding in further layers of detail as required. We were aware that we needed to iteratively develop the prototype tool with the close involvement of the intended end users, so that we could learn from their use of the prototype and adapt accordingly. We felt such involvement would help us to identify how users might want to use an online tool and what kinds of support and advice they would find useful from the system.
The existing learning design tools discussed earlier (DialogPlus, Pheobe, the pedagogic planner and LAMS) were considered but rejected for a number of reasons. We felt each adopted a particular approach and were therefore not flexible in terms of how they could be used to support the design process. LAMS provided the greatest degree of flexibility but as it operated at the level of a set of pre-defined tool-focused learning activities (voting, discussion, etc.). We felt this straight-jacketed the design process by overemphasising the importance of tools, to the detriment of the other elements involved in creating a learning activity. And by operating at the level of pre-defined tool-activities, we felt it did not enable the user to set their own criteria for the level of granularity of the activity they were designing. We wanted to use a more flexible tool as the basis for our initial prototype. We considered various drawing packages, as well as more specialised mind mapping tools (such as Inspiration and MindManager). In the end we choose to use Compendium, a visual representation tool, originally developed for enabling group argumentation, which was produced by researchers at our own institution. We selected Compendium for a number of reasons. Firstly because it was produced at the Open University, we felt there was more opportunity for further tool development specifically in terms of learning design requirements. Secondly, Compendium supports the creation of a range of visual mapping techniques, including mind maps, concept maps, web maps and argumentation maps (Okada and Buckingham Shum, forthcoming), which we felt offered the potential for a range of flexible approaches to the design process. Compendium comes with a predefined set of icons (question, answer, map, list, pros, cons, reference, notes, decision, and argumentation). The creation of a map is simple, users drag icons across and can start to build up relationships between these through connecting arrows. Each icon can have an associated name attached with more details contained inside the node, an asterisk appears next to the icon and if the user hovers their mouse over this the content inside the node is revealed. Other types of electronic files can also be easily incorporated into the map such as diagrams, Word files or PowerPoint presentations. The reference node enables you to link directly to external websites. Icons can also be meta-tagged using either a pre-defined set of key words or through user generated terms. Maps can be exported in a variety of ways from simple diagrammatic jpeg files through to inter-linked websites.
Our initial task was to agree a common format for representing learning activities visually. To begin with we worked only with the existing pre-defined icon set. Figure four shows an early example which attempted to visually represent a learning activity created by Oliver12 in Australia as part of the AUTC learning design project. The figure shows a series of columns representing the key elements involved in the learning activity, which consisted of a simplified version of a learning activity taxonomy developed as part of the DialogPlus project (see Conole, 2007 and Conole, forthcoming for more details on this). The diagram shows the use of a number of the icon sets (notes, reference, list) and the inclusion of a series of external file types (a picture and word files). The note icons ‘Authors’ and ‘Discipline’ have asterisks by them showing that they contain additional information which is displayed when the mouse hovers over the icon.
The next stage in the process was to work with an individual academic on one of their own learning activities to try and elicit both their thought processes in the design process and their initial reaction to using the Compendium tool. A third-level environmental course, U316, was chosen for this purpose. This course was chosen for two main reasons. Firstly it was recognised to be an example of good practice and innovative use of technologies. Secondly, it had been the subject of an extensive research and evaluation project funded by the Mellon foundation and so a lot of detailed research data had been gathered on different aspects of the course and the design process (Thorpe and Godwin, 2006). Stewart Nixon, the main lead for the VLE-related learning design work, and I worked with the main researcher involved in evaluating U316 to represent the learning activity, we noted her reaction to interacting with the Compendium tool as well as her general thought processes in representing the key learning activities contained in the course. A number of interesting issues emerged in the discussion. Overall her reaction to the tool was positive, she felt that it helped her articulate and share the key aspects of the learning activity. The ability to provide layered aspects to the information represented was also deemed useful. What constituted an appropriate level of granuality of information was also considered and it was agreed that a pragmatic and contextual approach should be adopted. Interestingly she also felt it would be useful to include indications of time to complete against each task, a factor which we had not considered in our early design prototyping, which further support our decision to adopt a user-centric and iterative approach to our prototyping and design of the tool. In describing her initial impressions of the tool, Thorpe notes the benefits of this approach as:
Learning designs can be explained, as here, using narrative accounts but these are often not at a level of detail sufficient to enable a practitioner to capture the key elements in their own teaching. More detailed narratives also benefit from diagrammatic representations that teachers can use to clarify the activities involved. (Thorpe et al., 2007)
Armed with this initial positive feedback about the potential use of Compendium as a learning design tool we decided to create a dedicated set of learning design icons, to complement the generic set available within the tool. As part of the core functionality of the tool it is possible for users to create and incorporate their own ‘stencils’ of icon sets. Once the appropriate set of icons have been identified, they are labelled with appropriate text and given an overarching stencil name set. We choose to focus on a simplified list of icons to represent what we felt were the key aspects of the design process, namely: task, role, tool, resource, output, group, assignment, and activity. All of the icons are of the same type except for the activity icon which is a variant of the generic map icon. As with the core Compendium icon set users are able to rename each of the icons to something more appropriate to their context. Once created the stencil set is opened via the tool drop-down menu. Figure four provides a screenshot of Compendium, showing the generic set of icons on the far left-hand side, along with the learning design stencil ‘LD2’ we created.
We used the new stencil set as a means of representing the learning activities being described in the case studies. As we began to represent this and based on feedback from users we realised that our initial iconic representation (shown in figure four) was overly complex and so we fixed on a simplified approach which consisted of a column for each role (student, tutor, etc) and an associated column for the ‘assets’ associated with that role (i.e. any resources, tools or outputs).
Figure five represents a screen shot of part of the learning activity associated with the U316 course. Two roles are shown (student and tutor) along with their respective tasks. Tools, resources and outputs (i.e. assets) associated with each task are shown alongside, with arrows indicating connections.
Our initial evaluation of the use of the tool to represent learning activities in a format similar to that shown in figure five proved positive and seemed to go some way towards addressing the first of our areas of research focus, i.e. how to capture and represent practice. The second aspect was to provide some form of intelligent scaffolding for the design process, in the form of guidance or additional support. As discussed earlier we were aware that no one approach to design would meet all users needs and hence the scaffolding needed to be adaptable and multi-faceted. Our ultimate goal is to provide adaptive and contextulised information on different aspects of the design process, tailored to individual needs and delivered on a just-in-time basis.
As a first step to this, our review of related tools and planners identified a number of different approaches that helped the user think creativity about different aspects of the design process. For example, both the DialogPlus toolkit and the Pedagogic planner offered mechanisms for the designer to map learning outcomes, tasks and assessment. The Phoebe planner provided some useful tips on thinking about particular tools and which types of activities they might support. However feedback from users also showed that they valued having a simple step-by-step set of guiding questions to think about and guide them through the design process. The JISC effective practice with e-learning includes one example in the form of a learning design template. Beetham and Sharpe’s (2007) recent book on learning design includes a series of Appendices which provide similar guidance. We wanted to experiment with using these different means of supporting the design process by creating a set of adaptable templates that users could work through and adapt to their own context. In addition to the creation of iconic stencil sets, Compendium also enables the user to create customisable templates. A template is a Compendium xml export file, which holds a set of maps/nodes which the user might use frequently. We used this template facility to create a series of learning design templates focusing on a core set of different approaches to the design process:
Figure six provides a screen shot showing the LD template set on the side, along with the open ‘Step-by-step’ template.
Finally we are beginning to draw together a comprehensive set of resources related to the learning design process. These we have collated in Compendium and exported to create a web-based version. The resources cover the following areas of support:
Figure seven: Learning design tools, resources, guidance and templates
Figure nine: Compendium maps of learning activities categorized by discipline
During March and April 2007 these resources were trialled through a series of workshops. The first consisted of a group of critical friends made up of e-learning researchers and educational developers. The second was a workshop with 17 Engineers at the University of Porto in Portugal. As part of the workshop participants created designs using the DialogPlus, Phoebe and Pedagogic Planner tools described in this chapter, and one group using our first customised prototype version of the Compendium tool (which incorporated the specialised LD icon set). Feedback from the group confirmed that they did want some form of structured guidance to the design process, that they valued case study examples, particularly from their own subject area and that they valued the opportunity to articulate their design ideas with other colleagues. Encouragingly the group using the Compendium tool seemed to have the most positive experience and got furthest in terms of representing their learning activity:
[Compendium is] Very good to visually describe the activity itself and the actors, resources, etc. but it’s not a planning tool in a sense it just describes the activity and it does not give you a framework. [University of Porto Workshop participants, 2nd-3rd April 2007]
They found the tool helpful in terms of developing a shared language and discussing and noting design decision making points. Based on this feedback the online resource described above was more extensively developed and restructured in terms of how the information was provided to the users. Our ultimate intention is to ‘mix and match’ these extensive resources so they appear at appropriate decision points in the users design process.
During April 2007 eight faculty-based OU workshops were run using the improved learning design focused Compendium tool and associated resources. The workshops included an introduction to the concept of learning design and a series of exercises getting participants to reflect on their current strategies for design. The second part included a hands-on session where users worked in groups to present their own learning activities in Compendium. Figures ten and eleven provide two different examples of designs that were produced in the sessions. What is interesting is the way in which the participants adapted the column-based role and asset structure we presented (for example see figure five) to suit their own needs; importantly the flexibility of compendium as a tool enabled them to do this and did not stifle their creativity. We were surprised at how far the participants got in representing their designs and it did seem during the sessions that Compendium acted as a useful tool to help them articulate and share their thought processes. A few participants however commented that they did not find representing their designs visually helpful, stating that, for them, pencil and paper/discussion would be preferable. It is likely that such a focus on the visual aspects of the design process will not suit everyone, but overall most participants were positive both during the session and in their evaluation feedback.
Figure eleven: A learning design emphasising reflection
Participants were asked to complete an evaluation at the end of the session. This was then used as the basis for a wrap up discussion highlighting what they found useful about the session and what they would like to see improved, along with an action list of ‘next steps’ for their respective facilities. The questionnaire asked them a series of open-ended questions such as: what topics they would have included/excluded and why, and what they most liked/disliked about the workshop. More general questions about the length of the session, quality of the presentations and materials, and suggested follow up were also asked.
Feedback from the workshops has been very positive with all groups reporting that they liked Compendium, found it easy to use and a useful tool to help them not only think about and articulate their design process, but also as a means of representing and sharing their design. There were mixed views on the balance of theoretical and the practical hands-on aspects of the workshops:
Session would have been improved by getting into Compendium straight away and having less of the front end stuff. We need to grapple with the tools that will help us organise learning design rather than the 'background to' LD.
I think the session held together as a whole with a good balance of input/discussion/activity. I feel like I have a handle on the basics of using a tool, access to a range of resourses [sic] and has been thoroughly linked to my own practice -- so I wouldn't want to change any of the content.
Similarly there were different views on how the material could have been ordered and the level of detail given to each sub-topic. Despite some negative comments about the amount of theory, as the workshop was introducing a new methodology and way of thinking about learning design we still feel that this theoretical underpinning is important and would want to include it in future workshops, however we will adapt this material and put more emphasis on the benefits to end users of adopting a learning design approach. Inherent in some of the negative feedback received is the fundamental problem that the concepts of learning design on the surface appear very simple but are in fact very complex. A classic example is the frequent call for examples or case studies, as users assume having access to these will be enough to give them ideas to create new designs.
Some more examples of good practice and their Compendium design
Perhaps some more concrete examples – helps me relate the words to something concrete
However, in our experience giving users a set of case studies can be overwhelming; they often don’t know how to work through them or apply them to their own context. In fact as illustrated earlier a comprehensive set of resources (including links to several large databases of external case studies) was provided as part of the workshop material, but evidence to date suggests that few users are prepared to invest the time needed to work through these to extract the necessary information they require. Indeed this uncovers a further issue, that despite the overall positive evaluations for the workshops very few of the participants are continuing to use the materials as a basis for creating designs. This suggests that further work is needed to make use of the system intuitive alongside faculty-specific support.
In previous work a learning activity taxonomy was developed which identified the components associated with a learning activity (See Conole, 2007). The level of detail of the taxonomy (which includes for example 72 possible types of learning task) illustrates the complexity of the design process. However, the evidence from our previous work (Fill et al., forthcoming; Falconer and Conole, 2006; Jeffery et al., 2006) suggests that users are impatient and want/expect quick solutions and are not prepared to invest the time necessary to create learning activities which take account of all the different factors involved. This is the key challenge for research in learning design: how to provide simple and easy to use guidance and tools to support the design process, which users are prepared to use and can see the benefits to them of investing time in using these tools, which at the same time don’t trivialise the process.
The project is timely as the OU is involved in two major initiatives on the use of technologies; the VLE programme described earlier and the OpenLearn project13 which is making OU content freely available. It is clear that there is a need for further research – practitioners are crying out for examples of good practice and guidance in design. However previous research shows that representing learning design practice and providing appropriate support for learning designers is both difficult and contested. By bringing together both narrative accounts of learning designs with notational maps showing the design visually, we hope to address and find practical ways of approaching the key issues in this area: How will users interact with the case studies and the learning design tool? Will practitioners find the tool useful? How will the tool be used in different contexts? What associated support mechanisms might be useful – such as individual expertise or interactive workshops?
Our initial evaluations of work to date is encouraging, Compendium seems to provide an easy to use visual tool to help represent different learning designs. The next stage in our work will be to try and structure the information emerging from our institutional case studies along with the wider set of resources on thinking about the different aspects of the learning design process into an adapted and contextulised set of scaffolds to guide users through the design process. If we can achieve this, we believe we will go some way towards addressing the problem outlined at the beginning of this chapter, namely the mismatch between the potential of new technologies in terms of how they can be used to create innovative and engaging learning activities and their actual use in practice.
The work described in this chapter is part of an institutional project on learning design. Others involved include: Stewart Nixon, Peter Wilson, Martin Weller, Simon Cross and Mary Thorpe.
1 Word count: 8078
3 Effective practice with e-learning - http://www.elearning.ac.uk/effprac/ and case studies of innovation - http://www.elearning.ac.uk/innoprac/