Interpretive analysis of an internet-based course constructed using a new courseware tool called Moodle

Martin Dougiamas
Curtin University of Technology, Perth, Australia
martin@dougiamas.com

Peter C. Taylor
Curtin University of Technology, Perth, Australia
P.Taylor@curtin.edu.au

(This paper was published at the HERDSA 2002 conference.  A PDF version of this paper is available at their site.)

Abstract

In 2001, the authors conducted an Internet-based postgraduate course called “Constructivism” at Curtin University of Technology, for a diverse group of teachers engaged in professional development through distance learning. The defined learning goals for students of the course were: (i) to learn about constructivism; (ii) to reflect critically on their own learning; and (iii) to learn collaboratively by engaging others thoughtfully and empathically. The second author constructed the web site using a new open-source courseware system called Moodle, developed by the first author.

This paper summarises an interpretive study designed to understand and represent the learning experiences of the students and ourselves. Our motivation was to investigate links between the participants’ experiences and the web site. Our intention is not only to improve the quality of this postgraduate course, but also to improve the ability of Moodle as a tool to create online courses that embody and further develop our social constructionist pedagogical framework.

Keywords: Internet, courseware, social constructionism

Introduction

The particular study outlined in this paper is part of an ongoing research program we are conducting at Curtin. In the first author’s case (Martin), it is part of doctoral research supervised by the second author (Peter) that is due to be completed in 2003.

There are many research questions that we are trying to answer, with new ones emerging all the time, but the major question addressed in this paper is: How can internet software successfully support social constructionist epistemologies of teaching and learning? More specifically, what web structures and interfaces encourage or hinder participants to engage in reflective dialogue within a community of learners – by reading openly, reflecting critically and writing constructively? We will explain some of these terms in the next section.

Our aims in answering these questions are, first, to improve our own skills at using the Internet to facilitate distance learning, and, second, to improve the pedagogical skills of other teachers by making our software tools freely available under an Open Source licence.

We chose to tackle these questions within the context of a course (known locally as a “unit”) called “Constructivism” that Peter teaches annually for teachers engaged in professional development through distance learning. The defined learning goals for students of the course were:

  1. to learn about constructivism,
  2. to reflect critically on their own learning, and
  3. to learn collaboratively by engaging others thoughtfully and empathically.

The eight students who enrolled were diverse both in terms of their ages, backgrounds and perspectives, as well as their physical locations (North Queensland, New Zealand, South Australia, Western Australia). Peter constructed the website (in his roles as teacher and co- researcher) using a prototype version of Moodle (Dougiamas, 2001) – a courseware system developed by Martin (in his roles as developer and co-researcher).

Theoretical perspectives

The past few years have seen a marked increase in research around online learning and the use of educational technology. There are now more than 40 academic journals specialising in these topics. After the early years of forays into computer-mediated conferencing and Web-based learning (Amundsen, 1993; Mason & Kaye, 1989), it is becoming clear that pedagogical use of the Internet should be informed and appraised by clear theoretical perspectives.

The most prevalent theoretical perspectives in research on online learning are those related to constructivism, particularly social constructivism and social constructionism. These epistemological positions privilege a focus on collaborative discourse (Amundsen, 1993; Bonk & Cunningham, 1998; Jonassen et al., 1999) and the individual development of meaning through construction and sharing of texts and other social artefacts (Ernest, 1995; Gergen, 1995; Papert, 1991). From these perspectives, learners are apprenticed into “communities of practice” which embody certain beliefs and behaviours (Lave & Wenger, 1991).

The Constructivist On-Line Learning Environment Survey (COLLES) was designed to help assess key questions about the quality of an online learning environment from a social constructivist perspective (Taylor & Maor, 2000). The instrument consists of 24 questions (actual and preferred) arranged into 6 scales:

  1. Relevance – how relevant is online learning to students’ professional practices?
  2. Reflection – does on-line learning stimulate students’ critical reflective thinking?
  3. Interactivity – to what extent do students engage online in rich educative dialogue?
  4. Tutor Support – how well do tutors enable students to participate in online learning?
  5. Peer Support – do fellow students provide sensitive and encouraging support?
  6. Interpretation – do students and tutors make good sense of each other’s communications?

The theory of ‘ways of knowing’, originally from the field of gender research (Belenky et al., 1986) provided us with a survey tool to examine the quality of discourse within a collaborative environment. The Attitudes Towards Thinking and Learning Survey (ATTLS) is an instrument developed by Galotti et al. (1999) to measure the extent to which a person is a ‘connected knower’ (CK) or a ‘separate knower’ (SK). People with higher CK scores tend to find learning more enjoyable, and are often more cooperative, congenial and more willing to build on the ideas of others, while those with higher SK scores tend to take a more critical and argumentative stance to learning. Studies have shown that these two learning styles are independent of each other (Galotti et al., 1999; Galotti et al., 2001). Additionally, they are only a reflection of learning attitudes, not learning capacities or intellectual power.

We find Habermas’ critical theory of communicative action (Habermas, 1984) a useful way to think about discourse in terms of ‘strategic’ or ‘communicative’ actions, and the intersubjectivity of the mutual understanding of intentions. Likewise, his theory of emancipatory knowledge explains how critical self-reflection can lead to a transformation of perspective and realisations of how the horizons of one’s professional ontology (or social reality) are shaped by historical, political and economic contingencies. Thus, our pedagogical intention to enable teachers to develop the skills of transformative professionals capable of appreciating the need to complexify the culture of learning in their own educational institutions so that the interests and aspirations of all students are met.

Interestingly, we have found no other published research that explicitly encourages successful engagement of students in connected online dialogue as defined by these multiple referents. We intend to move forward with this theoretical framework while maintaining a critical self- reflective attitude towards our pedagogical assumptions.

Methodology

Our research in general employs an interpretive research methodology (Denzin & Lincoln, 2000) in which we combine elements of participatory action research (Kemmis & McTaggart, 2000), virtual ethnography (Hine, 2000), and software design (Carter, 1999). In order to optimise credibility and transferability, we use multiple data sources, prolonged engagement and member checks (Guba & Lincoln, 1989). We proceed in a evolutionary manner similar to Cook’s (2001) approach of theorising about dialogical processes, in that we iteratively: (i) apply theory to software design; (ii) put design into practice; (iii) collect and analyse data; and then (iv) use the results to revise our theoretical perspective before embarking on the next study cycle. The results of this approach are evolving theory and evolving software.

While examining and interpreting student case studies in pursuit of answers to our main research question, we focus on elements of the students’ environment that are at least partially within our control.

These include:

  • the web site as a tool for navigating the course,
  • the web site content, activities and resources,
  • the online tutor’s participation and support, and
  • the students’ participation and support.

Of course there are other elements of the students’ environment that are beyond our direct control, nevertheless we recognise these as important to understanding the whole experience of a student. These include:

  • the students’ predispositions (to internet, distance education, authority, constructivism, etc.),
  • the cultures they are part of,
  • the conditions at the place they access the web site,
  • the hardware and operating system they are using,
  • the quality of their connection to the web site (availability, bandwidth), and
  • the context of this course within the student’s overall course of study.

Finally, our research includes critical self-reflection on the use of Moodle as a tool to construct and conduct online courses. We hope this will help us improve its capabilities for stimulating reflective practice in not only students, but also in teachers like ourselves.

Course design

The course was constructed in early 2001, as a result of collaboratively discussing our theoretical perspectives and prior experiences with online learning (Dougiamas & Taylor, 2000; Geelan et al., 2000). We combined software features (Moodle), course configuration and content to develop the following structure:

  • Weekly Structure. The 14-week course layout was based on a weekly structure, and the main course page provided an outline of the course with links to everything. The format was intended to act as an “advance organiser” (Ausubel, 1968).
  • Week 1. The first week was one of introduction, in which the course objectives were presented and discussed, and everyone introduced themselves on a personal page and in a discussion forum. All students filled out an online form in which they could choose to consent to our research. During the course, they wrote entries in their online journals (private from other students) in which they were required to reflect on their current knowledge of constructivism (i.e., the course content).
  • Online Activities. The next seven weeks were laid out in a repeating pattern. Each week students were required to: (i) read one academic paper (most of these had been mailed out to them in paper form before the course); (ii) engage in a collaborative discussion forum about issues in the paper; and (iii) make entries into three reflective journals (before the reading, after the reading, and after the discussion). The journals were assessed each week by the tutor, who provided feedback.
  • Final Essays. The remaining six weeks of the semester were free from collaborative contact to allow students to write essays that discussed the implications of the course content for their own professional and personal learning practices.
  • Other Forums. There were also three forums for open discussion throughout the course. These included a forum for socialising, a forum for technical issues and a forum for discussing teaching and learning issues related to this course (for example changes in the way the course worked).
  • Online Surveys. The COLLES survey was given three times (at the beginning, middle and end of the course), and the ATTLS survey was presented twice (beginning and end). The surveys were intended to obtain indications of any changes that may have occurred in key aspects of the online learning environment during the course.

Emergent findings

The eight students generated a large amount of data during the 14-week course. Apart from the survey data, about 150,000 words were written in the online journals and forums, and about 20,000 log entries were recorded (each entry denoting an “action” taken by a participant).

At the end of the course, our judgement (as teacher-researchers) was that the course seemed to have been quite successful in achieving the three learning goals originally set. This assessment was based on our experiences of teaching the course and monitoring student interactions, as well as on statements made in journal entries, essays and informal (email) feedback. The ATTLS scores we had obtained showed that the students had almost all scored quite highly as “connected knowers” (see Figure 1), and our experiences in the discussion room seemed to confirm this – all students were exhibiting empathy and avoiding adversarial stances. Likewise, a cursory examination of the COLLES results indicated that all students experienced close to an optimal learning environment on all six scales: their preferred levels were usually only marginally above their actual experienced levels (see Figure 2). All students were given quite satisfactory individual assessments.

However, the relatively low COLLES scores indicated for interactivity and peer support raised some questions (see Figure 2). Given that our students scored high as “connected knowers”, and that the collaborative goal of the course was worth 60% of the total assessment, why would so many of the students prefer interactivity and peer support only ‘sometimes’? To answer this question we decided to look closely at the quality of interactions through dialogue, and in particular to study the experiences of particular students.

For our first student case study we chose Carol (a pseudonym), because: her ATTLS scores showed that she thought of herself as a very connected learner (see Figure 1); her texts communicate a highly reflective understanding of her own journey as a constructor of knowledge; and she was a frequent and active user of the course (logging in several times a day).

image1
Figure 1 – A sample ATTLS result generated by Moodle, showing the mean results of student Carol superimposed over the class mean scores (with standard deviation bars).
image2
Figure 2 – A sample COLLES result from Week 7, generated by Moodle, showing the mean results of student Carol (actual and preferred values) superimposed over the class mean scores (with standard deviation bars).

According to the ATTLS research, connected knowing is correlated with “a willingness to ask questions and build on the ideas of others” (Galotti et al., 2001), also known as Socratic questioning. The survey includes statements such as, “I am always interested in knowing why people say and believe the things they do”, with which Carol consistently “strongly agreed”. However, a detailed examination of Carol’s texts revealed no evidence of her directing questions to other students in order to build on their ideas or inquire into their motivations. Looking more closely at the discussion texts, we noticed that there were very few instances of questions being asked by any of the students. Instead, students tended to support a statement in a discussion thread (indicating they had read and reflected, possibly even deeply) and then use this as a base from which to reflect and explain their own thoughts.

We could now see the discussions as a ‘series of loosely-linked reflective monologues’ (i.e., weak dialogue) rather than ‘communicative actions connected by empathic questioning’ (i.e., strong dialogue). In reflecting on our own experiences in various educational forums, we recognised that some of the most powerful learning experiences of our own have come about when we are stimulated by friendly questions about our own practice.

In reflecting on possible reasons for online interactions amongst students constituting mostly weak dialogue, we developed a list of hypotheses to guide our subsequent analyses (in the form of research questions) and the next stage of our course planning.

  • Hypothesis 1: Self-reporting Context. The ATTLS questions are phrased in a general way, and may not have spoken explicitly to the particular online learning context in which students were to engage (for the first time, in most cases). Thus, in responding to the ATTLS, students may have been considering their favoured way of knowing in more familiar contexts, such as on-campus face-to-face classes, rather than in the unfamiliar context of online distance learning.
  • Hypothesis 2: Modelling of Discourse. There might not have been enough explanation and demonstration of the online dialogical behaviours that we were promoting, either from the course outline, the tutor or the students. The tutor (Peter) did post a number of messages in discussion forums that modelled a collaborative, connected style, but perhaps not often enough.
  • Hypothesis 3: Journalling Prevails. Peter’s formative assessments of student journals throughout the course may have reinforced the desirability of lengthy, reflective accounts of learning whose momentum may have carried forward into the discussion forums. For the first part of the course, students were required to complete three journals per week, two before and one after each discussion forum.
  • Hypothesis 4: Personal Distance. A feeling of politeness and perhaps shyness among the small group may have prevailed. Despite the personalised introductions during the first week, students may have felt it was safer to empathise with others and then explain their own thoughts in a non-confrontational way.
  • Hypothesis 5: Learning Predispositions. As mature students probably used to self-directed learning (especially in the highly individualised ‘print-and-post’ mode of distance learning), students may have found themselves slipping into old patterns of learning, especially during extensive journalling.
  • Hypothesis 6: Time Pressure. There might not have been enough time available for students to engage properly in the online discussions, given the amount of private reading and journaling that was required. All students were professionals working full-time and studying part-time.

At this stage of the analysis, Martin interviewed Carol via email, sending her an electronic copy of our case study documentation with questions embedded within it. We asked her to comment on each of these hypotheses. Interestingly, Carol’s answers strongly supported Hypothesis 6 only:

YES,YES,YES!!!!! This is the BIG FACTOR!!!!! Time determines everything. The time to read, think, write and discuss at the end of a day where you have talked and listened and organised constantly is a very big ask. (Carol interview 1, 6 May 2002)

We thought this enthusiastic disclosure strange, given that Carol had spent more time online than most students in the course, and considering that it would probably take her less time to question fellow students than it would to write as much text as she had. In a following phone interview, we questioned her about this:

We were told to address the readings, you see, and by the time we did that, one’s time was taken up …I did find that to clarify it all to myself I needed to think it all through. … You can’t just make a statement without supporting it in some way and teasing it out.

I’d sort of used up my space. I was very conscious that I was the one going on and on and on and other people weren’t. […] I also felt I was being a bit domineering …

I always [write things out in longhand first]. (Carol interview, 12 May 2002)

Carol’s full interview contains supporting evidence for all six hypotheses.

We are still examining data and talking to other students, but already it seems likely that all six hypothesised reasons for weak dialogue prevailing during this online course are viable (with perhaps different combinations of reasons for different students). This conclusion leads us towards our next research cycle and some ways to improve the course environment.

Future directions

In the next version of the course (2002), we intend to reduce the emphasis on individual reflection on content, in favour of increased emphasis on stimulating ‘connected’ discussion.

Specifically, we will lengthen the weekly cycle to fortnightly cycles, to increase flexibility and allow more time for discussion. Each reading (one per fortnight) will now require only one journal (instead of three) and will focus on paraphrasing the key issues, rather than dealing with personal implications, which will be reserved as the main subject of the discussion forums.

We will be experimenting with better modelling of connected behaviour in the online social environment. Not only will the introductory week feature clearer description and some real examples of connected discussion, but there also will be “prompts” throughout the software (at the points of writing) that guide students to think about Socratic questioning and other features of connected dialogue. We will be working harder at practising such behaviour ourselves in the discussion forums.

Lastly, we plan to try a significant and possibly radical change in the structure of the forums. Our forums, like most Internet forums, have been content-based, so that everyone discusses the most recent paper in each forum. In the new course, there will be a forum for each student. After reading the set paper, each student will post a topic starter in their “home forum” that introduces their initial thoughts about the personal implications of the reading. In turn, each student will be interviewed by the other students (and tutors) in order to explore the issues raised by that student’s practice. We hope that this technique will privilege connected modes of discourse while stimulating and engaging students discussing their own practice.

The adventure continues!

References

Amundsen, C. (1993). The evolution of theory in distance education. In D. Keegan (Ed.), Theoretical principles of distance education. London: Routledge.

Ausubel, D. P. (1968). Education psychology: A cognitive view. New York: Holt, Rinehart & Winston.

Belenky, M. F., Clinchy, B. M., Goldberger, N. R., & Tarule, J. M. (1986). Women’s Ways of Knowing: The Development of Self, Voice, and Mind. New York: Basic Books, Inc.

Bonk, C. J., & Cunningham, D. J. (1998). Searching for learner-centered, constructivist, and sociocultural components of collaborative educational learning tools. In C. J. Bonk & K. S. Kim (Eds.), Electronic collaborators: learner-centered technologies for literacy, apprenticeship, and discourse (pp. 25-50). New Jersey: Erlbaum.

Carter, A. (1999). The Programmers Stone. Retrieved 5th May, 2002, from: http://www.melloworld.com/Reciprocality/r0/

Cook, J. (2001). The Role of Dialogue in Computer-Based Learning and Observing Learning: An Evolutionary Approach to Theory. Journal of Interactive Media in Education, 2001(Theory for Learning Technologies). http://www-jime.open.ac.uk/2001/cook/cook-t.html

Denzin, N. K., & Lincoln, Y. S. (2000). The handbook of qualitative research (2nd ed.). Thousand Oaks, Calif.: Sage Publications.

Dougiamas, M. (2001). Moodle: open-source software for producing internet-based courses. http://moodle.com/

Dougiamas, M., & Taylor, P. C. (2000). Improving the effectiveness of tools for Internet-based education. Paper presented at the Teaching and Learning Forum 2000, Curtin University of Technology.http://lsn.curtin.edu.au/tlf/tlf2000/dougiamas.htmlhttp://cleo.murdoch.edu.au/confs/tlf/tlf2000/dougiamas.html

Ernest, P. (1995). The One and the Many. In L. P. Steffe & J. Gale (Eds.), Constructivism in education (pp. 459- 486). Hillsdale, New Jersey: Lawrence Erlbaum.

Galotti, K. M., Clinchy, B. M., Ainsworth, K., Lavin, B., & Mansfield, A. F. (1999). A New Way of Assessing Ways of Knowing: The Attitudes Towards Thinking and Learning Survey (ATTLS). Sex Roles, 40(9/10), 745-766.

Galotti, K. M., Reimer, R. L., & Drebus, D. W. (2001). Ways of knowing as learning styles: Learning MAGIC with a partner. Sex Roles, 44(7/8), 419-436.

Geelan, D., Taylor, P. C., & Dougiamas, M. (2000). Developing distance education students’ skills in critically self-reflective practice using computer-mediated communication. Paper presented at the 1st International We-B Conference, Fremantle, Western Australia.

Gergen, K. J. (1995). Social construction and the educational process. In L. P. Steffe & J. Gale (Eds.), Constructivism in education (pp. 17-39). Hillsdale, New Jersey: Lawrence Erlbaum.

Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation. Thousand Oaks, CA: Sage. Habermas, J. (1984). The Theory of Communicative Action, Vol 1: Reason and Rationalization of Society. Cambridge, England: Polity Press.

Hine, C. (2000). Virtual Ethnography. London: Sage Publications. Jonassen, D. H., Peck, K. L., & Wilson, B. G. (1999). Learning with technology : a constructivist perspective. Upper Saddle River, N.J.: Merrill.

Kemmis, S., & McTaggart, R. (2000). Participatory Action Research. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of Qualitative Research (pp. 567-605). Thousand Oaks, California: Sage Publications.

Lave, J., & Wenger, E. (1991). Situated Learning: Legitimate peripheral participation. New York: Cambridge University Press.http://www.ewenger.com/ewSituatedbook.html

Mason, R., & Kaye, A. (1989). Mindweave: Communication, Computers and Distance Education. Oxford: Pergamon Press.http://icdl.open.ac.uk/literaturestore/mindweave/mindweave.html

Papert, S. (1991). Situating Constructionism (Preface). In I. Harel & S. Papert (Eds.), Constructionism, Research reports and essays (1985-1990) (pp. 1). Norwood, NJ. http://www.papert.com/articles/SituatingConstructionism.html

Taylor, P. C., & Maor, D. (2000). Assessing the efficacy of online teaching with the Constructivist On-Line Learning Environment Survey. Paper presented at the 9th Annual Teaching Learning Forum – Flexible Futures in Tertiary Teaching, Perth: Curtin University of Technology.http://cleo.murdoch.edu.au/confs/tlf/tlf2000/taylor.html

Acknowledgements

We thank all participants of SMEC 706 for taking part in this study.

This paper copyright © 2002 Martin Dougiamas and Peter C. Taylor.

1 thought on “Interpretive analysis of an internet-based course constructed using a new courseware tool called Moodle”

  1. Olá!
    Sou brasileira e utilizei o Inquérito COLLES para avaliar um curso que desenvolvi e apliquei por meio da plataforma Moodle, como parte da minha dissertação de mestrado. Gostaria de saber se esse instrumento é validado para o Brasil.

    Grata.
    Carolina

Leave a Reply

Your email address will not be published. Required fields are marked *