Pedagogy
“Pedagogy is the discipline that deals with the theory and practice of education; it thus concerns the study of how best to teach.”1
Stackable Regiments welcomes submissions of pedagogical research to academy@stackableregiments.com, which will be attributed to the provider and their institution (unless they wish to remain anonymous). We reserve the right to not include the submission.
1 https://en.wikipedia.org/wiki/Pedagogy.
Content
Learning in Groups
“The message is clear: What students learn is greatly influenced by how they learn, and many students learn best through active, collaborative, small-group work inside and outside the classroom” (p 22)1.
While the value of promoting learning in groups is widely recognized, the issues of effectively managing group learning may not be generally appreciated and extend beyond the universal concern of how to fairly and accurately assess group work and the individual contribution. Johnson and Johnson in various works explore key aspects of cooperative learning and these are summarized in “What is Cooperative learning?” where the following point is made: “Not all groups are cooperative (Johnson & Johnson, 20092). Placing people in the same room, seating them together, telling them they are a group, does not mean they will cooperate effectively. To be cooperative, to reach the full potential of the group, five essential elements need to be carefully structured into the situation: positive interdependence, individual and group accountability, promotive interaction, appropriate use of social skills, and group processing (Johnson & Johnson, 19893, 20054).”5
There are a number of issues associated with cooperative learning, that include (1) the benefits of cooperative learning, (2) how to structure group projects in order to maximise the benefits (as above), (3) the optimal size of groups, (4) the problem of how to assign students to groups, and (5) how to fairly assess the individual contribution to the assessment of group work. The latter two issues are particularly contentious.
The first relates to the benefits of cooperative learning compared to intergroup competition, individual competition or individual student tasks. Organizing students into cooperative learning groups has positive effects compared to not grouping students at all. There are beneficial effects on learning irrespective of whether the groups compete with one another or not, and this is true for students of all ability levels6. We need to embrace cooperative learning and move away from a predominant lecture mode. MeTL supports this change in the teaching dynamic and moving towards making it easier, more productive and fairer is a logical progression by using groups. We should “not hang on to the past by trying to make do with slight modifications of the status quo when faced with a need to change paradigms. Paradigms have to change, and sometimes they have to change quickly” (p.1:3)7
The second issue is complex and a very readable analysis of lecturing compared to active cooperative learning is provided by Johnson, Johnson and Smith7. If a young lecturer was to read only one book on teaching, especially if they are predisposed to lecturing, they could do worse than read this one. The works cited collectively highlight significant deficiencies in much group teaching and learning where, too often, students are placed in groups, simplistically expected to cooperate, and the outcome is assessed by a summative group project. Rarely are the projects structured to develop in students the skills necessary to work cooperatively, even though this is a highly sought after attribute by employers.
One problem has been the difficulty of monitoring group work in order to be able to provide contextual feedback in time for the group members to learn how to work effectively in a group environment. By working in a system like MeTL, where the teacher can observe group activity directly by visiting the individual group work pages periodically during the project, and by monitoring group analytics, the progress of group work and the characteristics of the contributors become more transparent to the teacher. The teacher can intervene remotely at any time and experience indicates that students welcome and respond to such feedback. An important finding is that “results confirm the importance of instructors actively intervening within groups to develop trust, promote participation, and create task interdependencies”8. The ability to intervene, especially in group work performed outside the class, is heavily dependent on the teacher being aware. Encouraging group work to be done in a medium where the teacher can observe and have access to participation analytics, such as MeTL Groups, has the potential to provide the teacher with a valuable window into group work.
The third issue concerns the optimal size of groups. It may appear obvious that groups should be kept small, but how small? When comparing group sizes of pairs, 3-4 and 5-7 it was concluded that groups of 3-4 seem to be more effective6. The reported study might be criticized for not being more granular when 5 is often advocated. A study on analysing ideal meeting sizes to achieve a management purpose, rather than a teaching class, suggest 5-7 is more effective, with the recognition that having an odd number has advantages in avoiding deadlocks9.
The fourth issue relates to the way students are placed in groups, all of which have different consequences. Conventionally groups are formed by self-selection, teacher-selection or random-selection, with stratified randomization seen as a variant7 of the latter. These seem straight forward but are not. For example since friends tend to sit together in classes, if the teacher groups by neighbouring students, this may have similar outcomes to students self-selecting. Randomly assigning students to groups may not be random and can be a little involved if a random number generator is used which can deter the use of groups.
The teacher can group students together by similar abilities or other demographic attributes (homogeneous grouping). Conversely the teacher could group by dissimilar attributes (heterogeneous grouping). Grouping students homogeneously or heterogeneously has advantages and disadvantages6. (The 2nd Edition of this work10 omits the research findings reported in the first edition). Homogeneous grouping on ability has a positive effect on achievement when compared to no grouping. However, students of low ability perform worse when placed in homogeneous groups with other students of low ability as opposed to students of low ability placed in heterogeneous groups. There is a small positive effect of homogeneous grouping on high-ability students. Medium ability students benefit most from homogeneous grouping on ability compared to heterogeneous grouping6.
Some authors contend there are ethical issues in placing high-ability students in heterogeneous groups as their achievement will be adversely affected. Conversely ability grouping has been criticized because of the implications for minority groups the members of which may disproportionally originate from disadvantaged backgrounds11.
The fifth factor is the vexing problem of how to assess group projects, particularly for the individual contribution to the group grade if that is appropriate. When individual contributions to group work are not assessed it may have implications for feedback and individual skill development. Assessment is more complex than assigning a grade12 and assessing comprehensively involves measuring knowledge, skill acquisition, retention and above all understanding. However, assessing individual contributions to group work is difficult and usually requires some form of student peer assessment13 and self-evaluation. A significant part of the problem is the inadequate information available to the assessor, who in many cases only has the final outcome to judge. If the final outcome, the product, and the process are important then somehow the process needs to be assessed14. Advantages of process assessment by the instructor and peers include the construction of work logs but reviewing work logs is time consuming and students may misinterpret traits15, as may the teacher with insufficient information. MeTL can provide data on student participation and other metrics are in development that will help validate peer assessment. Automating analysis of contributors and the quality of contributed content, has the potential of mitigating the time needed to produce and monitor work logs.
An important consideration is the scope and longevity of group projects which have been characterised as Informal, Formal and Base5. Informal cooperative learning projects last from a few minutes to one class and can, among other things, focus student attention, facilitate cognitive processing through rehearsal, and enhance summarizing skills. There is a distinct purpose in such a group activity. Teachers should use the grouping method that only includes students actually present, since there is no point in grouping absent students and by attempting to do so some groups may become suboptimal. Formal cooperative learning groups and cooperative base groups both have the requirement of group work that are longer term and therefore students who are absent when the group was established should be assigned so that they have an effective group when they are able to join the class. In this case teachers should use the class list of enrolled students to form the groups.
At this stage there is no definitive research supported protocol that indicates the best way to establish groups. Self-selection is the least recommended procedure7. Groups of friends tend to be less productive than groups of strangers as they may be more distracted. The consequences of self-selection or grouping by students sitting together should be carefully considered.
Random-selection is the easiest and most effective way to assign students7, especially if there is some application wizard that applies a random selection automatically. Stratified random selection is essentially a form of teacher-selection in that the teacher must make some fundamental decisions before applying the appropriate selection algorithm.
Teacher-selection is possibly the most widely used but the consequences of using either homogeneous or heterogeneous groups require the teacher to be well informed of the implications. MeTL provides the teacher with various automated methods of grouping that allows homogeneous or heterogeneous grouping strategies that are designed to help the teacher manage the competing interests of ease of making groups, fairness, and purpose. Teacher-selection, even when transparent, runs the risk of bias, unless the teacher consistently follows some rules and is equally well informed about every student in the class. MeTL Smart-Groups gives the teacher some options to select some rules and then MeTL will assign a student, which reduces the risk of teacher bias and saves time. If there is insufficient information for a selection rule to be applied that option will not be made available and MeTL will randomly assign students to groups, which is a safeguard.
This brief discussion of groups and cooperative learning suggests there are no easy solutions to this complex, but potentially powerful, paradigm. However, hopefully, it does alert the teacher to some of the problems associated with groups and provides some methods that the teacher can utilise to virtually automate the task and at the same time be as fair and productive as possible.
The teacher can always override the groups suggested by MeTL, or groups imported from the LMS, if MeTL is integrated with an LMS. However, students often have concerns about the grouping strategy the teacher employs and there are arguments for making grouping transparent and equitable, so the manual override options should be used carefully. If students are concerned about grouping strategies employed by the teacher they could be pointed to this discussion so they understand the complexity of the problem.
1 Springer L, Stanne ME & Donovan SS (1999) Effects of Small-Group Learning on Undergraduates in Science, Mathematics, Engineering, and Technology: A Meta-Analysis. Review of Educational Research, 69(1),21-51.
2 Johnson DW & Johnson F (2009) Joining together: Group theory and group skills (10th Ed.) Boston, Allyn and Bacon.
3 Johnson DW & Johnson F (1989) Cooperation and competition: Theory and research. Edina, MN, Interaction Book Company.
4 Johnson DW & Johnson F (2005) New developments in Social Interdependence Theory. Genetic, Social, and General Psychological Monographs 131(4), 285-358.
5 http://www.co-operation.org/what-is-cooperative-learning
6 Marzano RJ, Pickering DJ & Pollock JE (2001) Classroom Instruction that works: Research-Based Strategies for Increasing Student Achievement. Association for Supervision and Curriculum Development. Alexandria, Virginia, USA.
7 Johnson DW, Johnson RT & Smith KA (1998) Active Learning: Cooperation in the College Classroom. Edina MN, Interaction Book Company.
8 Hilton S & Phillips F 2010 Instructor-Assigned and Student-Selected Groups: A View from the Inside. Issues in Accounting Education 25(1),15-33.
9 Romano NC & Nunamaker JF (2001) Meeting Analysis: Findings from Research and Practice. Proceedings of the 34th Hawaii International Conference on System Sciences.
10 Dean CB, Hubbell ER, Pitler H & Stone Bj (2012) Classroom Instruction that works: Research-Based Strategies for Increasing Student Achievement. Association for Supervision and Curriculum Development. 2nd Edition Alexandria, Virginia, USA.
11 Adodo SO & Agbayewa JO (2011) Effect of homogenous and heterogeneous ability grouping class teaching on student’s interest, attitude and achievement in integrated science. International Journal of Psychology and Counselling 3(3), 48-54.
12 Johnson DW & Johnson R (1996) Meaningful and manageable assessment through cooperative learning. Edina MN, Interaction Book Company.
13 Fellenz R (2006) Toward fairness in assessing student groupwork: a protocol or peer evaluation of individual contributions. Journal of Education Management 30(4),570-591.
14 http://www.cmu.edu/teaching/assessment/howto/assesslearning/groupWork.html
15 https://uwaterloo.ca/centre-for-teaching-excellence/teaching-resources/teaching-tips/developing-assignments/group-work/methods-assessing-group-work
Self-Selection
Self-selected versus instructor-selected groups are often compared in research papers, but instructor selection can encompass a wide variety of methods from random, stratified random, homogeneous or heterogeneous groups.
The main advantages of self-selecting groups are the ease of administration; effectively let the students sort it out. In general students, at least the vocal ones, prefer it. The main disadvantages are that it is hard for students who do not know anyone else, it might further marginalize students who are already stigmatized in some way, and it is often seen as not being fair for all students1.
Self-selection, where students form groups with minimal influence from the instructor, is often favoured by students because they know each other and will be able to work together productively. However, in the workforce employees are rarely able to choose their collaborators2.
One study, exploring student characterization of group formation effects, showed that self-selected students generally had a more positive group experience. Self-selected groups initially performed better than instructor-selected groups, but this effect disappeared over time. The self-selected groups perceived that they did better in their group although final grades indicated that group formation method did not affect group achievement (as measured by group project grades)3.
However, project grades are not the only attribute that needs to be fostered. “An important achievement by individuals in instructor-formed involved learning to develop trust in others with whom they had no prior contact” and that development of trust might be viewed as “a superior outcome to the comparatively less challenging experience … that … existed within student-selected groups” (p 31)3. Although the authors found that student-selected groups had a more positive experience they “resisted the temptation to conclude that student-selection is the superior method for forming groups” (p 31)2, which is consistent with perceived benefits of heterogeneous groups.
There are potential difficulties with the group composition in experiments where students were allowed to self-select if they wished and have the teacher select the remaining students who chose not to self-select2. This was acknowledge as possibly affecting the results. However, there is another consideration with student-selected groups which is rarely discussed, which relates to the potential disadvantage of self-selection on students who might already be marginalized. On balance Self-Selection has to be considered very carefully before adoption. A common practice of instructor-selection by grouping students based on their proximity to each other is in effect a kind of student self-selection, and instructors need to be aware of the consequences of students sitting alone or in some form of stratification in the class. For example less motivated or engaged students might tend to gravitate towards the back of the class.
Harvard Law School advises faculty to thoughtfully construct groups. “With diversity in mind, teachers should assemble groups according to a common interest. If you allow students to self-select, it is important to help them build diverse teams”4. Further “Weimer has found that students who don’t know each other before they do group work often create the most outstanding work, whereas student who already know each other tend to socialize more and make less progress during group work, thereby not producing exceptional results” (reference not attributed)4.
MeTL does not support self-selection.
SLU users can generate self-selected groups in D2L Brightspace and MeTL will import those groups when generating a group page.
1 http://www.uts.edu.au/node/1161/forming-effective-groups
2 http://www.facultyfocus.com/articles/teaching-and-learning/group-work-are-student-selected-groups-more-effective
3 Hilton S & Phillips F 2010 Instructor-Assigned and Student-Selected Groups: A View from the Inside. Issues in Accounting Education 25(1),15-33.
4 https://blogs.harvard.edu/hlscasestudies/2015/07/14/keys-to-facilitating-successful-student-group-work
Teacher-Selection
If students are not allowed to select (Self-Selection), the onus of forming the groups normally falls on the teacher. In MeTL the teacher will be able to avoid making selective decisions by Random-Selection of students, or in the future by using the suggestions offered by SmartGroups. Currently SmartGroups requires the teacher to make some high-level decisions and then SmartGroups will assign accordingly, making it easier and faster for the teacher and reducing the risk of bias through inadequate information.
Without using random or smart_group selection, teacher-selection is not easy and requires considerable thought and experience. Increasingly teachers are selectively assigning students to groups based on criteria, which can be shared, homogeneous grouping, or balanced and spread among the groups, heterogeneous grouping. Apart from Random-Selection, all the methods available to the teacher are contentious for various reasons. However, not often considered is the problem of whether the teacher has enough information to apply selection criteria unless those criteria are based on exogenous factors such as socio-economic status, gender or race. Using gender or race in particular is highly problematic and if prejudicial may be illegal. Socio-economic status is known to have effects on education outcomes 1 in that educational attainment is positively correlated across generations, so using this as a criterion must also be carefully considered.
One obvious issue with any teacher-selection is that often groups are established early in the experience of the teacher with the class. This means that the teacher is very unlikely to have an informed base on which to make decisions, and those teachers willing to make selections should be aware of the Dunning-Kruger effect2.
The most common Teacher-Selection method is one based on ability, yet ability should be affected by the influence of the teacher during their exposure to the students, and therefore should be labile if the teacher has any influence. Ability has to be constantly nurtured and monitored and is multifaceted. Using only grades as a measure of ability is questionable on several counts, and makes Teacher-Selection difficult. In spite of these concerns there is a large literature (see Learning in Groups for a brief discussion) about the advantages and disadvantages of grouping based on similar or different attributes, often ability alone. The point here is not so much about the relative validity of those arguments and which to accede to, but rather, having made a choice, is the teacher sufficiently informed to make such groups? The solution requires analysis of significant amounts of data that is likely to be beyond the reach or capacity of any teacher. MeTL is attempting to address this issue with the construction of Smart-Selection, which is predicated on the assumption that the teacher might, for good reason, wish to choose some variant of attribute selection, based on a choice of rules the teacher should understand before using. If there is insufficient information for Smart-Selection then students will be assigned randomly.
1 https://www.americanprogress.org/events/2012/01/12/17181/the-rise-and-consequences-of-inequality/
2 Kruger J & Dunning D 1999 Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments Journal of Personality and Social Psychology Vol 77 pp 1121-1134.
Random-Selection
Randomness is complicated and not easy to achieve. For example the call-off system “is when the lecturer walks around the room and assigns each student in the class a number in a systematic call off (ie. 1,2,3,4,5 … 1,2,3,4,5 … etc. or A,B,C … etc.). Groups are then formed by putting all the 1’s, 2’s etc together. Other random appointment methods include students drawing numbers from a “hat” … or the lecturer drawing them out”1.
The first method is not random, it is systematic, and the group size, seating position and path the lecturer takes may affect the outcome. Drawing numbers from a hat is likely to be more random but takes time.
A potential advantage of a “random” method is the claim that there is little preparation needed, but this is not necessarily true as discussed above. The potential to break up friendship groups, to allow or even require people to work with others they normally would not, which is often seen as fairer, are seen as advantages1.
Students who prefer to work with friends or at least students of similar values and abilities tend not to like random-selection. There is the risk of incompatible groups being formed, which raises the important question of teaching students how to manage cooperative skills, including conflict1. A quick review of Gartner interview questions in various scenarios indicates the emphasis on establishing whether the potential employee can handle group situations. The first question in the group of difficult Gartner interview questions is “Tell me about a time where you had to deal with conflict on the job”2. Avoiding constructing incompatible groups is avoiding teaching students how to manage such challenges.
The observation that teachers who randomly assign students to groups are lazy and uninformed is, in itself, an uninformed criticism. Arguably randomly assigning students to groups, particularly when there is a variable amount of information available to the teacher, is always the fairest way. This is likely to be particularly true when groups are assigned early in a semester before the teacher knows enough to fairly assign students to groups. The risk of ad-hoc Teacher-Selection based on inadequate knowledge is high and may result in hidden or even unconscious bias and prejudice. In general a frank and open discussion with the class of how best to assign students to groups is worth considering.
SLU users can generate random groups in D2L Brightspace and MeTL will import those groups.
1 http://www.uts.edu.au/node/1161/forming-effective-groups
2 http://top-interview-questions.com/Gartner/interview+questions
Smart-Selection
Ideally there would be a perfect way to organize students into groups. However, research does not provide a clear answer to how to do that, at least at this time, and there may not be one solution for all occasions.
Grouping students by their similar attributes (homogeneous grouping) or distributing similar students across groups (heterogeneous grouping) are choices the teacher must make (see Learning in Groups). If there is no obvious reason to select one over the other then it might be safer to use a Random-Selection procedure.
MeTL’s Smart-Selection engine (SmartGroups) can suggest and characterize sets of grouped students to the teacher. It is likely that these suggestions will be more robust for students in more senior years, when the student’s strengths and weaknesses are more apparent. It is possible that in the senior years students will be grouped according to inferred high level attributes. For junior students, where less information is available, random selection is not only safer but possibly the fairest and most productive choice.
SLU users can generate random groups in D2L Brightspace and MeTL can import those groups.
SmartGroups is structured so that teachers will always have some overriding controls, to the point that students can be moved between groups by drag-and-drop at the time of group creation, or any time after using the Group Editor.
A common attribute for grouping is by ability, which is usually identified as the grade the students have achieved to date. MeTL intentionally does not provide the option of grouping by grade. Assessing a student’s ability is the focus of a vast literature, which might include conceptual understanding, capacity to summarize a process, apply a theory, solve a problem, work collaboratively, etc.
Effective characterization of ability is likely to be a composite of all of these kinds of factors. The ultimate goal of SmartGroups is to employ artificial intelligence to extract the best group allocations so that the teacher does not have to make potentially arbitrary decisions.
In its simplest form SmartGroups presents three grouping criteria, which can be applied homogenously or heterogeneously. These are activity level, class enrolment congruence, and response level. These have been deliberately chosen as attributes indicative of process rather than product, and can be derived from very small datasets, bridging the space between the beginning of a new semester and post multiple assessments, when traditional grade based recommendation engines begin to apply.