The purpose of this activity is to practice analyzing experiment results by applying the t test. The data you will analyze will be simulated data, randomly generated by a web service.
For this activity, you will need to use:
The data generator requires two inputs: an experiment and an experimental design. For this activity, we will use the Point-and-click experiment, which simulates the time it takes for a user to point and click a particular target on screen using up to three different pointing devices (a mouse, a trackpad, and a trackball). The other input, the experiment design, is a string specifying the conditions and trials made by each user in the experiment.
The output of the data generator is tab-separated-value data, which you can copy and paste into a spreadsheet to rearrange and compute statistics, and then transfer to the t test calculator to perform the statistical test.
Generate data from the Point-and-Click experiment using the experiment design MMMM,MMMM,MMMM,PPPP,PPPP,PPPP. (How many users and trials is this?)
Move the data to your spreadsheet. It's a good idea to split the time column into two side-by-side columns, one for the mouse condition (M) and one for the trackpad condition (P).
Use the spreadsheet to compute the mean (called AVERAGE in most spreadsheets), standard deviation (STDEV), and standard error (STDEV/sqrt(n)) of each condition. If you graphed the two means with error bars, would the error bars overlap?
Run a t test on this data at the 5% significance level. Determine the p value, the value of the t statistic, and the degrees of freedom.
Generate data from the Point-and-Click experiment using the experiment design MMPP,MMPP,MMPP,MMPP. (What's the risk of this experiment design?)
Move the data to your spreadsheet. Line up each user's M and P trials side-by-side.
Run a paired t test on this data at the 5% significance level. Determine the p value, the value of the t statistic, and the degrees of freedom.
What are best practices for designing group projects?
What is true for individual assignments holds true for group assignments: it is important to clearly articulate your objectives, explicitly define the task, clarify your expectations, model high-quality work, and communicate performance criteria.
But group work has complexities above and beyond individual work. To ensure a positive outcome, try some of these effective practices (adapted from Johnson, Johnson & Smith, 1991) or come talk to us at the Eberly Center.
While some instructors don’t mind if students divvy up tasks and work separately, others expect a higher degree of collaboration. If collaboration is your goal, structure the project so that students are dependent on one another. Here are a few ways to create interdependence:
|Ensure projects are sufficiently complex that students must draw on one another’s knowledge and skills.||In one course on game design, group assignments require students to create playable games that incorporate technical (e.g., programming) and design skills. To complete the assignment successfully, students from different disciplines must draw on one another’s strengths.|
|Create shared goals that can only be met through collaboration.||In one engineering course, teams compete against one another to design a boat (assessed on various dimensions such as stability and speed) by applying engineering principles and working within budgetary and material constraints. The fun and intensity of a public competition encourages the team to work closely together to create the best design possible.|
|Limit resources to compel students to share critical information and materials.||In a short-term project for an architectural design course, the instructor provides student groups with a set of materials (e.g., tape, cardboard, string) and assigns them the task of building a structure that conforms to particular design parameters using only these materials. Because students have limited resources, they cannot divide tasks but must strategize and work together.|
|Assign roles (.doc) within the group that will help facilitate collaboration.||In a semester-long research project for a history course, the instructor assigns students distinct roles within their groups: one student is responsible for initiating and sustaining communication with the rest of the group, another with coordinating schedules and organizing meetings, another with recording ideas generated and decisions made at meetings, and a fourth with keeping the group on task and cracking the whip when deadlines are approaching. The instructor rotates students through these roles, so that they each get practice performing each function.|
Devote time specifically to teamwork skills
Don’t assume students already know how to work in groups! While most students have worked on group projects before, they still may not have developed effective teamwork skills. By the same token, the teamwork skills they learned in one context (say on a soccer team or in a theatrical production) may not be directly applicable to another (e.g., a design project involving an external client.)
To work successfully in groups, students need to learn how to work with others to do things they might only know how to do individually, for example to...
- assess the nature and difficulty of a task
- break the task down into steps or stages
- plan a strategy
- manage time
Students also need to know how to handle issues that only arise in groups, for example, to:
- explain their ideas to others
- listen to alternative ideas and perspectives
- reach consensus
- delegate responsibilities
- coordinate efforts
- resolve conflicts
- integrate the contributions of multiple team members
Here are a few things you can do both to help students develop these skills and to see their value in professional life.
|Emphasize the practical importance of strong teamwork skills.||Explain the value of teamwork skills in (and outside) the workplace by offering real-world examples of how teams function and illustrating what can go wrong when teamwork skills are weak. One instructor asks students to generate a list of skills they believe employers look for. Often students answer this question with a set of domain-specific skills, such as drafting or computer programming. The instructor then contrasts their answers with the answers given by actual employers, who often focus on domain-general process skills such as “the ability to communicate clearly” and “the ability to work with others”. This activity serves to reinforce the process goals for group work assignments.|
|Address negative or inaccurate preconceptions about group work.||If students haven’t taken group projects seriously in previous courses or if their experiences were negative, it may affect how they approach assignments in your course. Consider asking them to list positive and negative aspects of groups based on their previous experiences and then to brainstorm strategies for preventing or mitigating potentially negative aspects of group work. Also explain how you have structured your assignment to minimize problems (such as the free-rider phenomenon) they may have encountered in the past.|
|Provide structure and guidance to help students plan.||Model the process of planning for a complex task by explaining how you would approach a similar task. Build time into the project schedule that is specifically devoted to planning.|
|Set interim deadlines.||Break the project down into steps or stages and set deadlines for interim deliverables, e.g., a project proposal, timeline, bibliography, first draft. In addition to setting interim deadlines, give students a rough sense of how long various steps of the project are likely to take and warn them about matters they will need to attend to earlier than they might expect.|
|Establish ground rules.||Create ground rules for group behavior or ask students to do so themselves. Group ground rules can include things such as: return e-mails from group members within 24 hours; come to meetings on time and prepared; meet deadlines; listen to what your teammates have to say; respond to one another’s comments politely but honestly; be constructive; criticize ideas, not people. You might then ask students to formally agree to these ground rules by signing a group learning contract (Barkley, Cross & Major, 2005). Find sample team contracts here…|
|Teach and reinforce conflict-resolution skills.||Disagreements within groups can provide valuable opportunities for students to develop both better teamwork skills and better end products (Thompson, 2004). But conflict can also erode motivation. To help students handle disagreements and tensions productively, provide language they can use to voice objections and preferences constructively and reinforce listening skills. Structured role-playing can also be helpful: present students with a hypothetical source of tension (e.g., a domineering personality, a slacker, cultural differences in communication style) before real tensions arise and then ask them to work toward a resolution, improvising dialogue and actions. Role-playing conflict-resolution in advance can help students recognize similar issues when they arise and respond to them creatively and appropriately.|
|Alert students to common pitfalls.||Point out potential pitfalls of team projects and/or your particular assignment. Common pitfalls may include underestimating the amount of time required to schedule meetings, coordinating access to labs, computer clusters, or studio space, getting research materials from Interlibrary Loan, obtaining IRB permission for research interviews, mailing reports to external clients, preparing presentations, revising reports, etc.|
|Foster metacognitive skills.|
Encourage students to assess their own strengths and weaknesses (e.g., tendency to procrastinate, openness to criticism, strong oral communication skills) and to consider how these traits could potentially affect group dynamics. One instructor gives students a self-assessment survey and lets group members compare their answers. Find sample self-assessments here...
He then asks: What mechanisms could your group put in place to capitalize on these strengths and compensate for these weaknesses? Answers generated include setting hard deadlines (if a number of group members are procrastinators), developing a system of turn-taking to make sure that everyone has the chance to speak (if there are shy group members), using flow charts to represent the task (for group members with a visual orientation or weak language skills), etc.
|Incorporate process assessments.||Ask students to periodically evaluate their own or others’ contributions to the group in relation to a set of process goals, such as respectfully listening to and considering opposing views or a minority opinion, effectively managing conflict around differences in ideas or approaches, keeping the group on track during and between meetings, promptness in meeting deadlines, etc. Then give groups a chance to generate strategies for improving their group processes.|
Build in individual accountability
It is possible for a student to work hard in a group and yet fail to understand crucial aspects of the project. In order to gauge whether individual students have met your criteria for understanding and mastery, it is important to structure individual accountability into your group work assignments.
In other words, in addition to evaluating the work of the group as a whole, ask individual group members to demonstrate their learning via quizzes, independent write-ups, weekly journal entries, etc. Not only does this help you monitor student learning, it helps to prevent the “free-rider” phenomenon. Students are considerably less likely to leave all the work to more responsible classmates if they know their individual performance will affect their grade.
To create individual accountability, some instructors combine a group project with an individual quiz on relevant material. Others base part of the total project grade on a group product (e.g., report, presentation, design, paper) and part on an individual submission. The individual portion might consist of a summary of the group’s decision-making process, a synthesis of lessons learned, a description of the individual student’s contributions to the group, etc.
One statistics instructor assigns student groups the task of presenting, synthesizing, and evaluating a set of articles on a particular topic. It is important to him that every group member have a firm grasp of the complete set of readings, even if they individually only present one or two. Thus, he builds individual accountability into the project by warning students in advance that he will ask each of them questions about the readings they did not present. This helps to ensure that students read the full set of articles, and not just the readings they present.
Barkley, E.F., Cross, K.P., and Major, C.H. (2005). Collaborative learning techniques: A handbook for college faculty. San Francisco: Jossey-Bass.
Johnson, D.W., Johnson, R., & Smith, K. (1998). Active learning: Cooperation in the college classroom. Edina, MN: Interaction Book Company.
Thompson, L.L. (2004). Making the team: A guide for managers. Upper Saddle River, NJ: Pearson Education Inc.