Design-Based Research
Design-Based Research
About: Design-Based Research
Outcomes:
The foci of this engagement are:
Learnings:
Certificate of Completion:
To receive your Certificate of Completion, please evaluate this module and complete the five true/false questions.
- Understand the difference between research practices based on methodology, paradigms and paradigms for contemporary evaluation practices.
- Understand the four major inquiry paradigms that influence contemporary evaluation practices.
- Describe several evaluation models that are relevant to evaluating interactive learning systems.
- Understand the importance of Design-Based Research for interactive learning systems.
The foci of this engagement are:
During this engagement, we will explore the research paradigms best
suited to evaluating technology-based learning.
Learnings:
- Develop an understanding and implementation of a Design-Based Research project.
Certificate of Completion:
To receive your Certificate of Completion, please evaluate this module and complete the five true/false questions.
Focus
areaInstitutional Change
Mode of
deliveryHybrid
ParticipantsBusiness Intelligence Developers,
Institutional Leads
Theory of
ChangeDoing
DifficultyBasic (No prior knowledge required)
Learning
approachTransformative (Authentic learning)
Research
Categories of Research
Depending on the purpose, methodology and scope of the research, the
approach can be
categorised in different ways.
Based on Methodology
Which of the following research methodologies are you familiar with? Please select all that apply.
Based on Research Paradigms(Generated by ChatGTP, 2024)
A research paradigm is a framework that guides how research is conducted, shaping the methods, design and interpretation of findings. It reflects underlying assumptions about how knowledge is constructed and what constitutes reality. There are several key paradigms in research, each suited to different types of inquiries and philosophical orientations.ChatGTP (2024)Which of the following research paradigms are you familiar with? Please select all that apply.
Classification of Educational Technology Research and Development articles (2009 -
2014)
However, Reeves and Hedburg (2003) i
argued that the field of interactive learning "is replete with alternative,
some would say incompatible, paradigms and diverse models," and they "attempt (ed) to
delineate four major paradigms related to the revaluation of interactive learning" (pp.
30).
Which of the following research paradigms are best to evaluate interactive learning? Please select only one paradigm.
Educational research is inspired by use (no or yes) or for understanding (yes or no)? Please select only one item.
About Design-Based Research
“..educational design research is an innovative and exceptionally
promising approach to
improving the quality and impact of educational research in general, and
educational
communications and technology research in particular.”
McKenney, S & Reeves, T. (2013). Educational Design Research. Handbook of Research on Educational Communications and Technology: Fourth Edition. 131-140.
McKenney, S & Reeves, T. (2013). Educational Design Research. Handbook of Research on Educational Communications and Technology: Fourth Edition. 131-140.
“EDR is not a specific research methodology, but rather an evolving research genre in which the iterative development of solutions to complex educational problems and the refinement of theoretical design principles provides the setting for rigorous scientific investigation.”
Reeves,
T. C., & Reeves, P. M. (2017). Educational design research. Mapping
the Field
of
Adult
and Continuing Education: An International
Compendium, 77.
“Design
research is not defined by its methods, but by the
goals of
those who pursue it.”
Bereiter,
C. (2002). Design research for sustained innovation.
Cognitive
Studies, 9(3), 321-327.
Design-Based Research
Design-Based Research (DBR) is a methodology used in educational research that focuses on the iterative design and refinement of educational interventions in real-world settings. The goal of DBR is to solve practical problems while simultaneously advancing theoretical knowledge. It is characterised by collaboration between researchers and practitioners, as well as continuous testing and modification of interventions. Key features include:- Iterative Process: DBR involves cycles of designing, testing, analysing, and refining interventions. Each iteration is used to improve both the intervention and the understanding of the underlying theory.
- Real-World Context: Research is conducted in authentic educational settings rather than in controlled, artificial environments. This allows for more relevant and applicable findings.
- Collaboration: Researchers work closely with practitioners (teachers, educators) to ensure that the intervention is practical, feasible, and aligned with the needs of the learners and the educational context.
- Theory and Practice Integration: DBR aims to contribute both to practical solutions (how to improve learning in a specific context) and to broader educational theories (why certain approaches work or don’t work).
- Focus on Learning: The primary goal is to improve learning outcomes by designing interventions that are informed by theory but grounded in the reality of the educational environment.
- Documentation and Reflection: Throughout the process, researchers document what happens and reflect on the successes and challenges. This reflective process helps refine the design and deepen understanding of the learning processes.
Design-Based Research (Fig. 1) progresses through several cycles of
analysis-exploration, design-construction and evaluation-reflection to
develop and improve a
solution to solve a particular problem or several related problems.
Figure 1: Educational Design-Based Research (redrawn from McKenney
&
Reeves1)
The identification of a problem (analysis-exploration) includes experts and
users.
Developing a solution to the problem consists of two components creating a
design before
constructing a learning artefact. To improve intervention and identify
design principles
(theoretical understanding) the testing of the solution in practice includes
two components:
evaluation and reflection. Several iterations through each stage, or the
whole process,
improve the solution and design principles that can be used in other
problem-solving
activities.
DBR is often used for developing and studying new instructional materials,
curricula, teaching methods, or technology-based learning tools. It is a
flexible approach
that adjusts to changing conditions in educational settings while aiming to
make meaningful
improvements to both educational practice and theory.
We
will explore an article entitled "Tool-mediated authentic learning
in an
educational technology course: a designed-based innovation" i to
explore the use of a Design-Based Research approach.
Figure 3: Student assessment of the authentic task design
principles.
Figure 4: Student assessment of the acquisition, participation,
and
contribution models of learning (**Friedman’s significance <
0.001, *Friedman’s significance <
0.05).
Table 2: Analyses of students’ opinions from 2007 and 2010
to several
questionnaire statements
on the design and delivery of the ICT course.
Figure 2: Research activities associated with the different
phases of the design
experiment: (a)
problem identification; (b) theoretical framework; (c, e, f) course
design;
(d, g)
student opinion
instrument; (h) authentic learning instrument; (i)
participation–facilitation–contribution
instrument; (j) course examination assessment; (k) design principles.
Over three design iterations, students participated in the
course
evaluation that included an assessment of the design principles
(Fig.
3); explored
the acquisition, participation, and contribution models of
learning
(Fig. 4); and offered their opinions on the design and delivery of
the
technology-based course (Table 1).
And the paradigm? Please select only one paradigm.
Design Principles
While Action Research and Design-Based Research both focus on solving practical issues, their approaches and outcomes differ:- Action Research focuses on localised problem-solving with the primary goal of immediate improvement in practice rather than theory generation.
- Design-Based Research aims to create generalisable theories by developing and testing educational interventions in real-world contexts. The iterative nature of Design-Based Research supports the development of declarative and procedural design principles.
Declarative Design Principles focus on what should be learned
or
understood, emphasizing facts, concepts, and knowledge structures. They are
about
establishing a foundation of knowledge that learners can recall and describe,
such as
historical dates, definitions, or concepts.
Procedural Design Principles, on the other hand, focus on how
to perform
tasks or apply knowledge through skills and procedures. These principles guide
learners in
practising processes or steps, such as solving equations, conducting
experiments, or
performing techniques in sports.
The principles of the above research included:
Declarative principles:
- Activity theory supports course design and evaluation
- Authentic learning tasks promote effective learning
- Educational technology (as tools) mediates knowledge construction.
Procedural principles:
- Use of the activity diagram as a heuristic to conceptualise course design
- Implement authentic learning as the object
- Incorporate educational technologies as tools to facilitate knowledge construction
- Reject course designs when educational technology functions as the object
- Use of activity diagram as a device to evaluate course design.
Activity: Exploring Designed-Based Research
Suggested time:
45 minutes
Authentic Scenario:
Your institution has identified several high-impact modules
(large
student enrolment with low pass rates). The DVC Teaching and Learning would like
you to
develop a plan to address this problem.
Task:
- Identify the practical problems with researchers, teaching and support staff and students.
- Develop an intervention using an appropriate theoretical framework.
- Identify and develop instruments for evaluating and testing aligned to the research paradigm.
- Develop a plan to document the processes to support reflection and development of the Design Principles.
Activity Learnings:
Develop an understanding and implementation of a
Design-Base Research
project.
Activity: Exploring Designed-Based Research
Suggested time:
45 minutes
Authentic Scenario:
Your institution has identified several high-impact
modules (large
student enrolment with low pass rates). The DVC Teaching and Learning
would like you to
develop a plan to address this problem.
Task:
- Identify the practical problems with researchers, teaching and support staff and students.
- Develop an intervention using an appropriate theoretical framework.
- Identify and develop instruments for evaluating and testing aligned to the research paradigm.
- Develop a plan to document the processes to support reflection and development of the Design Principles.
Activity Learnings:
Develop an understanding and implementation of a
Design-Base Research
project.
Resources
- Amory, A (2014) Tool-mediated authentic learning in an educational technology course: a designed-based innovation, Interactive Learning Environments, 22:4, 497-513
- Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational researcher, 33(7), 14-26.
- McKenney, S & Reeves, T. (2013). Educational Design Research. Handbook of Research on Educational Communications and Technology: Fourth Edition. 131-140.
- Reilly, C., & Reeves, T. C. (2024). Refining active learning design principles through design-based research. Active Learning in Higher Education, 25(1), 81-100.
Reflection and feedback.