Data Based Decision Making

Standard – Value the use of data as the starting point for professional work.

Context

I worked as an educational technology consultant for the American Embassy School – New Delhi (AES) on a project to conduct a performance analysis on the way that the school is using student assessment data to improve student learning. This consulting opportunity provided me a way to demonstrate my mastery of the data-based decision making standard.

This consultancy was part of my experience in at 795A: Seminar In Educational Technology. The purpose of this assignment was to establish a client consultant relationship with a local company and develop a project which would showcase my talents as an instructional designer. My client was the American Embassy School – New Delhi a private coeducational K-12 day school that serves 1400 ex-pat children in New Delhi , India . I worked on this project in conjunction with Warner Apel the Director of Technology at the school and Jan Patten the Director of Curriculum. My role in this project was to develop the following:

  • Interviewing members of the AES Data Team to determine actuals and barriers at each of the schools (elementary, middle, and high) within AES.
  • Conducting a literature review and analyzing exemplar schools to help determine our optimals.
  • Survey the staff electronically to determine barriers and needed supports
  • Interpret and analyze the results of the staff survey.
  • Report findings and recommendations to Warren Apel and Jan Patten (Director of Curriculum and Instruction).

Outcomes

The outcomes of this performance analysis was to cultivate sufficient data through interviews, surveys, and a literature review to make clear statements concerning the use of data at AES to improve student learning, and to specify a series of recommendations for future improvement. By determining the efficacy of data use currently and providing a solution set based on the findings from this analysis, the AES Leadership Team could both make recommended changes and use the data provided as a base for further research.

Demonstrating Data based Decision-making

This Performance Analysis Plan (Table 1) demonstrates my mastery of the data based decision-making standard, “value the use of data is the starting point for professional work.” During this performance analysis project I effectively used the following methods and sources of data collection as the basis for my final recommendations:

Performance Analysis Plan

Problem: Data from common assessments is not being used most effectively to make instructional decisions

Stage Sources of Data Sample Key Questions
1. Sponsors –

Interviews with AES Data Team:

  • Jan Patten
  • Tricia Apel
  • Russ Daw
  • Scott White
  1. What is the problem?
  2. Why hasn’t it been solved so far?
  3. Why haven’t past attempts been successful?
  4. Is there a clear description of the process?
  5. What are the biggest barriers to making data driven decisions?
2. Our existing common assessment data, carts and graphs, and action plans with an expert (myself, Warren or Jan)
  1. What info do we have that defines or illustrates the problem?
  2. What is happening?
  3. What should be happening?
  4. What are the major problems in these work products?
  5. What are the most common errors?
3. Experts or model schools
  1. What is your process for making data driven decisions?
  2. What barriers did you run into?
  3. How did you deal with them?
  4. What advice do you have?
4. Job Incumbents

-Electronic Survey to all teaching staff

-Focus Group of Teachers

  1. What is getting in the way of making data driven decisions?
  2. Why are teachers having these problems?
  3. How would you solve the problem?
5. The Literature
  1. What does the literature say about the most typical barriers to success in these areas?
6. Supervisors

  • Warren Apel
  • Jan Patten
Share the major findings.

  1. Do they match your perceptions?
  2. Why do we have each of these barriers?
  3. What are the root causes?
  4. What can we do about them?

Table 1 Performance Analysis Plan

After collecting and analyzing my data I provided the American Embassy School with a set of specific recommendations for improving the use of student data in the data based decision-making process at AES. In my accompanying artifact, my final presentation PowerPoint I shared both the data found through my survey, literature review and interviews and then using that data presented graphs and recommendations based on the findings.

Lessons Learned

The lessons learned during this experience centered on moving outside my comfort zone, being flexible in applying performance technology skills, and communicating effectively. Departing the safety of the classroom after nineteen years was a challenge. This performance analysis practicum was my first time dealing intensively with an issue that affects the whole school. Initially I felt that the project I chose was a good fit as I was familiar with the initiative of using data to improve student learning from the classroom teacher perspective. What I was not prepared for was the complexity of the problem from the administrative point of view.

A second lesson learned was that flexibility is an important skill. First Things Fast served a cornerstone for my work, but what looked good on paper sometimes worked much differently in the field. For example the problem continued to become more complex at each turn. Initial conversations indicated that this was an organizational issue involving the tools that were used to warehouse the data. Initially, the task was to create a training program for teachers at AES on using a new data warehouse and set of analytic tools that the school was rolling out. However, the software that the previous Technology Director had purchased was not compatible with the student information system and was losing or corrupting the data that was being entered. The system was shelved. After further discussion about the process of using data to improve student learning at AES, it became clear that the issue was much bigger than just a faulty data warehouse. At this point it was determined that a performance analysis on data use for improving student learning at AES was in order. During the development of the solution system it became clear quickly that a one size fits all approach would not work. Keller’s ARCS model for motivation was an important piece of address the barriers at the high school in particular. Addressing relevance and confidence were especially important. In addition since the process of analyzing data at AES is inherently constructivist it made sense that the training interventions should mirror the process. Not being married to one approach and being flexible proved to be an important skill in creating the solution system.

The final lesson learned was that clear and effective communication is vital to getting results. To begin, it was important that when the initial idea fell through to bring in Jan Patten (Director of Curriculum and Instruction) to help provide insight into the process. Leveraging interpersonal skills to create this first important communication focused the analysis. In addition it allowed me to receive necessary support from the process end of things. By establishing these multiple relationships early in the project the analysis ran fairly smoothly. Finally, using many modes of communication expedited the gathering of data and the garnering feedback.