Standard – Understand that we live and work within systems of cause and effect in which actions may have multiple origins and consequences.
As part of ED795A under Dr. Rossett, I worked as an educational technology consultant for the American Embassy School – New Delhi (a private coeducational K-12 day school that serves 1400 ex-pat children in New Delhi, India) to conduct a performance analysis on the way that the school is using student assessment data to improve student learning. In terms of systems I worked in conjunction with two separate but related entities within the school: the department of technology and the department of curriculum and instruction. As related to the systems standard, my role in this project was to develop the following:
- Interview members of the AES Data Team to determine actuals and barriers at each of the schools (elementary, middle, and high) within AES.
- Survey the staff electronically to determine barriers and needed supports
- Interpret and analyze the results of the staff survey.
- Report findings and recommendations to Warren Apel (Director of Technology) and Jan Patten (Director of Curriculum and Instruction).
By determining the efficacy of data use currently and providing a solution set based on the findings from this analysis, the AES Leadership Team could both make recommended changes and use the data provided as a base for further research.
When crafting the solution system it was vital to consider the systems in place and the inertia that accompanied them. To begin with the actual situation at each school (elementary, middle and high revealed) that each was at a completely different place and had very different systems in place for using data to improve student learning. The elementary school was using external data from the Iowa Test of Basic Skills and the International Schools Assessment. In addition grade levels were using data from internal common writing assessments to improve student writing. The middle school was also relying on the IT BS and ISA external assessments, while additionally all subject areas had developed internal common assessments that provided rich data to mine. The high school, at least those departments that were doing any data driven decision-making, were relying on a variety of acts turn all assessments including IB, AP, and Terra Nova test results. The inconsistent nature of data and the accompanying process for examining the data was yielding unsatisfactory results in all cases. The systems in place were not functioning as designed. In addition the school systems (administration, department of curriculum and instruction, technology department) were not addressing the barriers in a meaningful way.
Another set of systems that came into play with crafting the solution set was the interplay between teachers, administration, the department of curriculum and instruction and the technology department. Each of these systems seemed to have different optimals for the way that they envisioned using data to improve student learning. Crafting a solution set that would get all members on board was one of the challenges of this project. By interviewing members of the data team, the administrators themselves, and surveying the teachers I was able to accurately gauge the climate of each respective system. In terms of understanding the philosophy of using data to improve student learning the elementary and middle schools were very comfortable with the philosophy while the high school was not. In terms of motivation to use data again the elementary and middle schools were onboard however the high school lagged behind. In terms of knowledge of the process of using data to improve student learning and the skills to analyze data the middle and high schools were significantly lower than elementary. All three schools agreed that the ability to locate data was an issue. What this meant for me was that as I crafted the solution system I needed to take into account the variety of systems that were in place at each school. As a result a one-size-fits-all solution would not be appropriate. Armed with this data, Allison Rosset’s book First Things Fast and applying Keller’s ARCS model, my solution set addressed the barriers from each group and hopefully provide meaningful information for moving the school forward with this initiative.
Organizational systems are like eco-systems: what you do to one system impacts the others. For example one of the first pieces of the solution set was to have the Department of curriculum and instruction take a complete inventory of all assessments used at the school. This inventory will that help the technology department create a data warehouse which will serve as a one stop shop for all of the assessment data gathered in kindergarten through grade 12. In turn teachers at each school will be able to access data in a quick and efficient manner which will facilitate the data driven conversations. Finally a data warehouse will allow administration to be able to access data and teacher recommendations easily. This one intervention would have positive impacts on a number of systems throughout the school.
When dealing with complex systems effective communication is vital to success. In the case of using data to improve student learning at AES one of the stumbling points was that there was a lack of clear communication between the administrative team the teachers, the Department of curriculum and instruction, and the technology department regarding expectations and optimals. This ineffective communication lead to confusion during the rollout and a lack of motivation and buy in on the part of teachers for an otherwise important initiative.
Finally a structured instructional systems design model such as ADDIE is important when dealing with complex systems. In this case the analysis was solid however the design development and implementation of the initiative to use data to improve student learning at AES was lacking, and any meaningful evaluation of the initiative was absent. Using the ADDIE model would provide the various systems within an organization a common framework and language that would give system wide initiatives a better chance for success.