First Advisor

Swapna Mukhopadhyay

Date of Publication

Spring 5-19-2016

Document Type


Degree Name

Doctor of Education (Ed.D.) in Educational Leadership: Curriculum and Instruction


Curriculum & Instruction




Environmental education -- Study and teaching (Middle school) -- Oregon -- Evaluation, Environmental literacy -- Study and teaching (Middle school) -- Oregon -- Evaluation, Educational tests and measurements



Physical Description

1 online resource (xii, 242 pages)


This mixed methods study evaluated the validity, and reliability of an instrument designed to assess a middle school student's proficiency in systems thinking as described in the 2010 Oregon Environmental Literacy Plan. In Stage 1, a forum of middle school students, formal, and non-formal educators used the Delphi technique to reach consensus regarding which skills were important to include in a scoring guide for systems thinking. In Stage 2, the scoring guide was field tested by formal and non-formal educators using a sample of students' work. The two groups' scores were compared using Cohen's kappa to make inferences regarding inter-rater reliability. Concurrently, an autoethnographic narrative was written to explore issues of equity related to the assessment of environmental literacy.

The commonalities between formal and non-formal educators revealed a high level of validity for the construct of proficiency with systems thinking, and a moderate level of reliability between the scores assigned by two groups of educators. In the words of the middle school students, formal, and non-formal educators, who volunteered to create the scoring guide, the ability to make responsible decisions with natural systems, community, and the future in mind involves: creating solutions for systems that are not in balance, presenting the complex inner workings of a system in a simple and succinct way, collaborating, exploring multiple solutions, and sharing ideas in a way that people will understand you.


In Copyright. URI: This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).

Persistent Identifier