Concerns-Based Adoption Model (CBAM)
Video: Levels of Use

Text Transcript of the Video

Archie George: Mostly, the problems that come about with change from the CBAM perspective have to do with, not so much that the innovation is not well designed or not effective, but rather it's use is not at the highest level right from the beginning, and so those are the aspects that you want to improve."

Narrator: Before you can assess the effectiveness of an innovation, you have to make sure that the innovation is being used. Levels of Use, or LoU, uses an interview protocol to measure the behaviors of the users -- to identify the extent to which they are using the innovation. The LoU dimension measures eight distinct Levels of Use. Level 0: Nonuse - The individual has little or no knowledge of the innovation, has no involvement with it, and is doing nothing toward becoming involved. Level 1: Orientation - The individual has acquired or is acquiring information about the innovation. Level 2: Preparation - The individual is preparing to use the innovation for the first time. Level 3: Mechanical Use - The user is focused on day-to-day use of the innovation and on mastering the tasks required to use the innovation. Mechanical use is often disjointed and superficial. Level 4A: Routine - The user has stabilized the ongoing use of the innovation and is making few, if any, changes. Level 4B: Refinement - The user is refining the use of the innovation in order to increase the impact it has on students. Level 5: Integration - The user is combining efforts with colleagues in order to have a collective impact on students. Level 6: Renewal - The user seeks major modifications or alternatives to the current innovation in order to increase its effectiveness and maximize the impact it has on students.

Gene Hall: "Levels of Use is an interesting look because in that work we're not looking at feelings, we're looking at behaviors. What is the person doing? The traditional way of thinking about it, in terms of especially the research world, is you have a treatment group and a control group. The world is dichotomous: use vs. nonuse. What we ended up figuring out was it's not that simple. So that's why we talked about "using." And we can identify three different ways behaviorally a person can be a nonuser. And then there are five different ways that they can be a user."

Narrator: As a general rule, 60 to 70% of the first-year users of an innovation will be at the mechanical level, Level 3. After an innovation has been in use for some time, however, the majority of users will be at Level 4A, routine use.

Gene Hall: "The other thing that happens is each Level of Use is assumed to be independent. If you look at them, there is a logical progression from zero through the highest Levels of Use, but each one has to be looked at independently."

Narrator: LoU is measured though a direct, focused interview, which can be supplemented with classroom observation.

Archie George: "The Levels of Use is an interview, and there are a standard set of questions in a branching format. So a trained interviewer starts out with a typical set of questions, and then, depending on the responses, they will ask different questions. So that they narrow in on the things they need to know in order to classify, in this case, an individual according to one Level of Use."

Gene Hall: "You have to get creative when it comes to the probing, but the key questions around each decision point in each category are standardized. That's because we found these questions worked, and also as a way of establishing the rigor and consistency for anyone doing Levels of Use interviews, so that if you collected data, we know it's going to be the same as if I collected the data; so we want people to ask the questions."

Narrator: The LoU manual provides more information on the LoU-focused interview.

Gene Hall: "Alright, I appreciate your taking time to meet with me. "We're trying to do a research study and learn more about what happens in the life of teachers in real classrooms. And as a part of that, is it okay if I tape record?" "Please do." "Okay. Thank you. What kinds of science teaching do you do?"

Archie George: "We learn a lot during those interviews about what's going on."

Teacher: "So they program it in a computer. They can look at it in 3D. They can look at the different interactions, atomic interactions, "or if it's a large protein, they can look at how different parts of the protein interact or how it folds."

Gene Hall: "Every person you interview becomes an interesting experience, and for the interviewee, it gives them an opportunity to describe what they're doing. So, for them, it becomes an enjoyable conversation."

Narrator: The information gathered through the LoU-focused interview can be used to monitor the progress of an implementation as well as to identify and address problems teachers might have in connecting a specific innovation or standards-based reform with their classroom practices.

Gene Hall: "One of my most favorites is when they're at that Level 3, mechanical use, which is how most of us use something the first time. There is a disjointedness to what we are doing. You have to refer to the user's manual, and you can't predict what you're going to do next week. It's the best you can do to plan for tomorrow. So just that Level of Use there alone points out some significant implications for us when we're involved in either evaluating or trying to facilitate a change process. You have to look at and understand where people are in terms of not just being a user, but their particular level within it."

Shirley Hord: "And so, monitoring to see where I need the help is very important to be able to give me appropriate help and assistance so that I become highly proficient in using the new program in my classroom with my students."

Gene Hall: "Levels of Use is not only useful for diagnosing for interventions, it becomes useful for researchers when they're trying to make sure, again, that they have treatment groups and control groups that are pure."

Narrator: Understanding, measuring, and applying Levels of Use data appropriately can play an important role in implementing and sustaining standards-based reforms. Studies have shown that teachers who are at higher Levels of Use will see a general pattern of higher student achievement.

Gene Hall: "So, once you've got the data, that's nice, but we have to figure out how do we help move that change process forward in constructive, positive ways."