WEBVTT

00:00:05.046 --> 00:00:06.826
My name is Liz City.

00:00:06.826 --> 00:00:08.436
I am the executive
director of the Doctor

00:00:08.436 --> 00:00:10.966
of Education Leadership Program
at the Harvard Graduate School

00:00:10.966 --> 00:00:13.756
of Education, where I am
also a lecturer on education.

00:00:14.546 --> 00:00:16.476
When you have collected
all of your data,

00:00:16.476 --> 00:00:18.566
you are facing a
large pile of data.

00:00:18.836 --> 00:00:21.186
Sometimes it's hard to
know how to dive in and how

00:00:21.186 --> 00:00:23.026
to analyze those data.

00:00:24.146 --> 00:00:26.626
One of the things that I
have found very helpful

00:00:26.626 --> 00:00:28.846
in my work is using protocols,

00:00:28.846 --> 00:00:31.586
or structured-based
conversations.

00:00:32.256 --> 00:00:34.346
Especially when you
are talking about data

00:00:34.346 --> 00:00:37.176
with other people-which is
really the best way to talk

00:00:37.176 --> 00:00:38.766
about data, is to do it

00:00:38.766 --> 00:00:41.216
with other people-they will
see things you don't see;

00:00:41.216 --> 00:00:43.446
you will collectively
decide what to do next.

00:00:44.106 --> 00:00:47.046
But the thing is, you have
very limited time in school.

00:00:47.046 --> 00:00:50.556
You may have 45 minutes for
your common planning time,

00:00:50.556 --> 00:00:53.596
and you need to figure out how
to efficiently use your time

00:00:53.936 --> 00:00:56.126
to look at data and
get somewhere with it.

00:00:56.716 --> 00:00:59.856
So different sorts of
protocols can be very helpful,

00:00:59.856 --> 00:01:02.476
even if it's as simple
as, "We are going to start

00:01:02.476 --> 00:01:06.636
with what do you see in
these data," to slow yourself

00:01:06.716 --> 00:01:10.066
down from everybody leaping to
conclusions and deciding what

00:01:10.066 --> 00:01:13.336
to do about the data, which is
something we educators often do.

00:01:13.666 --> 00:01:15.276
We want to do something
for kids,

00:01:15.276 --> 00:01:18.036
we want to very quickly leap
to something that's going

00:01:18.036 --> 00:01:21.106
to be helpful for
students, so we go from data

00:01:21.106 --> 00:01:23.136
to solution in five minutes.

00:01:23.666 --> 00:01:25.126
Well, that's not very helpful

00:01:25.126 --> 00:01:28.626
because you've skipped the whole
inquire part of the data cycle.

00:01:29.046 --> 00:01:32.136
You skip saying, "Why do
the data look this way?

00:01:32.256 --> 00:01:33.586
What are we seeing here?

00:01:33.586 --> 00:01:34.576
What do you notice?"

00:01:34.946 --> 00:01:37.926
So, often we use protocols that
start with "What do you see?

00:01:37.926 --> 00:01:38.756
What do you notice?

00:01:39.246 --> 00:01:41.986
What kinds of questions do you
have as you look at these data?

00:01:42.876 --> 00:01:45.276
What other data might we
want to explore to dig

00:01:45.276 --> 00:01:46.786
into these questions some?

00:01:47.396 --> 00:01:49.146
And what are our
next steps based

00:01:49.146 --> 00:01:50.676
on what we are seeing
in these data?"

00:01:51.126 --> 00:01:52.356
So something very simple

00:01:52.356 --> 00:01:54.356
like that can be helpful
for looking at data.

00:01:55.536 --> 00:01:58.616
Another thing that can be
helpful is, before you sit

00:01:58.616 --> 00:02:02.326
down in your collaborative
team meeting with those data,

00:02:02.986 --> 00:02:06.506
if somebody has processed
the data a little bit,

00:02:06.506 --> 00:02:09.116
especially if there is
number data, interim

00:02:09.116 --> 00:02:10.476
or standardized test data.

00:02:11.336 --> 00:02:14.786
Can you move the data from
charts and numbers to try

00:02:14.786 --> 00:02:16.326
to make pictures
out of the data?

00:02:16.916 --> 00:02:18.646
That will also help
us be more efficient.

00:02:18.646 --> 00:02:20.976
If we are looking at a bar
graph instead of trying to look

00:02:20.976 --> 00:02:24.406
at lines and lines of numbers,
can we just look at the picture

00:02:24.756 --> 00:02:27.986
to try to see, "Oh, what's
the story line here?"

00:02:28.636 --> 00:02:30.566
Whenever you are looking
at data, you are trying

00:02:30.566 --> 00:02:33.606
to figure out: What do I
see, what are the patterns,

00:02:33.866 --> 00:02:37.016
what's the story line, what
are these data telling me,

00:02:37.106 --> 00:02:38.566
and now what do I want to know?

00:02:39.006 --> 00:02:42.246
So that's really a
strategy for using data.

00:02:42.566 --> 00:02:43.936
It's not too fancy.

00:02:44.166 --> 00:02:46.586
You don't need fancy
programs necessarily.

00:02:46.586 --> 00:02:48.956
They can be helpful,
but really just sitting

00:02:48.956 --> 00:02:52.166
down with those kinds of
questions can help you work

00:02:52.166 --> 00:02:53.656
through data quite efficiently.

00:02:54.366 --> 00:02:55.866
Whenever you are
looking at data,

00:02:56.216 --> 00:02:57.826
you want to ask the
question why.

00:02:58.696 --> 00:03:00.826
You also want to
think about using data

00:03:00.826 --> 00:03:04.016
like you are a scientist, which
for those of us like myself

00:03:04.066 --> 00:03:06.576
who are English teachers
doesn't come naturally.

00:03:07.076 --> 00:03:09.106
But you want to think
about it like a scientist.

00:03:09.366 --> 00:03:12.116
You are presented with something
and you want to ask why.

00:03:12.386 --> 00:03:14.116
Why do the data look this way?

00:03:14.596 --> 00:03:17.326
So you might say
"testable hypotheses"

00:03:17.326 --> 00:03:19.366
but what you're really
trying to figure out is why.

00:03:20.266 --> 00:03:22.346
So when you look at a set
of data, you are going

00:03:22.346 --> 00:03:23.656
to ask the question why.

00:03:23.896 --> 00:03:25.806
A very popular protocol
that comes

00:03:25.806 --> 00:03:29.676
out of the business sector
is to ask why five times.

00:03:30.126 --> 00:03:33.546
You ask it once, "Why do
the data look this way."

00:03:33.856 --> 00:03:36.266
And then you say, "Ah,
we have an answer.

00:03:36.556 --> 00:03:37.786
Why is that true?"

00:03:37.786 --> 00:03:38.906
So I will give you an example.

00:03:39.746 --> 00:03:44.186
We are struggling with
students drawing inference

00:03:44.186 --> 00:03:47.106
when they are reading, a very
common problem in many schools.

00:03:47.566 --> 00:03:50.186
So why are students
struggling to draw inference?

00:03:50.636 --> 00:03:54.376
Well, we are not sure we are
explicitly teaching inference.

00:03:54.986 --> 00:03:57.016
Okay, why-why aren't
we teaching inference?

00:03:57.516 --> 00:04:00.616
Well, we are not sure we
share a common definition

00:04:00.616 --> 00:04:01.856
of what inference is.

00:04:02.446 --> 00:04:05.586
Hmm, why don't we share a common
definition of what inference is?

00:04:06.056 --> 00:04:08.796
Well, we haven't really done
professional development

00:04:08.796 --> 00:04:09.676
on inference.

00:04:09.676 --> 00:04:12.186
We have been really
focused on comprehension,

00:04:12.426 --> 00:04:14.166
but we think inference
might be different.

00:04:14.166 --> 00:04:16.616
All right, why haven't we
done professional development

00:04:16.616 --> 00:04:17.305
on inference?

00:04:17.846 --> 00:04:20.856
Because we haven't
allocated our time to that,

00:04:21.226 --> 00:04:22.916
and maybe if we focused on that,

00:04:23.076 --> 00:04:25.036
we could allocate our
professional development time

00:04:25.066 --> 00:04:25.256
to it.

00:04:25.256 --> 00:04:29.156
So if you ask these five
whys, you stop when you get

00:04:29.156 --> 00:04:30.356
to something actionable.

00:04:30.356 --> 00:04:33.006
A couple of other
characteristics

00:04:33.006 --> 00:04:36.916
of testable hypotheses are,
one, that they are testable,

00:04:37.336 --> 00:04:40.566
that you could figure out what
data would I look at to try

00:04:40.566 --> 00:04:43.836
to answer this question
to see if we are right

00:04:43.936 --> 00:04:45.336
about this hypothesis.

00:04:45.686 --> 00:04:47.146
So I will give you an example.

00:04:47.606 --> 00:04:51.376
It was in a district in
Iowa, not too long ago,

00:04:51.676 --> 00:04:53.846
and they were struggling
with transfer.

00:04:54.086 --> 00:04:54.986
They said, "You know what?

00:04:55.246 --> 00:04:58.196
Students aren't transferring
their knowledge from class

00:04:58.276 --> 00:05:02.476
to class within the same
grade or from grade to grade.

00:05:02.696 --> 00:05:04.646
They know something in fifth
grade, and then they go

00:05:04.646 --> 00:05:06.906
to sixth grade and it's like
they were never taught it."

00:05:07.146 --> 00:05:09.406
Something many of us have
experienced as teachers.

00:05:09.656 --> 00:05:12.706
So they said, "Why are
students struggling

00:05:12.706 --> 00:05:13.566
with inference [transfer]?

00:05:13.566 --> 00:05:15.176
What are our hypotheses?"

00:05:15.476 --> 00:05:18.156
They said, "Well,
we think it might be

00:05:18.156 --> 00:05:20.366
because we are not really
giving them high-level tasks,

00:05:21.406 --> 00:05:24.496
and we think we are maybe not
giving them enough opportunity

00:05:24.496 --> 00:05:27.466
to apply those tasks,
and we think

00:05:27.736 --> 00:05:30.496
that maybe we are doing
all the work-that we're

00:05:30.496 --> 00:05:32.066
up there teaching our brains out

00:05:32.066 --> 00:05:33.766
and students are
just sitting there.

00:05:34.396 --> 00:05:37.346
So let's see if we can
test those hypotheses."

00:05:38.086 --> 00:05:43.676
They then made a theory that
said if we taught students

00:05:44.046 --> 00:05:46.806
and had them do the work, if
we gave them opportunities

00:05:46.806 --> 00:05:48.906
for application, and if we
gave them high-level tasks,

00:05:49.656 --> 00:05:51.486
then we think they
would transfer.

00:05:52.216 --> 00:05:55.826
One of the things that's helpful
with testable hypotheses is

00:05:55.826 --> 00:05:57.956
that you have some
kind of theory

00:05:57.956 --> 00:06:00.926
about how you think the
learning should be working

00:06:01.306 --> 00:06:02.786
that you are tying back to.

00:06:03.296 --> 00:06:06.756
In this example in Iowa,
they had that theory coming

00:06:06.756 --> 00:06:09.566
out of their own experience
and also the literature.

00:06:09.886 --> 00:06:12.726
They went in and read about
transfer-what do we know

00:06:12.726 --> 00:06:15.376
about transfer, what does
the brain research say

00:06:15.376 --> 00:06:16.796
about how transfer works?

00:06:17.196 --> 00:06:19.696
And then they narrowed down
their list of hypotheses

00:06:19.696 --> 00:06:22.266
to those three and then
observed in classrooms

00:06:22.266 --> 00:06:23.666
to test their hypotheses.