Sheila B Robinson

Reflections of an everyday educator/program evaluator/professional developer…LEARNER


15 Comments

When a Direct Question is NOT the Right Question

Who hasn’t answered the question, “What did you learn?” after attending a professional development session? As a PD facilitator and evaluator, I’ve certainly used feedback forms with this very question. After all, measuring participant learning is fundamental to PD evaluation.

In this post, I’ll share examples of actual data from PD evaluation in which we asked the direct question, “What did you learn?” I’ll then explain why this is a difficult question for PD participants to answer, resulting in unhelpful data. Next, I’ll offer a potential solution in the form of a different set of questions for PD evaluators to use in exploring the construct of participant learning. Finally, I’ll show where participant learning fits into the bigger picture of PD evaluation.  Continue reading


4 Comments

Checklists, and Protocols, and Rubrics…oh My! Evidence without Anguish?

Effective evaluation requires evidence. Documentation and data are the lifeblood of evidence. How can an evidence-based organizational culture balance the need to feed on the artifacts of work and other outputs and still respect the responsibilities of those producing them? How can we collect rich and meaningful data that informs our work and helps us make effective programmatic decisions while reducing respondent burden?

We are a data-hungry yet over-surveyed over-observed over-interviewed generation of workers. While data collection and analysis continue to grow and embed themselves into the fibers of organizational culture until they are indistinguishable from the “work,” are they considered the crabgrass or the crocuses? Insidious or delightful? Do they help or hinder the work?  Continue reading


4 Comments

A Roundup of Survey Design Resources (cross-post with actionable data)

Sheila here, writing with the magnificent Kim Firth Leonard of the actionable data blog.

Since agreeing that we would co-author a series of blog posts on surveys with a focus on composing good questions, we have discovered countless other blog posts, websites, journal articles, and books on survey research from a variety of fields and perspectives, many of which feature discussions of and advice on question construction. Of course, we have a few personal favorites and well dog-eared texts:  Continue reading


9 Comments

What makes a good question GOOD? (cross post with actionable data)

Now, THAT’S a very good question!

Sheila here, writing with the fabulous Kim Firth Leonard of the actionable data blog.

We have been tweeting and emailing about challenges related to survey design and as a result, this is the first in a series of posts we have co-authored about our discussions.  Continue reading


5 Comments

Like an evaluator in a data store…

I got to thinking about what I love about evaluation and it occurred to me that at least one aspect of it is particularly appealing due to its remarkable resemblance to shopping. Yup, that’s what it is. What I love about evaluation is collecting data!

The same rush of excitement I would get during a day at the mall I now get checking SurveyMonkey for incoming responses. It’s that “thrill of the hunt”, the acquisition, the addition to the collection that ignites the passion. As I become less materialistic and more fiscally responsible (as my golden years approach), I find myself engaging more in the latter than the former.

It happens even prior to data collection. Have you been to the Mall of America? Like planning out my day – thinking about what am I shopping for, how much I really need vs. how much I want, and which stores I will visit – I love designing the surveys. Who do I need to reach? How many of them? What do I really need to know from them vs. what I want to know? How will I get the biggest “bang for my buck?”

Continue reading