Project Outcome for Academic Libraries

Confusion over scale order?

Hi, thanks so much for creating this tool. We have been using it all semester and look forward to sharing the data gathered with our campus community.

I do have one question that has come up. We’ve received several surveys in which the respondents marked “Strongly Disagree” for all their answers, but left very positive comments in the open-ended section. It seems like maybe they misread the scale?

I’m wondering if user testing was done to find out which order makes more sense to respondents.

Thanks for any insights you can share!

Kendra Perry

Hi Kendra,

It’s great to hear you’ve been using Project Outcome and plan to share your data with stakeholders!

Regarding the order of the Likert scale options, there are a couple explanations. The simplest is that regardless of the order, there will likely be a few people who select the wrong option. Given that, there is some evidence that because people read left to right, putting the more positive answer on the left can create a positive bias - people choose the first option they see - which contributes to the ceiling effect in Likert-scale responses. Putting strongly agree on the right means that people should (theoretically) read through the options. Also, if you think of it numerically, 1 to 5 makes more sense than 5 to 1.

This model was designed and tested for the original public library toolkit; the ACRL task force discussed it and opted to keep the format consistent.

I hope that helps!

Sara (ACRL)

Thank you very much for the quick response! I understand that user error is unavoidable, and I’m glad to hear about the thought and research that went into the setup.

Thank you again!

Kendra

I’ve noticed the same thing, with a few respondents selecting all “strongly disagree” and then talking about how helpful the session was in the comments. When giving the surveys in classes, I’ve started verbally telling students that the scale goes from “strongly disagree” on the left to “strongly agree” on the right. It will be interesting to see how wide-spread this situation is.

-Dianna