What’s your view out your window? When you wake up in the morning and look out your window, what do you see? Do you see tall skyscrapers, your neighbor leaving for work in a quaint suburban neighborhood, or a line of trees at the edge of a big field?
Depending on where you live, what you understand and know to be “home” and your experiences at that home may be different. Where you live affects where you go to school, the types of extracurricular activities at school that you enjoy doing, the foods you eat, the hobbies you do with your friends, and even how confident you feel about your ability to do various tasks. In other words, people of various backgrounds (whether that is geographical location, socioeconomic status, race/ethnicity, age, or other cultural factors) may have similar or different life experiences that affect their interests and skills.
The way people understand and interpret things may vary due to time or cultural differences, so Kuder works hard to provide high quality and valid career guidance tools, and offer them in such a way that they can be accessible to anyone – regardless of their background. Therefore, the Kuder research team is currently collecting data in several regions of the United States in its effort to update the English versions of the Kuder Career Interest Assessment® (KCIA) and the Kuder Skills and Confidence Assessment® (KSCA) for secondary students. In doing so, focus groups and think-aloud sessions have taken place to help the team update the assessments’ items. Here’s an example of an assessment item:
In the current version of the KCIA, assessment-takers are asked to evaluate whether they would be likely to engage in various activities, and one of the options is “Operate heavy machinery.”
Even though the Kuder research team crafts each assessment item to serve a specific purpose, if they were to do this work without the input of others, in a vacuum so to speak, there may be unintentional bias due to their own life experiences, beliefs, or values; we can’t assume that the people we serve will always read the assessment items in the way in which we intend them to do.
A few qualitative methods that the Kuder research team implements when reviewing our assessments are focus groups and think-alouds. They’re both useful ways to get feedback from students who are similar to the respondents of the assessments; in essence, the participants represent their larger peer groups in the United States.
Specifically, both methods have allowed the Kuder research team to gain specific insight about the assessment questions (e.g., content, appropriateness, meaningfulness, relevance, word choice, etc.) and if any of the items are outdated or inappropriate for American students today. These sessions also allowed researchers to get students’ suggestions on how to update or replace the content of the items.
In general, there is a consensus among students in how they understand the assessment items and find them to be helpful in thinking about their own interests and skills confidence. Students have identified some assessment item content as potentially outdated (e.g., building shelves for CDs since “no one uses CDs anymore”), and they have shared their opinions regarding the phrases that they and their peers prefer to use.
Not only has this feedback helped us gain valuable insight on how students in various regions of the United States understand the assessment items, but we also gained valuable information on what their lives are like in their schools, homes, and communities – all of which informs our team as we endeavor to improve upon our assessments.
In order to learn how students from various parts of the country with diverse backgrounds understand and experience the Kuder assessments, the Kuder research team continues to work collaboratively with our client partners. The focus groups and think-aloud protocols with middle school and high school students would not have been possible without the help of leaders in schools and organizations in several regions.
Among them, the Houston Area Urban League (HAUL) graciously worked with the Kuder team over the summer to recruit students in the Houston, Texas, area to participate in our research project. HAUL is part of the Project Ready Post-Secondary Success Program, powered by Kuder®. Their feedback was vital to continuing our research along with their counterparts in the Midwest and Southeast.
As we continue to work to update our assessments to ensure they remain valid, we can’t express enough gratitude to our clients for their willingness and hard work to collaborate with us in our research efforts. On behalf of the Kuder research team, I want to say a big “thank you” to HAUL and the National Urban League, as well as our client partners in education who are committed to providing the best career guidance tools and resources available.
Regardless of what the view out your window may look like, everyone deserves a fair chance at discovering and exploring their interests, skills confidence, and the right to dream big and plan accordingly.
Deborah D. Lee is a doctoral candidate in the educational psychology program at The Pennsylvania State University. As a Kuder research fellow, she works closely with Dr. Hoi Suen on the development and psychometric analysis of the Kuder assessments. From conducting psychometric/statistical analyses to designing and leading think-aloud protocols, Deborah works to detect and remove cultural and/or linguistic biases to ensure Kuder assessments are reliable and valid. Deborah holds a bachelor’s degree in psychology from Emory University and a master’s degree in educational psychology from Penn State.
