Criteria’s Employee Testing Blog

The Alaskan State of Mind

 digg it |  reddit |  del.icio.us |  StumbleUpon 

Just Who Lives in Alaska Anyhow?

A recent article in The Week entitled "Sizing Up the Last Frontier" notes that "after John McCain tapped Alaska Gov. Sarah Palin to be his running mate, the nation took a sudden interest in the quirky 49^th state," and asks "How is Alaska different than the lower 48?"

Well, Alaska has always attracted hunters, oil drillers, and gold miners, and men outnumber women there by a margin of 7 to 6. Alaskans consider themselves to be rugged individualists, and there's good reason for this: the state covers 656,424 square miles and has only 683,000 inhabitants. More than 60% of Alaskans were born elsewhere; in other words most Alaskans chose to live in the vast, beautiful and empty expanse they call home.

What kind of people make this choice? In our last blog post we commented on the Wall Street Journal article that summarized a recent study on personality with 600,000 internet volunteers from all 50 states and the District of Columbia. We discussed the state level rankings on personality traits such as agreeableness, openness and extraversion, and pointed out that there was a tendency for the states with smaller numbers of respondents to show up at the higher and lower ends of the rankings.

But even accounting for increased variance in small states, the data from Alaska stands out from the rest. Alaska scored 49^th in extraversion, 51^st in agreeableness, 51^st in conscientiousness, 47^th in neuroticism, and 49^th in openness. That's consistency--49th state indeed! It also makes you wonder a bit about how seriously the 4000 Alaskans were answering the survey.

With respect to agreeableness and conscientiousness, Alaska didn't just rank last in both, their scores were more than 3 standard deviations lower than the mean—very few states were more than one standard deviation off. Take a look at the plot below where we've shown how the states look on agreeableness and conscientiousness. The red dot is Alaska.

According to the data, then, Alaskans consider themselves much less agreeable and conscientious than their compatriots in the lower 48. But before we take any angry calls from the Alaskan Tourism Board, our point here is not to highlight Alaska's singularity, but to use Alaska as an example of how a single outlier can affect the conclusions one draws from a small sample size. What impact could Alaska have on the national scene when the personality measures are correlated with health, crime and business measures? This what statisiticians are refering to when they call something influential.

According to the plot, Alaska is as far removed from the other states psychologically as it is geographically. It is showing the same trend as the rest of the data shows, in that low scores on agreeableness seem to be associated with low scores on conscientiousness. But Alaska's position on the graph exaggerates that trend. When you include Alaska, the correlation between the two personality traits is r = .66. When you remove it, the correlation is r = .56. Like we said, in this case Alaska doesn't change how you would look at the data (you'd come to the same conclusion that there is a positive association), but it does affect the strength of the conclusion.

By contrast, look at a plot of the national data for conscientiousness plotted against neuroticism. The trend is that the less conscientious a state is, the more neurotic they appear to be. However, in this case Alaska (the red point) bucks the trend. Including Alaska, the correlation is r = -.27; excluding Alaska the correlation is r = -.38. Here, according to standard tests of statistical significance, whether or not you include Alaska would make a difference to the inference.

The take home message is that the personality data in the research study were matched up with dozens of potential interesting variables, and in some cases small correlations were observed. You'd have to look very closely at the individual plots of the data to determine whether Alaska was exaggerating or diminishing the final conclusions. It's obvious status as an outlier in these internet data makes it a potential deal maker or deal breaker in the national data.

The message as regards pre-employment testing is that small to medium sized companies should examine data on pre-employment assessments and job performance carefully. In a smallish sample, say 50 recent hires, there could be influential outliers that make the assessment look better or worse than it truly is.