Yesterday’s NY Times reports on a dispute concerning the posting of the Rorschach Inkblots on Wikipedia. With all 10 inkblots posted, along with common answers, many psychologists argue the test has been compromised. Free information advocates argue that the test no longer had copyright protection, and therefore posting it is perfectly acceptable. It’s fitting that a debate about the grand-daddy of all projective tests — tests in which ambiguous stimuli provoke reactions that reveal aspects of a person’s mental state — should elicit this kind of polarized reaction.

I think most practicing professional psychologists would feel some degree of concern, knowing full well that they would not want their measures to be gamed, or to become so culturally exposed as to become irrelevant and invalid. However the advocates for posting the inkblots sound indifferent — one is quoted as laughing at the idea that a German publisher might be considering legal action, while the person who posted the Rorschach says he doesn’t care what experts think, he wants to be shown the actual damage caused by his actions.

At one point the article says that those opposed to the posting of the test feel that it is akin to posting a future version of the SAT. This is an iteresting point. Suppose we were discussing someone attempting to post next October’s SAT on Wikipedia. There would be no more laughing at the prospect of Educational Testing Services pressing legal action — Wikipedia would be doing everything in their power to absolve themselves. Presumably it would be an absolutely clear cut situation. However, there would be very little empirical evidence of the degree to which the SAT was compromised — the “no brainer” status would all be about the legal copyright. It seems like the free information advocates are failing to recognize that the hypothetical case of the compromised SAT and the actual case of the Rorschach are both worrisome to testing professionals because of the potential to undermine the integrity of the tests.

We at Criteria are also concerned about test security, as is the testing industry as a whole (Click here for a Wall Street Journal article related to test security). For some of our tests, like our neurocognitive tests, exposure isn’t really an issue. These are performance tests that show real time processing and responses. Other tests, such as our employment personality tests, are more like the Rorschach in that they do not have absolute correct and incorrect answers, although it is true that certain response profiles are considered more or less appropriate for certain jobs.

For special cases there is an interesting compromise between free information advocates and testing professionals. In some cases, it might be possible for the full range of testing materials to be freely available. Consider what ETS has done with the writing prompts for the GRE taken annually by half a million applicants to graduate schools. The prompts for which the students must write a short essay are posted online here. There are hundreds of prompts, although on test day an applicant will only encounter one. If they have prepared answers to each and every one — well more power to them; they have shown a remarkable degree of perseverance which ought to count for something. If they have memorized answers from a website, their work will easily be recognized as generic (think about that the next time you play the lottery numbers from your fortune cookie, and you have to share the Powerball prize with 110 other people).

We at Criteria think that open source testing might also have a place in undergraduate education. As of now, multiple choice tests in college classes are difficult to keep secure. Too many sororities and fraternities have files with pirated versions of tests. One solution to this is to build out a universe of potential test items. Students could have access to the open source version with all the possible testing materials, and come the time for the final exam, the instructor could use a subset of the items for the test. We’re working on ways to make this possible…..we’ll keep you posted. We’re just saying that if there were 2000 Rorschach inkblots, having them up on Wikipedia wouldn’t be such a compromising event. And that guy from Saskatoon probably wouldn’t have bothered posting them in the first place, making it even less of an issue.

A recent Department of Labor Report noted that mass layoffs at large companies surged in January. Not surprising, really, since we already knew the unemployment rate for January rose to 7.6% and job losses seem to be increasing by the day. But how is the job picture with respect to small and medium-sized businesses (SMBs)?

In my last blog post I mentioned a statistic that we track internally here, the Hiring Activity Index. The HAI is essentially a measure of how actively our customers (made up mostly of SMBs of between 10 and 500 employees) are administering pre-employment tests through our system (and presumably, therefore, hiring).  It is useful to us because it provides an overview of the breadth of usage of our pre-employment testing software.  It is calculated in a very simple way: the HAI is the percentage of our customers who are actively hiring (administering tests) in a given month.  From January 2008 (when we began tracking the HAI) to October 2008 the HAI remained very steady, within a few points of 65%. (If this seems low, consider that even in the best of times many 30 or 40 person companies will not be hiring every month.)

But as the financial markets plummeted and the unemployment rate surged in November, the HAI sunk about ten points, and by January reached its lowest level since we started tracking it, 53.28%. (The monthly readings for the HAI since January 2008 are listed below.)  Small companies, like their larger counterparts, had severely curtailed their hiring.  The financial news has not gotten much better since then; mass layoffs are continuing, we learned that the US GDP shrank 6.2% in Q4,and the stock market’s descent shows no signs of abating.

So I was very pleasantly surprised to see a fairly strong uptick in the HAI in February, to 61.41%. It is only one data point, to be sure, but it suggests that for SMBs the hiring picture improved somewhat in February. Could it be an upwards blip in a downward trend? Of course, but the eight point jump in the HAI is the biggest we’ve seen since we started tracking the index. For those, like me, inclined to think that the current recession, although brutal and severe, will not be as long-lasting as some suppose, the February HAI reading is cause for hope. I don’t expect that January’s 7.6% figure for the overall unemployment rate is the end of it–we’ll almost certainly see it get north of 8% soon. But as big public companies in the worst hit industries (financial services, construction, etc) continue to shed jobs the February HAI reading offers a glimmer of hope for the job market. Small and medium-sized businesses did not lead us into this recession, but they may just lead us out of it–and don’t look now, but it may have already started.

Hiring Activity Index