Author

My photo
Racine, Wisconsin, United States
We (my wife and I) are celebrating the 11th Anniversary of HAPLR, and more importantly, our 38th Anniversary. The HAPLR system uses data provided by 9,000 public libraries in the United States to create comparative rankings. The comparisons are in broad population categories. HAPLR provides a comparative rating system that librarians, trustees and the public can use to improve and extend library services. I am the director of Waukesha County Federated Library System.

Sunday, November 29, 2009

Still Misbehaving

LJ’s Rebecca Miller's response to my question blames federal numbers rather than the LJ Index design. The question is not just about the possibility that there are numbers that are inaccurate in the IMLS dataset. That is a given.

The question is “How can the LJ Index “Score Calculation Algorithm” allow one measurement to swamp the entire score for a library?” This type of data “misbehavior” does not happen with HAPLR because it uses percentiles. When a library gets to the 99th percentile, that is the end of things and no measure swamps the entire score.

In the LJ Index, three measures may be the worst in the category, but if one score is an outlier, the LJ Index Score for the library will be high. This misbehaving data is by design and it is the point that Miller misses in her response. Why design an index this way?

Miller also states: “While the circumstances are embarrassing for San Diego [County], the LJ Index did precisely one of the things it was designed to do: shine a spotlight on inaccurate data so it can be corrected.” I must have missed that “spotlight on inaccurate data” in the LJ report. She responded to my questions about the Index. It looks more like my spotlight than LJ’s that caused the discussion.

Miller argues that LJ, Bibliostat, and Baker and Taylor (the sponsors of the LJ Index) “cannot use time consuming and expensive methods” to check the information provided by IMLS. I agree about time consuming methods, but the method I used is neither time consuming nor expensive.

I simply divided “Public Internet Uses per Capita” by “Visits per Capita.” The result is Public Internet Uses per Visit. That calculation is all it takes to spotlight nearly 100 libraries where the reported number of Internet Uses exceeded the number of visitors. No reasonable observer would fail to question such high outputs. Each San Diego County visitor was reported to have used a Public Internet Terminal over 4 times at each and every visit. In one library the number was 8 uses per visit by every visitor! There are plenty of spotlights that LJ chose to ignore until asked about the problems. Where are the remaining spotlights for these libraries?

I have taken a lot of criticism for not including electronic use measures into HAPLR but the numbers always seemed too unreliable to me. Several years ago LJ Index co-author Keith Lance agreed with me at an ALA Conference. He noted that HAPLR should not use the electronic use data because the numbers were too unreliable. When he chose to use these numbers for the LJ Index, I assumed, apparently wrongly, that he had satisfied himself that they were now more reliable.

I liked Susan Mark's analysis. It is true, as she notes, that population and how it is assigned to libraries has a very large amount to do with both ratings.


http://library.utah.gov/programs/development/statistics/documents/HAPLRversusLJ.pdf

No comments:

Post a Comment

Blog Archive