Thursday, April 26, 2012

Indexers & Arbitrators: Finding Common Ground

Scene 1 - I was visiting with my sister and we started talking about indexing. She's been working on some death records and told me how she doesn't submit her batches very quickly because she loves reading every detail & creating a story in her mind regarding the names she sees (Mom, is there another genealogist in the making?) She shared with me some of the interesting and sad details that she had come across

While looking at her account, most of her batches came back in the 97-99% accuracy range except for one batch that was 84%. We looked through that batch and saw multiple places where the arbitrated results did not match what we both felt was the correct result. We both felt frustrated that her average was going to be docked & the records would be incorrect. 

Scene 2 - I attended a virtual chapter meeting for the Utah Genealogical Association with Thomas McGill from FamilySearch (you can watch the recorded meeting in the members section of the UGA website) this month and the subject was indexing the 1940 Census. It was amazing to see that we have blown away all projections & there are more people signing up to index every day! We looked at graphs showing how the number of indexers has dramatically increased (as has the number of batches completed). The number of arbitrators has increased as well, but not at a high enough rate to keep up.

Many attendees brought up similar situations to the one above where they had a batch or two that received a very low accuracy percentage when in fact many of the "corrections" were wrong! Thomas explained that they were very aware that not all of the arbitrated results are accurate and that they are aware (i.e. they've been told many times over) how discouraging it can be from an indexers point of view. 

Scene 3 - I asked several friends who are indexing about their experience and most of them had at least one story to tell where an arbitrator corrected their work and the results were far from perfect. Many of them were frustrated that their hard work was being discounted. 

Scene 4 ("Ah-ha" moment) - While on FB today, a friend talked about how arbitrators are desperately needed right now. You see, all these batches are being submitted, but they can't be completed until someone arbitrates them. There were many comments, but one that stood out to me was from a former arbitrator:
I actually quit arbitrating because I was so upset about what my Facebook "friends" were saying about arbitrators. Yes, there is a greater purpose and I need to consider that, but the negativity and name calling broke my spirit.
Here comes the "ah-ha" moment. Arbitrators are human. They are people. Maybe you are an arbitrator. Arbitrators are not out to "get" you & they don't mess up your hard work on purpose. In fact, aren't we are working towards the same goal? To make this work accessible. To make it available. To help in future research. YES!!! 

So, yes, I'm a perfectionist & I don't want to see my accuracy tarnished by a batch that was not arbitrated the way I would agree is correct. I also want to make sure, the most accurate transcription is being produced. But I will support my arbitrators and realize that they, too, are doing their best. I will no longer grumble and gripe over that one bad batch, but I will consider the many others that were completed quickly and accurately. I thank my arbitrators for a job well done. Heck, maybe I'll even join the ranks. 

Are you with me?

And if you're still concerned about the accuracy, here's an excerpt from an blog that FamilySearch released on this subject:
  1. All indexing values from both indexers and the arbitrator are preserved in the FamilySearch database. At some future date, if needed, all three could be published side-by-side and made searchable.
  2. In the future, when FamilySearch publishes its public family tree, patrons will have the ability to make corrections or add alternative information, giving more richness to the collection.
  3. It is also likely in the future that patrons will have the ability to index individual records that they come across in their research, essentially on the fly. These researchers will likely be more familiar with the records than the average indexer and will provide a higher-quality index.
  4. Computers are getting more sophisticated all the time, and in some future scenarios it may be possible to program them to read handwriting so accurately that they will surpass the capabilities of both the casual and experienced indexer. This sounds futuristic but may, in reality, be closer than most would guess.
Here are a few more blog posts to read from Familysearch:

Carrie Keele
(a.k.a. DearMYRTLE's daughter)
www.NotYourMothersGenealogy.com

10 comments:

  1. Thoughtful and sensible approach, my dau-tur dear.

    ReplyDelete
  2. A good discussion to have, provided that we remember that we all are only indexing, not creating a record (the record will remain as accurate or inaccurate as it ever was). I do both indexing and arbitrating, and although most indexers were obviously doing a good job and trying hard, I could tell some stories about what some indexers do! But you won't hear them from me.

    I have chosen to work on states where I know the "terrain" better, and I think it makes my work on both sides of the divide better. So I pretty much disregard the "priority" states and keep to my own. Is there any thought whether that's a good approach?

    ReplyDelete
    Replies
    1. Good point, Harold! I think that a lot of people think the arbitrated result IS the END result. I used to think that until I dug a little deeper.

      When Thomas McGill was speaking at the virtual chapter meeting, he talked about how he selects batches to index much the same way you do. It really makes sense to work on what you know.

      Delete
  3. Yep, I'm with you Carrie! Arbitration does seem to be a thankless and underappreciated job, kinda like a referee; not everyone will be happy with the call you make. :)

    ReplyDelete
    Replies
    1. Hmmmmmm ... reminds me of motherhood!!

      Let me just say, THANKS Jana for being a great arbitrator! :)

      Delete
    2. Well, I'm not an arbitrator yet, I've only been asked to be one recently. Haven't decided yet to enter the referee arena. So, let me join you and say a big THANKS to all the referees, er, I mean Arbitrators out there!

      Delete
  4. You have a virtuous sense of fairness (!) towards both indexers and arbitrators! Great!

    Alsom I really like the idea of future patrons doing some adding and indexing. For example, when some census takers write our family name in cursive, it looks like KEWIN and often devolves to Ke** or Ki** on the census. Our name is KIRVEN or KERVIN, but write IRV or ERV in cursive and it looks just like EWI. Only messier. (Try it!) Took me months to figure out what was happening.

    There are probably more names lost in cursive.

    ReplyDelete
  5. I have a fairly oonsistent 98% rate. Haven't even looked at individual cases. It hasn't bothered me, and I remind myself of the capability on Ancestry.com for individuals to enter corrections or alternative interpretations. I have done a lot of that on the censuses in St. Augustine, Florida, where there are a lot of Spanish and other foreign names. As a history student at the University of North Florida, I made St. Augustine during the Second Spanish Period (1784-1821) my special area of study, so I am quite familiar with these names.

    I am also a paleographer (meaning I read old handwriting). I do not see that a computer program will be able to do what a well-trained and careful paleographer can do, because there is a lot of nuance in interpreting old handwriting.

    The bottom line here is: I don't get fussed about arbitration, and there is a lot of "wiggle room" in reading handwritten documents.

    ReplyDelete
  6. Great post, Carrie! We will all do well show some empathy. Arbitrator's need to be mindful of how the indexers feel when their work is corrected and indexers need to understand the difficulty of passing judgement when two people see the same information differently. We're all doing our best and a little grace is helpful.

    ReplyDelete