the Jockers rebuttal

[Anonymous]

I don’t know how familiar you are of the Jocker’s study from 2008. In a nutshell, 3 Stanford University researchers published a paper in the Oxford Journal called Literary and Linguistic Computing. The authors concluded that major portions of the Book of Mormon exhibited Sidney Rigdon and Solomon Spaulding’s writing style, thus creating a resurgence of interest in the Spaulding Theory.

Last month, 3 BYU researchers and a man from Intermountain Health care published a paper refuting the claims for Jockers et al. I did a review in quite a bit of detail on my blog if you’re interested in a few details: http://www.mormonheretic.org/2011/02/09/debunking-the-jockers-study/

The conclusion of the study went like this:

Quote:

Using closed-set NSC, Jockers et al. (2008) attributed 37% of the chapters to Rigdon, 28% to Isaiah/Malachi, 20% to Spalding, 9% to Cowdery, 5% to Pratt, and 1% to Longfellow. In contrast, using open-set NSC, we conclude that 73% of the chapters cannot be reliably attributed to any of the candidate authors. We first note that Jockers et al. (2008) bolstered their NSC attributions by claiming close agreement between attribution results due to Burrows’ delta and those due to closed-set NSC. That these stylistic measures would nominally agree well numerically is not surprising because Burrows’ delta stylistic distance is closely related to the quadratic delta stylistic distance (Argamon, 2008) upon which NSC is based.

However, there actually is strong disagreement between the closed-set NSC results and the delta results. This is because delta-z scores should not be taken seriously unless they are very small (i.e. very negative). Burrows (2003) found that a threshold of 1.9 separated most false positives from true attributions for a set of 17th-century poets. Jockers et al. (2008) failed to do this. In the Jockers et al. (2008) study, only 16 of the 239 chapters had delta-z values as small as 1.9 (Fig. 9). Ten of these 16 chapters were essentially verbatim quotations of Isaiah/Malachi, and all 10 were correctly attributed to Isaiah/Malachi. Four additional chapters were attributed to Isaiah/Malachi and the others to Rigdon and Spalding. The remaining 223 chapters had large delta-z values and were thus apparently false positive. Hence, the delta results of Jockers et al. (2008) actually say little more than what is already uncontroversial about Book of Mormon authorship: that some of the chapters are quotations of Isaiah and Malachi. The delta-z results do not, in fact, attribute sizeable percentages of the chapters to Rigdon, Spalding, or Cowdery.

I have to say that the BYU guys really thought through this problem well. Jockers has plans for an updated study to include Joseph Smith, and other others. Judging from the BYU study, I think the Stanford folks have some serious problems. What are your thoughts?

[Anonymous]

My thoughts are that it’s all very interesting.

[Anonymous]

Yes, interesting. Thanks for that!

[Anonymous]

I’m a bit sceptical of the calculation methods. Isaiah/Malachi easy done, but the other stuff? Longfellow?! What’s that? Nephiawathahah?

[Anonymous]

My thoughts are I know nothing about this kind of analysis. I am more interested in why I can not find any left over steel swords and breastplates from a million or so dead warriors. 😆

[Anonymous]

You must be looking in the wrong place Cadence.

I honestly have never heard this song, I just found the lyrics on Google:

Look around

Where do you belong

Don’t be afraid

You’re not the only one

Don’t let the day go by

Don’t let it end

Don’t let a day go by in doubt

The answer lies within

Life is short

So learn from your mistakes

And stand behind

The choices that you make

Face each day

With both eyes open wide

And try to give

Don’t keep it all inside

Don’t let the day go by

Don’t let it end

Don’t let a day go by in doubt

The answer lies within

you’ve got the future on your side

You’re gonna be fine now

I know whatever you decide

You’re gonna shine

Don’t let the day go by

Don’t let it end

Don’t let a day go by in doubt

You’re ready to begin

Don’t let a day go by in doubt

The answer lies within

I know it’s absurd in the context of the discussion above, but that’s why I wanted to say it. To often we look to the external – for validation, for excitement, for reason, for answers, for proof. I’d like to reach a place that values all the external stuff, but really doesn’t need it.

[Anonymous]

mormonheretic wrote:

I don’t know how familiar you are of the Jocker’s study from 2008. In a nutshell, 3 Stanford University researchers published a paper in the Oxford Journal called Literary and Linguistic Computing. The authors concluded that major portions of the Book of Mormon exhibited Sidney Rigdon and Solomon Spaulding’s writing style, thus creating a resurgence of interest in the Spaulding Theory.

Last month, 3 BYU researchers and a man from Intermountain Health care published a paper refuting the claims for Jockers et al.

I have to say that the BYU guys really thought through this problem well. Jockers has plans for an updated study to include Joseph Smith, and other others. Judging from the BYU study, I think the Stanford folks have some serious problems. What are your thoughts?

The fact that they are from BYU already raises red flags for me. I’m not saying their research isn’t legitimate but I can’t help but be somewhat suspicious about the potential apologist mindset of assuming there is only one acceptable answer that they are already looking for before they even start. I would be interested to see how accurate these methods really are at positively identifying authors in cases where we know for sure who the authors were just to test how well they work. It seems strange that the Stanford researchers tried to test Solomon Spalding, Oliver Cowdery, and Sidney Rigdon but not Joseph Smith as potential authors.

[Anonymous]

Quote:

DevilsAdvocate,

It seems strange that the Stanford researchers tried to test Solomon Spalding, Oliver Cowdery, and Sidney Rigdon but not Joseph Smith as potential authors.

That’s my biggest problem with the Jockers study, though they did have legitimate reasons for excluding Joseph. NSC is a new statistical method for wordprint studies, and requires a writing sample from each test author. Joseph frequently used scribes, even for his journals. The Stanford researchers never felt like they could authenticate anything from Joseph Smith, so that’s why they left him out. I have a bit of sympathy for them, as I think they were making a sound judgment, but leaving Joseph out as a potential author is a major red flag with their paper. I have heard that they plan to write a follow up paper to include Joseph Smith.

As for bias, Craig Criddle is the third author of the Stanford article and former Mormon. He has a bit of an ax to grind, and was a motivating force for the project. While I understand the discomfort with BYU researchers, I was frankly impressed with how thorough they were. Not only did they base their paper on some pretty solid mathematical proofs, but they tested them out on the Federalist papers. They purposely excluded Alexander Hamilton, and used the same authors that Jockers used with Jockers method. Sidney Rigdon overwhelmingly was a false positive as having written the Federalist papers! This clearly shows the problems with the Jockers study.

The BYU guys proposed an open-set of authors proposal with and without Hamilton as author of the Federalist papers. When Hamilton was in, he was correctly identified as the author. When Hamilton was out, the test authors still didn’t give false positives. I think the BYU guys really put together a thoughtful experiment. While the Stanford guys are to be commended for using NSC as a legitimate wordprint method, the BYU guys refined it to weed out false positives. However, more work needs to be done, because the BYU guys ended with false positives too, but they were drastically reduced over the Stanford study.

[Anonymous]

Orson wrote:

Y

I know it’s absurd in the context of the discussion above, but that’s why I wanted to say it. To often we look to the external – for validation, for excitement, for reason, for answers, for proof. I’d like to reach a place that values all the external stuff, but really doesn’t need it.

Sorry that place is not for me. I want tangible definitive answers and solutions. I need that stuff. That is probably why religion is getting less appealing to me.

[Anonymous]

Cadence wrote:

Sorry that place is not for me. I want tangible definitive answers and solutions. I need that stuff. That is probably why religion is getting less appealing to me.

I understand the draw to the tangible, it’s just not going to exist for religion. When I decided I was going to StayLDS I realized to make it work I would need to let all that go; at least as far as religion is concerned. That’s my only point. I get where you’re coming from, and I know it’s hard to want anything different after a lifetime of taking things literally. You want to know what is solid and what you can hold onto. You need to find that for yourself first.

[Author]

Posts