Liberal survey of priests on new translation is inaccurate

A little while ago, some people who hate on the new translation of the Roman Missal had a little survey of priests and their reception of the translation. It may not come as a huge surprise that the survey found that the majority of priests didn’t like it!

An expert on polls looked at the survey and offered that the results are probably not accurate. The respondents were self-selecting. Therefore, those who responded were those who wanted to vent. Even then, only only 59% didn’t like it, compared with 39% who did like it.

Sheesh! My polls are self-selecting too, but even I get better results than that!

From CNA with my emphases and comments:

Survey on priests’ dislike of Missal may be inaccurate
By Carl Bunderson

Hamden, Conn., May 25, 2013 / 04:03 pm (CNA/EWTN News).- A survey of U.S. priests’ attitudes towards the new English translation of the Roman Missal showing “widespread skepticism” may be inaccurate because of its methods, according to a polling expert.

On May 21, St. John’s School of Theology, located in [liberal]Collegeville, Minn., released its survey results saying that the majority of priests in America dislike the new Missal.

Of the some 1,500 priests who responded to the survey, 39 percent like the new text, and 59 percent dislike it, according to the Collegeville survey.

“All 178 Roman Catholic Latin rite dioceses in the U.S. were invited to take part in this study; 32 dioceses participated…in the period February 21 – May 6, 2013, priests in participating dioceses were invited to participate in the online survey via an email to all priests on the diocesan distribution list,” according to the survey’s executive summary.

Peter Brown, who is assistant director of Quinnipiac University’s Polling Institute, discussed polling procedures with CNA May 23. [NB:]“Random sampling is the key to getting accurate poll results,” he said.

Since only a few dioceses chose to participate in the survey – just under 18 percent – and only some priests in those dioceses chose to respond, survey respondents were “self-selecting.”  [That means that the sample wasn’t representative of priests in the USA.]

“They participated not randomly, but because they were the ones that chose to respond,” Brown explained. “Self-selected samples are not generally thought of….they don’t produce a random sample.

Since polls rely on a small number of people to represent the attitudes or beliefs of a larger population, “you have to be absolutely sure that the random group is a random group.

The Collegeville survey, Brown said, “might not meet those criteria” since its participants were self-selecting.

“It’s very difficult to know exactly” in this particular case, he added, though he had noted that self-selecting samples are generally not random.  [Did he mentioned “self-selecting”?]

The survey’s project manager, Chase Becker, is a graduate student in liturgical studies at St. John’s School of Theology, and holds a bachelor’s degree in philosophy. No ostensible polling experts were involved, and the survey’s professional consultant was an associate professor of psychology at the institution. [LOL!]

The poll also had no indication of its margin of error.  [It just get’s better, doesn’t it?]

The survey’s results were welcomed by vocal critics of the new translation, such as Bishop Donald W. Trautman, Erie’s bishop emeritus.  [There’s a shocker.]

[…]

 

You can read the rest of the autopsy over there.

 

FacebookEmailPinterestGoogle GmailShare/Bookmark

About Fr. John Zuhlsdorf

Fr. Z is the guy who runs this blog. o{]:¬)
This entry was posted in "How To..." - Practical Notes, Liberals, Liturgy Science Theatre 3000, POLLS, Priests and Priesthood, The Drill, Throwing a Nutty and tagged , , . Bookmark the permalink.

24 Responses to Liberal survey of priests on new translation is inaccurate

  1. acardnal says:

    I saw this article elsewhere and wondered why they did not gather and publicize the age and date of ordination of the respondents. It would have been interesting. I suspect that by and large the older priests who were (mal)formed in the seminaries of the the 1960s-80s were the one’s who did not like the new and more faithful translation of the missal.

  2. acardnal says:

    I bet you weren’t “selected” to respond to any questions in the survey were you Father Z? LOL.

  3. acardnal says:

    This poll is similar to asking Planned Parenthood employees (and not the general population) if they support abortion on demand. Daaaaah!

  4. iPadre says:

    I wonder what the average age of those who dislike the translation is. I’m sure it wouldn’t be surprising.

  5. Jack Regan says:

    While I have my issues with the New Translation (like the fact that some of it just isn’t functional English) there does come a point at which we have to just get on with life. It’s happened. We’re not going back. End of.

  6. rtjl says:

    “some of it just isn’t functional English”

    I’m not sure what you mean by that. Do you mean that it is not grammatically correct? Or do you mean that, even though it may be technically correct, it just doesn’t work?

  7. PhilipNeri says:

    Some of the language is a bit awkward. It takes some preparation on the part of the celebrant to look at the collects, etc. and make sure he has the right inflection and rhythm. And the vocabulary is occasionally. . .odd.

    I hope any future translations will be done with the help of a classically trained poet, someone to help the bishops smooth out the sound and pace of the prayers w/o damaging the theological content.

    Fr Philip Neri, OP

  8. wmeyer says:

    The root problem, of course, is that the translation to the vernacular fixes that which was not broken. And by use of a mechanism not supported by the documents of Vatican II.

  9. Tradster says:

    I look forward to a comparison with the survey taken in 1970 of the first liturgical changes that the liberals shoved down our throats. oh…wait…there wasn’t any because they didn’t give a rat’s tail about our opinions.

  10. Catholic Hokie says:

    A couple of points I took away from the survey, especially from the aspect of methodology:

    1) It is not that crazy that the professional consultant was a psychology professor. Many domains of psychology, such as community, social, and industrial/organizational use surveys and have developed lots and lots of statistical techniques to deal with survey research. Since the methodology (or reported methodology) does seem to be lacking, it very well could be that it’s just some psychology professor who didn’t know much. But don’t knock psychology professor as being consultants on these sorts of projects, they very often are the right people for it.

    2) The way this survey appears to be set up is like a lot of the on-line course evaluations at colleges and universities. At the end of a semester, each students gets an email with a survey to fill about each class he or she is enrolled in. It has the similar problem of self-selection, where the students that don’t like the class or are doing poorly (either through the instructor’s fault or, more likely, their own) writes the review. So if you look on these professor rating websites, they are heavily skewed towards saying negative things about the instructors. I think it’s the same thing here. You are given a survey to complain about the new translation, so the only priests who participants are the ones who want to complain, and therefore you get a big bias.

    3) Going back to the methodology, it would be interesting to see what kinds of demographic information they took from those who responded and have more detailed questions about specific aspects of the new translation. For this kind of research, you often have loads of questions dealing with different aspects of the Mass. It looks like they may have asked some different questions, but it would be interesting to see what other ones they might have asked. The demographic questions could also look at the background of leanings (for lack of better word) of the priests who responded. Wouldn’t it be interesting to have an age breakdown, and see how younger vs. older priests react to the new Missal?

    There are definitely some huge flaws in this report, and it’s unfortunate that people are jumping on this as support that priests dislike the new translation. As someone who has studied lots of these and similar methodologies, it is really sad how poorly this study was was carried out and publicized.

  11. ray from mn says:

    It is interesting that the administrator of the Pray Tell blog that emanates from St. John’s took a month’s leave starting just about the time that the results of the poll were released.

  12. louder says:

    Interesting that no religious order was asked to participate, and they make up a large percentage of priests in the USA. What is of no surprise is that the survey was used to discredit the new translation, especially since Collegeville had a huge financial/emotional stake in the old translation. It’s over, there’s no going back, deal with it Collegeville.

  13. JonPatrick says:

    To me it is discouraging that we are still fighting this battle over a baby step in the “Reform of the Reform”. What is going to happen when the necessary bigger reforms such as a return to Ad Orientam, communion kneeling on the tongue, etc. happen, if it is this hard just to have an accurate translation of the Mass?

  14. No doubt there is a problem statistically with a non-random sample, which this seems to be. But my brother Philip Neri makes a good point. The real question is not whether one likes the current translation or not, but whether one “likes” it better than the old one. I like it better than the old one, but that does not mean I like it. My problem is not with the vocabulary used, but with the syntax. The new translation often fails to follow standard (not colloquial) English word order and the natural order of clauses. This makes the result often unnatural and tortured. Thus it is hard to read out loud. Most of the time the major problems could have been changed if the translators had gotten an ordinary (literate) person to read the text out loud and then adjust it whenever the reader got confused by the syntax or word order. In fact, the Latin orationes are usually masterpieces of “proclaimable” Latin prose, with poetic elements like the cursus and balanced clauses, that make them easier to declaim or sing. In this the English failed to capture the true nature of the Latin. Again, that does not mean that I prefer the old ICEL.

    I am not sure that I agree with the statement above that students who do online course evaluations only do them if they are unhappy. The best teachers and the worse teachers attract many on-line evaluations at places like ratemyprofessors.com. So the results skew to the ends of the spectrum. But if the reviews are poor, the professor is probably poor indeed. After 25 years of teaching at Univ. of Oregon and Univ. of Virginia, where I was on promotion committees many times, I assure you that this is the case. The usual outlier is the professor who is an excessively hard grader: that professor will get both “worse” and “best” professor evaluations but nothing in the middle. That says something about the relationship between this responding group and the translation too, I think.

  15. RobertK says:

    I have to agree with Bishop Emeritus Trautman and these liberal Bishops. Except I prefer the 1962 translation :).

  16. RJHighland says:

    In my humble opinion I think is the same as Fr. Z’s on this that the new translation is better than the old but it would be better if everybody did the mass in Latin for universal uniformity in the Church. The best English translation to me of the prayers in general is that in the 1962 Missal. I for the life of me cannot figure out why they just didn’t keep the 1962 English translation. It just amazes me how much they changed and how much they took out. The new mass is more like a Methodist of Pray than the TLM in my opinion. Just do the Latin it has a natural flow and the music for any given mass has been around for centuries and sounds beautiful and flows with proper worship. The last Novus Ordo I went to was just awkward everything the responses, the music, the postures were very chaotic. Just do the Latin, Say the Back do the Read. And get rid of the missalettes just ask everybody that can afford them to buy them if you can’t afford them have a special offering after mass to fund those in the parish that cannot afford them and you can start off the fund with all the money your saving by not buying the missalettes.

  17. Sam Schmitt says:

    Evidently there was a final revision of the translation, but it seems it was done by a non-native speaker of English. This is the only way I can explain all the awkward words and phrases, and the lack of normal English rhythm in many of the prayers.

  18. The Masked Chicken says:

    What a stupid, idiotic survey. Asked how many of the respondents actually have a good reading knowledge of Latin, I’d bet the answer would be in the 20% range. What right do the rest have to any opinion about the new or old translation if they could not even translate the original, themselves? Silly.

    Clearly, the new translation is a better translation, because it more accurately reflects the underlying Latin text. The old translation was abysmal in some places. There is no opinion on this. It simply is a fact and anyone even answering the survey is automatically disqualified because they simply have no concept of the objective nature of the translations.

    This does not mean that the new translation is not without its faults, but really, this survey is silly.

    The Chicken

  19. chonak says:

    Going by the information on the survey website, it appears that the questionnaire did not include items about age, ordination year, educational level, native language, location, or any other characteristics that would allow for analysis. Because of that, it appears impossible to investigate whether the respondents were in any way “representative” of US priests as a whole.

  20. frival says:

    I could be wrong, but I think Fr. Powell just volunteered to be the poet in residence for the next round of translations. We ought all to commend him for his selfless offer.

  21. Joy says:

    I agree wtih frival – and I second Fr. Philip’s nomination to be poet in residence for the next round of translations. :-)

  22. Athelstan says:

    They’re certainly in a fit over at PTB over criticisms of the survey. And it seems they don’t quite understand how selection and response bias really work. It’s possible to have both if your methodology doesn’t rigorously try to achieve a representative random sample, even if you have the best intentions in the world (which we must assume, in charity, that the Diekmann Center does, in the absence of evidence to the contrary). They attempt to compare it favorable to a recent survey by CARA of the priesthood, but without noticing that previous CARA surveys on that subject were more rigorous and had far high response rates. In short, we simply cannot know how representative this survey is.

    If the point of the folks in Collegeville is merely that there are many priests who do not like the new MR3 – who really doubts that? Obviously there’s considerable discontent. But we should note that there’s multiple possible reasons for that, and not every objection is illegitimate. Yes, some are progressive theological objections like those of Bryan Cones at U.S. Catholic, terrified of all the monarchical imagery of God and so much emphasis on the Four Last Things. Yes, some are just old priests who don’t like change. But like Fr. Thompson says, there are valid concerns about the syntax and style of the new translation, which is often rather clunky: “The real question is not whether one likes the current translation or not, but whether one “likes” it better than the old one. I like it better than the old one, but that does not mean I like it. My problem is not with the vocabulary used, but with the syntax.”

    Like Fr. Thompson, I will take much greater accuracy over superior syntax, every time, and twice on Sunday. But to read older English translations of the Roman Rite, it’s obvious that an opportunity was missed. Traditionalists need look no further than their Lasance missal translation; but you can also see stylistically sound and beautiful translations in the Anglican Book of Common Prayer tradition, a variant of which is used by the Ordinariates now. There are many things for which we can deprecate Cranmer, but he had a real gift as a English stylist when he translated Sarum, and his gifts were shared by the Caroline divines.

  23. PhilipNeri says:

    If nominated, I’d accept; if elected, I’d serve.

    However, considering that I just turned 49 and new translations seem to come around only every forty years or so, the chances of being around to serve in the august position of Poet-in-Residence to the USCCB are slim to none.

    Alas, I was born three decades too soon!

    Fr. Philip Neri, OP

  24. Pingback: "One of Us" The Science of Life - BIG PULPIT