- This topic is empty.
-
Posts
-
I have been subject to a lot of situations in leadership (both in and out of the church) where I am responsible for certain metrics. Often these metrics are a result of the agency of other people. The old home teaching program is a good example. We were responsible for seeing families, but the success of this metric was based on whether the family would let us see them. Under the new ministering program, stake leaders are likely held accountable for whether or not ward priesthood leaders are holding personal priesthood interviews. But the success of this metric depends on whether the local priesthood leaders decide to hold these interviews. It also depends on whether ministers in the quorum are willing to accept a personal priesthood interview.
Even now, Wards are held accountable for baptisms when the non-member has a strong influence over whether they actually get baptized. Same with “percent of endowed members with current temple recommends”. This metric is heavily dependent on whether endowed members have decided to renew their recommends.
Is it fair to hold leaders accountable for metrics that depend on others exercising their agency a certain way? How do you measure results FAIRLY when success of these measures depends so much on the agency of others, and not only the agency of the person/leader being held responsible for those metrics?I want to take one step back. It’s a church, right? Why are we measuring people against metrics and holding them responsible in the first place?
I am fairly involved with my local toastmasters club. There are similar metrics. How many members a club has. How many members are progressing. How many club officers attend trainings. Etc. Etc. For a global organization, metrics are an easy way to differentiate what areas are growing and thriving from what areas might need extra help.
Unfortunately, when emphasized too much these metrics become the focus of the leadership. If a CEO is judged by the stock price, (s)he might do things that are unwise long term but give a temporary boost. If the teacher’s evaluation is heavily weighted by standardized test scores, (s)he might teach almost exclusively to the test content. I have witnessed arm twisting and outright fabrication in order to achieve metrics. It can be a real problem. Getting over the goal line can take priority over more organic relationships and “pastoral care.”
I do think it’s important to measure things and set goals for improving things. But it’s a problem when the numbers become the goal or the wrong numbers are emphasized. We need to put what is must important at the top and measure the things we can that relate to it. For example if making and keeping temple covenants are the true end goal, then endowed members without a current recommend would be good to measure because it can show a trends that need to be investigated. The issue comes when the focus becomes raising the number by any means necessary in stead of figuring out if there’s a bigger reason behind a downward trend in an area. Numbers for their own sake became a big issue for me on my mission and that was really disenchanting. I had one companion in my second area who was laser focused on numbers. We were in the area that historically had the worst numbers across the board (for years), but once this elder got there the area “miraculously” had the highest numbers of lessons taught for the entire mission. We did work hard, but sacrificed “quality for quantity”, and obviously nothing real came of it. Then when that companion moved on (with a pat on the back and glowing praise from mission leaders) things went back to normal when my next companion came in (guess which companion I became good friends with and am still in touch with and guess how I was seen by mission leaders after that).
Quote:“When performance is measured, performance improves. When performance is measured and reported, the rate of improvement accelerates.”
This is the quote in Preach My Gospel that they always beat us over the head with.
Yes, “performance” improves when measured and reported. But what is “performance”? What are we actually measuring? Let’s take baptisms. Are you measuring the number of people coming to Christ? No. Number of people who will be lifelong members of the church? Also no. You are simply measuring how many people get submerged in a font. The problem that comes up, not only in church but also in business and education, is that usually the things that matter most are hard to measure. So we use easy to measure metrics. If we remember that we’re not actually measuring what we care about, we can be careful about how we interpret the numbers and keep focused on what really matters.
There’s also something called a perverse incentive. There’s an example where the British in India wanted to control the cobra population, so they offered people money for bringing in dead cobras. But then people started doing things like breeding cobras to make more money. Sometimes our focus on numbers can do similar things and lead to “baseball baptisms” and such.
As for how to measure, there’s not a really good way to measure that’s not at all based on other people’s agency. Maybe we should have opinion surveys that ask people how satisfied they are with their leaders. Ask if they feel their bishop is kind, respectful, forgiving, etc. Ask if they feel welcome. Solicit feedback, and try to improve on those metrics.
Actually, in college I find that the best measure of a professor is often the reviews on ratemyprofessor. Not test scores, publications, or anything else. We could crowdsource our metrics. Have missionaries have a QR code on their pass-along cards that lets their contacts take a survey about them. Were they pushy? Polite? What is your overall impression? Did they make you more or less interested? Offer a chance to win a gift card to get people to take the survey. Ok, that might be a little too far but I think the general concept of crowdsourced metrics could be interesting. I have taken similar surveys about the quality of service at the grocery checkout line, so some organizations do use this kind of thing.
When I posted the opening post, I was thinking more about the way leaders are held accountable. How can it be done with fairness when it is the agency of others that often determines the metric? How do you protect yourself as a leader when people above you simply look at low numbers and assume you are doing a poor job? The latter happened to me a lot in a Ward with low functioning priesthood members. In one Ward, I was sort of raked over the coals when we reported 99 prospective elders and only 1 active. Yet we had visited every one of them in the last 6 months to encourage activity and had made it a big area of focus. Upon reflection, I think it’s a matter of reporting the result metrics AND the process/activity efforts in pursuit of those metrics.For example, if you are a stake leader/high councilor/stake presidency member responsible for seeing the EQ presidents are holding PPI’s, you would report two things. You would report the number of PPI’s held by the EQ presidency, AND your own efforts to motivate the EQ presidency. This would mean appending your report with activity such as a) having held a meeting with the EQ president to go over PPI’s b) training you gave the EQ presidency in how to approach PPI’s c) the plan you asked the EQ Presidency submit showing their schedule for holding PPI’s.
If after all that activity the EQ still had low PPI’s, the Stake President, or person monitoring you, would know that the low numbers were due to the agency of the EQ, and not lack of leadership from the High Councilor or Stake President’s Counselor.
nibbler wrote:
I want to take one step back.It’s a church, right? Why are we measuring people against metrics and holding them responsible in the first place?
I think it’s important to measure results. Otherwise, you don’t know how you are doing. I don’t have a problem with metrics, but I do have a problem with a) coming down hard on priesthood leaders who are trying their best and b) undue emphasis on numbers without quality.
I also think it’s important for the people using the numbers (the leaders above the leaders) to DO THEIR HOMEWORK before making assumptions based on the numbers. The leaders above the leaders need to ask questions about what kind of activity went on to improve the numbers. Part of this would include questions about quality.
Is it fair to hold leaders accountable for metrics that depend on others exercising their agency a certain way?Not in a voluntary organization. Especially if I’m a volunteer in the organization.
Business organizations, schools, or other organizations that define success in dollars & cents than Yes.
I think in some situations, the church is moving away from using metrics. For example, Family History never uses measurements
for names submitted, baptisms, or number of names indexed or reviewed. They used to. You can’t generate a report by
individual member. (As far as I know.)
How do you measure results FAIRLY when success of these measures depends so much on the agency of others, and not only the agency of the person/leader being held responsible for those metrics?I use happiness, job satisfaction, extra curricular activities, friends. All of the requires have conversations with my wife, children & Grand
children.
I would never accept a position in the church if it were predicated on metrics of performance. period.
Interesting topic! My academic background is research-oriented and so numbers are important to me. But we have to know what those numbers represent. It is easy to assume that the number IS the construct that we are measuring rather than an indicator of that construct. When a missionary companionship reports “five baptisms” for one month, the simplest interpretation is that five people got down into the water and went through the baptismal ceremony. But what we REALLY mean (hopefully) is that five people converted and came into the Church (and by extension) Christ. But that may or may not be the case and so the “5” becomes an indicator only of conversion. (Numbers give the illusion of accuracy and preciseness but when we’re dealing with human behavior, those numbers are simply not going to have the same meaning). I do believe that performance improves when we measure something but we have be sure that we know what we’re measuring and what the numbers actually mean. The so-called “baseball baptisms” from the 50s and 60s occurred because those ceremonies were being counted (which is fine) but interpreted as being a sign of conversion (which most were not). This speaks to the evaluation that we often make of individuals (missionaries, bishops, businesspeople, teachers) for whom success is highly dependent upon the behavior of others. I think it’s problematic to assume that there is a straightforward relationship between a teacher teaching a lesson and a student’s score on a test of knowledge about that lesson. Absolutely, there’s a relationship but it’s not straightforward. The teacher has to ensure that the information is being presented but that teacher has no control over the chaotic homelife the child comes from, or the undiagnosed learning disability that the child may have, or the thoughts of the child dominated by the new computer game just purchased. Let’s measure, yes, and let’s hold people appropriately accountable but let’s also remember that the number is not the construct and that, when talking about incredibly reactive human beings, there are many, many variables over which any teacher, bishop etc, will have no control.
Quote:Actually, in college I find that the best measure of a professor is often the reviews on ratemyprofessor. Not test scores, publications, or anything else. We could crowdsource our metrics. Have missionaries have a QR code on their pass-along cards that lets their contacts take a survey about them. Were they pushy? Polite? What is your overall impression? Did they make you more or less interested? Offer a chance to win a gift card to get people to take the survey. Ok, that might be a little too far but I think the general concept of crowdsourced metrics could be interesting. I have taken similar surveys about the quality of service at the grocery checkout line, so some organizations do use this kind of thing.
As someone whose been in and out of academia, I have to say that ratemyprofessor is another good example of a metric which is, at best, an indicator only and, at worst, horribly inaccurate. But the idea of using something similar to measure church-related concepts is certainly interesting (ratemybishop? ratemystakepresident? ratemyeldersquorumpresident?) Now THOSE evaluations would make for some interesting perusing.
For years, our ward lacked meaningful accountability. The result was that few of our youth stayed past 15. Youth would show up for an activity & were left to do whatever they came up with. We’d go years w/o a missionary. Eagle scouts became a legend from a generation ago. Once an auxiliary leader was called, that was the end of the process. They might be pointed to a handbook that mentioned their calling. Forget about getting help assigned, no matter how many names were submitted. One time, I was so frustrated that I stapled a calling sheet to the ward roster and submitted that.
I think a better question than “Should there be accountability?” is “What should accountability look like?” We need some metrics to understand how we’re doing. If those figures get used to highlight shortcomings, however, we’ve lost our way.
In my experience, helpful, beneficial accountability looks like a stake/ward leader asking what they can do to help, regularly.
Also in my experience: When a ward goes a generation w/o helpful accountability, members can become conditioned to the unproductive status quo. Anything else can seem like contention. Members can rise against an auxiliary leader who’s trying to revive nearly dead programs. That’s a terrible state to be in. To quote Pres Hinckley, “I don’t recommend it”.
SilentDawning wrote:
In one Ward, I was sort of raked over the coals when we reported 99 prospective elders and only 1 active.
I am sorry that happened to you SD. I was also treated unfairly at times (I remember one time when I was publicly berated for not having brought a notebook to a leadership meeting to write down what was said). This was not the correct thing to do.When you say, “Hold leadership accountable,” I wonder what that looks like. I know that there are ways to socially and religiously pressure/punish/reward individuals. I suppose I am at a point in my faith evolution where most of these would no longer work on me.
NoahVail wrote:
For two decades, our ward lacked meaningful accountability. The result was that few of our youth stayed past 15. Youth would show up for an activity & were left to do whatever they came up with. We’d go years w/o a missionary. Eagle scouts became a legend from a generation ago.Once an auxiliary leader was called, that was the end of the process. They might be pointed to a handbook that mentioned their calling. Forget about getting assistants (teachers, etc), no matter how many names were submitted. This was our reality for over 20 years. Once I was so frustrated that I stapled a calling sheet to the ward roster and submitted that.
I think a better question than “Should there be accountability?” is “What should accountability look like?” We need some metrics to understand how we’re doing. If those figures get used to highlight shortcomings, however, we’ve lost our way.
In my experience, helpful, beneficial accountability looks like a stake/ward leader asking what they can do to help, regularly.
Also in my experience: When a ward goes a generation w/o helpful accountability, members can become conditioned to the unproductive status quo. Anything else can seem like contention. Members can rise against an auxiliary leader who’s trying to revive nearly dead programs. That’s a terrible state to be in. To quote Pres Hinckley, “I don’t recommend it”.
I understand where you are coming from and I know SD has seen the results of lackluster programs and follow through. I know that I volunteered at another Christian church for a time. At the end of the year they had a thank you dinner for all of the volunteers. They talked about how they couldn’t run the programs without all the volunteers, gave recognition, and some prizes (gift cards in small amounts). It was a nice way to feel appreciated. Certainly, this is a different approach to appealing to our sense of duty and guilt again (for what seems the hundredth time).
All the measuring we do at church is one the issues that makes it harder for me to stay LDS so some of this may get raw. 1) I feel most of the metrics we fixate on at church don’t have anything to do with helping people, they’re mostly focused on helping the church (organization), or in many cases not even helping the church, just being able to cite a good stat when reporting to leaders up the chain of command.
To what end? To avoid being reamed out? To earn a calling promotion?
2) Metrics, measuring performance, improving, improving at a rate that is higher than the rate of improvement of prior years – things that may be fine for corporate culture and developing a career but is church meant to be an extension of the corporate world, a respite from the corporate world, or somewhere in between? Or is it all just a part of any aspect of life?
3) How much of our measuring helps point us in the direction of people that need help? How much of our measuring tears us down because we feel we don’t measure up? Where’s the cutoff where measuring becomes an obsession that leads us down the same path the Pharisees went down… because I feel we’ve crossed that line in many areas.
4) Most of the metrics report on church programs. Historically if the numbers looked bad we browbeat members until they participated in the program. Maybe… just maybe a program has bad stats because there’s no interest in the program and lack of participation in a program isn’t a call for people to repent, it just means different people have different needs.
To be fair, we’ve seen some programs change recently to accommodate people but what I haven’t seen change is all the measuring and how unfavorable stats mean people aren’t doing a good enough job with participation in a program.
5) It’s hard to fault someone for not measuring up in a calling. They didn’t ask for that calling, they were told to do it and there’s immense pressure to not say no. In that light, many members couldn’t be called volunteers, they’re conscripts. How is it acceptable to tell someone to do something they’re not enthused about doing and then call them out for doing a bad job at it? If we aren’t going to bother aligning people with passions then I don’t see how we can have a discussion about expectations.
6) I’m tired. I’m so very, very tired of measuring and being measured in a place I’d like to go to escape from all of that.
This is another area where my ward or stake isn’t going to be like your ward or stake but in some areas I’ve seen us take things to extremes. My only solution is to ignore it all but I do not like the measuring we do at church and question why people need to be measured.
My issue with metrics in the church is that what counts can’t be counted. Our salvation or exaltation is extremely personal and almost entirely without meaningful intervention by others. We individually have to do whatever it takes and others can only minimally help – and then (after all that we can do no matter how little that is) there’s the infinite atonement of Jesus Christ. The atonement can’t be measured either. Thus I think we do more harm in trying to measure what we think can be, and the old home teaching program was a perfect example of why it doesn’t work. (And SD, if the current interviews that are supposed to take place are being done like they’re supposed to be done they are anything but a PPI. However, I too know those who can’t let go of old ways – another part of the problem.)
Roy wrote:
When you say, “Hold leadership accountable,” I wonder what that looks like. I know that there are ways to socially and religiously pressure/punish/reward individuals. I suppose I am at a point in my faith evolution where most of these would no longer work on me.
I had a long wrestle with what accountability should look like, and as Nibbler said, whether there should even be any.
My answer is that there MUST be some accountability. Otherwise, leading volunteers is a free-for-all where volunteers do whatever they please. The organization and higher leaders have needs just as volunteers do. The idea is to meet the needs of both volunteers, leadership and the organization at the same time and wherever possible. That doesn’t happen on a large scale when there isn’t accountability and isn’t an attempt to meet everyone’s needs.
So, what does accountability look like? I have no problem with there being numerical metrics that serve as a guide for suggesting areas volunteers need support from their leaders. For me, accountability leads to support, not censure.
Accountability also means the leaders put the right people in the right positions. No obligatory callings, no strong-arming people into callings with made-up claims about having revelation they are to be the next EQ president, for example. These things lead to installing people who have no interest in being held accountable.
Leaders need to do their homework on what motivates the members, and give them callings that serve the volunteers’ needs and the organization alike. This at least means there is grist for the mill – volunteers who have a desire and some ability to serve. If these fundamentals aren’t in place, then it’s hard to hold people accountable with creating hard feelings or push back. There has to be SOME commitment to doing well and getting with the organizational program for accountability to work.
For volunteers, it means being judicious about whether they agree to a calling. They should commit only when they have a desire to do a reasonable job of the calling. No half-hearted commitments just to make the person extending the calling go away.
The sad part of this is that our church does not have an appreciative, mutual approach to callings. It’s very top down, conscription type of model of service. People who indicate where they would like to serve are often put down as not following the top-down inspired model. It’s almost taken as an affront to the leadership if you come forward and share the places you would like to serve. For me, this is important information I would welcome from any volunteer. It shows where the volunteer has passion and would definitely inform my own inspiration on the subject of where to put that person.
With this foundation in place, I don’t think volunteers or leaders would hestitate to rely on metrics of some kind that help leaders give support. Metrics that also help the volunteers see how they are doing.
DJ makes a good point — that it’s hard to measure what is important. But even when what is important is hard to measure, we should try. In trying, we learn to refine our metrics. Also, sometimes all you can measure is process or the flurry activity. I am OK with that. If there is no activity we know nothing substantial is happening in the metrics that really matter.
I do think the church should be measuring how the members FEEL about attending their wards and serving in the church. Metrics like these would go a long way to helping the leadership make plans about what is truly important.
In summary — I think metrics are important, that the organization has a right to collect and share them, and that volunteers should embrace some accountability. But this accountability should be applied to people who WANT to be serving in their position. And such accountability should lead to kind, loving support from leaders and not censure like I’ve seen in other contexts.
- You must be logged in to reply to this topic.