Home
Jul. 20th, 2008 @ 07:19 pm Stunning Statistics
During U.S.Grand Lodge's Convocation in L.A. last weekend, IHQ's Grand Treasurer General cited a statistic that completely blew me away: Since being revived by Grady McMurtry, O.T.O. has initiated 13,000 individuals worldwide.

Think about it. We're talking roughly a 35-year timeframe, the same amount of time between Crowley becoming Grand Master of the UK (1912) and the time he died (1947). I don't have a list of every O.T.O. member in Crowley's lifetime (it would be interesting to compile), but I'd be surprised if it topped 200. So, in Crowley's 35 years in O.T.O., around a couple hundred people were initiated. In the last 35 years, O.T.O. has initiated thirteen thousand. It was really moving for me to think about how far the organization has come since A.C.'s days of struggling tirelessly to make his message and ceremonies accessible to people.

Also at the Convocation, the US Grand Treasurer General explained that USGL had tightened its criteria for counting "current" members.  Anyone who is more than 1 year in arrears nationally may be considered dues-lapsed (6 months for Minervals). [It had previoulsy been two years.] This new criterion leaves O.T.O.'s worldwide, active membership at 3,407 [source: O.T.O. IHQ GTG]. Combining the total number of individuals initiated and current membership (3407/13000), we arrive at 26% membership retention.

To judge whether this is high or low, let us consider retention rates described by another fraternal, initiatory, dues-based membership society: The Freemasons. According to 75 Ways to Attract and Retain New Members: A Handbook for the Leaders of Masonic Organizations (published by the Masonic Renweal Committee of North America):
  • "more than 80% of the members have not been present or active in their lodge for years" (Guide 2, p. 5)
  • "After the first year, more than 80% of all new members fail to return to meetings and remain active in their lodge" (Guide 6, p. 9).
  • "in most lodges in North America, only about 15% are active" (Guide 2, p. 5).
Based on these figures, we see that retention in Masonry is somewhere around 15-20%...apparently closer to 15% since the first two quotes above say "more than" 80% cease being members.

How does O.T.O.'s retention of 26% compare? Sure, it's a larger number than 15-20%, but is it appreciably larger or basically in the same ballpark? For this, we turn to the one-sample z-test for proportions. Assuming 20% is the norm, then O.T.O.'s 26% retention among 13,000 members falls 16.08 standard deviations above the mean. If we assume 15% is the norm, then 26% falls 29.04 standard deviations above the mean. Thus, a 26% retention rate is whoppingly larger than 15% or even 20%. (Technically speaking: If we assume the actual retention rate for fraternal organizations is 20%, then the odds are less than one in a million that the 26% reported by O.T.O. is due to nothing more than random variation. We therefore reject the idea that both organizations have identical retention rates and conclude that O.T.O. retains members at a significantly higher rate.)

While both groups would certainly like to see higher retention rates (provided both the members' and the organizations' needs are met), what factors may account for O.T.O.'s higher retention? While I expect O.T.O. would like to think they're doing a rockin' job--and that may well be the case--additional possible explanations could include some combination of the following:
  • Since it's possible to become a Master Mason (III°) in 2-3 months, members may reach a psychological "end of the line" sooner. (To reach the Master Magician III° degree of O.T.O. takes at least 1 year 9 months, though typically longer).
  • According to the aforementioned 75 Ways, two-thirds (66%) of those dropping out of Freemasonry indicate their reason being "that Masonry was not what they expected." No comparable statistic is available for O.T.O., although my anecdotal experience as a past master agrees with this: The overwhelming majority of people dropping out of O.T.O. do so because they discover it is not what they were looking for, and leave on amicable terms. What's interesting about those leaving Masonry is that those 2/3rds elaborate that "they were not prepared for the initiation, it surprised them and they were uncomfortable. The initiation was very uncomfortable for many. They said that no one had told them what to expect and this made them uncomfortable and even embarrassed" (Guide 6, p. 5). Given that the overwhelming majority of those joining O.T.O. are interested in, and most likely have some experience in, ceremonial magick, it seems unlikely that O.T.O. members would cite "being unprepared for the initiation" as their reason for leaving.
  • Unlike Masonry, O.T.O. includes both initiatic and ecclesiastical functions, which may offer members more incentive to remain active. Similarly, local O.T.O. bodies' offerings of rituals, classes and other events may provide more membership benefits than Masonic bodies, which are more oriented toward social and charitable activities. [In a survey about Masonry, the vast majority said they joined because they wanted more friends (85%) or connections (100%, Guide 2, p. 9); in addition, 92% said "charitable efforts were important to their joining," Guide 3 p. 7).]
  • Since it admits both men and women as members, O.T.O. members may experience less domestic pressures than members of an exclusive organization like Masonry.  [In a study of prospective members, 85% of men said "they would not join an organization that would not allow women as members," Guide 2, p. 9.]
At the end of the day, while we can conclude that O.T.O. has a statistically significantly higher retention rate than Freemasonry, the exact reasons cannot be determined from simple membership counts. These groups are far from identical, and differences in retention may reflect characteristics that differentiate the organizations from one another.

Hmm, I think I just wrote someone's Master's thesis for them. ;o)
What can I do?
Lon
[User Picture Icon]
From:[info]stevensteven
Date: July 21st, 2008 03:32 am (UTC)
(Permanent Link)
That was very interesting, and of course helpful to anyone running a local body. Of the 9k+ that quit, I would love to see a breakdown of what degree they quit at. I suspect most of them would be minervals, but it would be useful to know.
[User Picture Icon]
From:[info]richard_kaczyn
Date: July 21st, 2008 03:43 am (UTC)
(Permanent Link)
I don't have those figures, but--given the number of people in each degree--I agree that most would likely be Minervals.
[User Picture Icon]
From:[info]paradoxosalpha
Date: July 21st, 2008 03:52 am (UTC)
(Permanent Link)
In this same comparative frame, we can also introduce the fact that O.T.O. "makes higher demands" on members than Masonry does (to use the Iannacone/Stark supply-side theory language) or that it possesses the tactical advantage of "outsider" religious status (per R. Laurence Moore's work). These two different models treat overlapping characteristics that enhance the power of ritual/credal bodies to attract and retain members.

With respect to your "not what they expected" item, it occurs to me that one of the reasons for success in retaining O.T.O. initiates could be the extent to which the degree rituals actually do furnish the unexpected. Thelemic aspirants, in theory at least, have an appetite for personal transformation that makes such events desirable, in contrast to Masonry's espoused goals of bourgeois moral education and philanthropy.
[User Picture Icon]
From:[info]richard_kaczyn
Date: July 21st, 2008 01:04 pm (UTC)
(Permanent Link)
Thelemic aspirants, in theory at least, have an appetite for personal transformation

FWIW, my doctoral dissertation looked at the spiritual beliefs and experiences of a diverse group of believers. Those who self-identified as pagan/thelemic/magical (or categories to that effect) differed from more traditional groups in that they
  1. scored higher on the "Questing Scale," meaning they did not believe they had all the answers to life's questions, and were still looking, and
  2. were far more likely to report having had a transformative religious experience.


Edited at 2008-07-21 01:05 pm (UTC)
[User Picture Icon]
From:[info]jbarros
Date: July 22nd, 2008 07:52 pm (UTC)
(Permanent Link)
Disclaimer: I am not a mason.

That being said, I do have friends who are masons. They seem to dedicate quite a bit of time and money to their lodges, and I'm not sure on what you base the "higher demands" of the OTO on.

Disclaimer 2: I'm new to this. I'm not saying this isn't true, I'm just curious what the differences you see are regarding demands placed on members by the OTO vs the Masons.

Thanks :)

-- James
[User Picture Icon]
From:[info]rodneyorpheus
Date: July 21st, 2008 06:09 am (UTC)
(Permanent Link)
Whereas that's all very well, I think it important to remember the single basic fact: 3/4 of the membership have left, and we have no clue why.

Comparisons to Masonry are interesting, but let's bear in mind that Masonry is a dying organisation - literally. The average age of a Mason is what now? Over 80?

And we're more than a fraternal group. You point out that we offer ecclesiastical functions, which may offer more incentive. OK, let's compare our retention rates to some other religious bodies then as well. What do you think retention is for Scientologists? Or take your pick of other New Age religions, there are plenty out there. Or even Roman Catholics or Methodists? I'd be willing to bet that all of those have better than 26%...

Yes, we're doing relatively well compared to most small occult groups and fraternal orders. I still don't think it's enough over a 35 year period, by a long shot. Retention rates are too low, and the rate of initiating new members is way too low. I won't be happy unless we're sticking another zero on the end of our membership figures within the next 10-20 years.
[User Picture Icon]
From:[info]richard_kaczyn
Date: July 21st, 2008 01:47 pm (UTC)
(Permanent Link)
Thanks for your interesting points.

3/4 of the membership have left, and we have no clue why.

While we don't have national or international figures, I do think most local body masters know why particular individuals have left. "Not what I expected" is pretty common, in my experience. Some will cite financial reasons, being unable to afford dues (especially amongst students, the unemployed, the overly-indebted, etc.). Some are enjoying a spiritual buffet and are moving on to try Wicca, Druidry, the Golden Dawn, the A.'.A.'. or some other path. And some will leave because a personal conflict with an individual member makes their participation in the local body uncomfortable.

For those cases where the individual-O.T.O. fit isn't right, and the parting is on amicable terms, I think that shows the Minerval degree is working in that it allows people to check us out before making a big commitment. While the law is for all, I don't think the O.T.O. is for everyone. I don't say that in an elitist sense, but simply that we may not be everyone's cup of tea. And that's cool. They're doing their thing, and we've made an impression on them.

I agree with you in theory that we'd like to retain a lot more members, but what would that mean in practice? Neither of us, I'm sure, would permit O.T.O. to be watered down in order to make it more palatable for the average inquirer. And if we implemented some kind of careful Minerval screening process to ensure we only initiate those we're reasonably sure will stick around...we become bourgeois and contradict the notion that every man and woman has a right to the M.O.E. degrees (provided they find two sponsors).

That being said, I don't think that 26% is a magic number; if we can do better, that would be super. I just have no idea what a realistic expectation would be.

OK, let's compare our retention rates to some other religious bodies then as well. What do you think retention is for Scientologists? Or take your pick of other New Age religions, there are plenty out there. Or even Roman Catholics or Methodists? I'd be willing to bet that all of those have better than 26%...

Comparing MMM/EGC to other religions is tricky, although it would be good to look at that spectrum, too. Traditional religions have tremendous "hereditary" pressures from family members (I can still hear my mother saying, "What's the matter? Isn't the religion you were raised with good enough for you?") which--hopefully--doesn't exist in a faith that embraces freedom. Nor do we have concepts like sin, guilt and damnation ("Pay your local dues, or you'll spend eternity in heaven with Jerry Falwell!" "Nooooo!").

Also, once you're a member of the Roman Catholic church (for example), you're pretty much a member for life whether or not you ever again attend Mass or give another penny to the collection plate. So how does one actually count membership and retention?

If one believes the media accounts about Scientologists-and seeing how the media has historically treated A.C. and O.T.O., I take all that with a huge grain of salt--there are tremendous institutional pressures against leaving.

As for the New Age religions...I've seen plenty of dramatic departures, schisms, coven-hopping, etc. in that community too. I'd be very curious to see how retention statistics for a few of these groups compare.

Retention rates are too low, and the rate of initiating new members is way too low. I won't be happy unless we're sticking another zero on the end of our membership figures within the next 10-20 years.

So mote it be, bro. So mote it be.
[User Picture Icon]
From:[info]thiebes
Date: July 21st, 2008 09:18 pm (UTC)
(Permanent Link)
Whereas that's all very well, I think it important to remember the single basic fact: 3/4 of the membership have left, and we have no clue why.

Actually I think we have a lot of clues why, but it's not that important really IMO. Attrition is inevitable, and desirable, and ours is on the low side, so I think we needn't worry about that for the present. I think that what is really important to remember is that our worldwide membership would not fill 10% of the Green Bay stadium and that we need to bring a lot more people in if we plan to accomplish the many goals set forth in the Blue Equinox.
[User Picture Icon]
From:[info]lionserpent
Date: July 21st, 2008 03:20 pm (UTC)
(Permanent Link)
Thanks for the excellent and informative post. Those numbers are very exciting in context! I'm left with one question: how current are the numbers in 75 Ways to Attract and Retain New Members?
[User Picture Icon]
From:[info]richard_kaczyn
Date: July 21st, 2008 03:49 pm (UTC)
(Permanent Link)
The text of 75 Ways to Attract and Retain New Members refers to one of their data sources being a survey conducted "five years ago." The book has a copyright notice, but not copyright date, so it's hard to say how recent it is. The bibliography only cites books as recent as 1994; however, other pubs I have from the Masonic Renewal Committee are copyright 2007.

[User Picture Icon]
From:[info]solarmyth
Date: July 21st, 2008 04:25 pm (UTC)
(Permanent Link)
Have you taken a look at the loss/retention rates over time? I'm wondering what kind of fluctuations we might see there; especially since the Order has gone through a number of transformations since Grady reactivated it. I would expect to see spikes in loss every time a new, significant policy change was announced; but it would be interesting to see whether the non-spike rates changed over time.
[User Picture Icon]
From:[info]solarmyth
Date: July 21st, 2008 04:34 pm (UTC)
(Permanent Link)
Oh, yes--we shouldn't forget the fact that we've had a few mass dropouts due to schism over the years in certain regions, such as England and Brazil.

Furthermore, I wonder how areas like pre-war Yugoslavia were treated in this analysis. We had a huge member population there, at least on the books. The number we have in the current South Slavic states is considerably smaller than that, due to the war, to schisms, to record-keeping and reporting problems, etc. I think we have a good organization on the ground there now, especially in Croatia, but things were pretty wild-and-woolly there in the past.

I'd like to see how these worldwide numbers compare to the U.S. numbers.
[User Picture Icon]
From:[info]richard_kaczyn
Date: July 21st, 2008 04:44 pm (UTC)
(Permanent Link)
No, I haven't looked at these numbers over time. I only had the most recent figures to work with. I agree that it would be very interesting so see fluctuations over time, particularly (as you suggest) in connection with significant policy changes, founding of Grand Lodges, etc. We wouldn't be able to establish a cause-effect relationship, but it could be interesting to see if there are any associations.

Re fluctuation: That's the tricky thing about rates. They're a snapshot of a particular point in time. We need to be aware of what, in statistics, is called "right censoring:" we don't know how many people we're currently counting may eventually leave (and that is equally true for the Masons' figures). But that's also offset by new members joining.
[User Picture Icon]
From:[info]kakurowski
Date: July 21st, 2008 06:17 pm (UTC)
(Permanent Link)
Accurate data anywhere? Who tracks these things? I used to track why people leave, if they gave an answer. Most members don't. An accurate answer is far more slippery. This would require some type of consistently administed "exit survey" OTOI, but most people seem to just drop out of site without reason.
From:[info]slq
Date: July 23rd, 2008 09:18 pm (UTC)

Retention

(Permanent Link)
I think it is misleading to call the 26% figure a "retention rate". We are talking over 4 decades, here. There is a certain amount of attrition over that period of time which is inevitable.

For a snapshot of how things are doing today, at least in the U.S., look at the latest USGL Annual Report. Of the 1,513 members of USGL as of the end of the last fiscal year, there were 129 who left, for whatever reason (including being determined inactive). That makes the current retention rate more than 92%.
[User Picture Icon]
From:[info]z111
Date: July 28th, 2008 04:52 am (UTC)
(Permanent Link)
I'd expect that in many cases, there would be an initial drop (based on new policy) but a longer term gain that might not be able to be analyzed.

Sometimes you need The Tower to make things grow and the initial statistics would point down. I've seen it on local levels and I'd assume it happens on a national and international level too.
[User Picture Icon]
From:[info]threefold31
Date: July 21st, 2008 04:53 pm (UTC)

membership increase

(Permanent Link)
I for one don't understand the desire to dramatically increase OTO membership tenfold, or any other number. Is the organization somehow better with more members? Not necessarily. Is it desirable to have more people 'agree' with you and be in your club? Only if you're insecure in what you believe. Will you get more dues with more members? Yes, but if money is the point, then OTO is no better than a storefront church.
An increase in membership, as a goal, is no different than the proselytizng Christians/Scientologists, et. al. I sense an underlying righteousness and egotism in desiring dramatic membership gains for the OTO. And the more you attack that statement, the more you will prove my point.
It's all a simple case of quality vs. quantity. Not everyone who enlists in the Army gets to be in the Special Forces. And if everyone did, the forces wouldn't be so Special anymore, would they?
[User Picture Icon]
From:[info]rodneyorpheus
Date: July 21st, 2008 05:08 pm (UTC)

Re: membership increase

(Permanent Link)
Well exactly, O.T.O. isn't Special Forces. As both Richard and I point out above, it's open to everyone, that's one of our basic tenets.

And yes, I *absolutely* believe it's right to have more members, because I *am* secure in my belief in the rightness of our system. That's *precisely the point*.

And yes, damn straight I want a lot more dues-paying members, and more money in the Order. I want big-ass kitted-out temples, profess houses, educational programs, a rest home for retired members, the whole nine yards. We aren't going to achieve that with 3000 people, unless you have a couple of million bucks going spare...?

No, didn't think so. Back to Plan A then.

[User Picture Icon]
From:[info]stevensteven
Date: July 21st, 2008 05:11 pm (UTC)

Re: membership increase

(Permanent Link)
I for one don't understand the desire to dramatically increase OTO membership

Because it will take a lot more people than what we have to implement the full range of ideas in the Blue Equinox.

It's all a simple case of quality vs. quantity. Not everyone who enlists in the Army gets to be in the Special Forces.

One doesn't have to be special to join the MoE, only free, of full age, and of good report. Crowley is pretty clear about the need for quantity at this point.

"Members are however asked to remember two things; firstly, that at the beginning of a movement it is incumbent on all to exert themselves to the utmost to bring in new members irrespective of any special aptitudes that they may seem to possess; it is for the higher authorities to discover such; in the meantime the Lodge needs numerical strength."
-Aleister Crowley in a letter to Charles S. Jones
[User Picture Icon]
From:[info]agrc
Date: July 21st, 2008 06:35 pm (UTC)

Calculating Growth Rate

(Permanent Link)
93
Just a quick note, especially for those interested in the growth of the O.T.O. It would appear that the growth rate is: 182.86% per year over the last 35 year period. I specifically did not complicate the numbers by trying to calculate retention and my comparison figures listed below did not state any retention calculations were included. To come up with growth rate I used the Calculating Percent (Straight Line) Growth Rates formula. Based on the information in the original post: 200 initiated members (past value) and 13,000 initiated members (present value) over a 35 period (N).

Also, comparing that with the information I found on the 25 largest U.S. religious bodies (source: online article “Mainline Protestant churches no longer dominate NCC Yearbook’s list of top 25 U.S. religious bodies” March 30, 2008), the O.T.O. is doing extremely well in its attempt to grow its membership.

Here is an excerpt from the article discussing specific numbers:

"The Catholic Church remains the largest faith group in the U.S. with 67,259,768 members and a growth rate last year of 1.28 percent. The second largest denomination in the U.S. is still the Southern Baptist Convention with 16,439,603 members and a growth rate of 1.18 percent. The United Methodist Church is third largest with a reported membership of 8,251,175 and a growth rate of .002 percent.
The Church of Jesus of Latter-day Saints, with a reported membership of 5,503,192, rose from the fifth to the fourth largest church in the U.S. The yearbook noted that the church “continues to grow remarkably” at a rate of 1.71 percent last year.
A reported surge in membership of the Orthodox Church in America (OCA) has placed the communion on the list of the largest American churches. The Syosset, N.Y., based church grew 11.11 percent to 1-million members, according to the yearbook.
Other churches in the top 25 that continued to grow in 2004 are the Assemblies of God, 2,729,562 members and a growth rate of 1.57 percent; the Episcopal Church, 2,320,221 members and a growth rate of .57 percent; the African Methodist Episcopal Zion Church, 1,432,795 members and a growth rate of .14 percent; and Jehovah’s Witnesses, 1,041,030 members and a growth rate of 1.82 percent.
Churches that declined in membership in 2004 are the Evangelical Lutheran Church in America, 4,984,925 members, down 1.05 percent; the Presbyterian Church (USA), 3,241,309 members, down 4.87 percent; The Lutheran Church (Missouri Synod), 2,488,936 members, down .95 percent); American Baptist Churches in the USA, 1,433,075 members, down 3.45 percent; and the United Church of Christ, 1,296,652 members, down 2.58 percent.
The data is gathered by the churches in 2003 and reported to the yearbook in 2004."

Hope you enjoy the information.
93, 93/93
[User Picture Icon]
From:[info]richard_kaczyn
Date: July 21st, 2008 06:49 pm (UTC)

Re: Calculating Growth Rate

(Permanent Link)
Thanks for the informative comments! I'm not sure if basing the Order's growth rate on my guesswork figure of 200 members throughout Crowley's tenure is the best approach, given the periods of low activity and dormancy that followed his death. IHQ's GTG has reported that O.T.O. experienced, on average, a 4% growth rate the last couple of years...which is still higher than for the groups you describe.
[User Picture Icon]
From:[info]brandywilliams
Date: July 21st, 2008 08:37 pm (UTC)

Thank you

(Permanent Link)
This is great information and clearly put.
[User Picture Icon]
From:[info]richard_kaczyn
Date: July 21st, 2008 09:27 pm (UTC)

Re: Thank you

(Permanent Link)
Who knew that a couple of dry statistics could generate so much side-conversation? Next time I'll have to use bar-charts!
From:(Anonymous)
Date: July 21st, 2008 11:31 pm (UTC)

Retention

(Permanent Link)
Your method for determining retention rates is, unfortunately, inaccurate. There is a hidden assumption that the current 3,407 members are all "retained", when in fact, a large number of those are merely passing through OTO and will be ex-members very shortly. (In other words, you fail to account for the very thing you are trying to estimate -- the retention rate -- in the figure of 3,407.)

I was in email correspondence with the TG, and attempted to get to a figure of "core members" (akin to your "retained members"), but it's actually very difficult to get good statistics from their data, because of the membership structure of OTO and how dues payments are related to initiation fees. One needs to be careful about the criteria. As an example, Vere ran a query on members who are "I° and above who have at least 3 years (uninterrupted) active membership", which comes out at 795. But there is a problem with this in that it would exclude anyone who has (for whatever reason) missed a dues payment over the last 3 years. A member of 10 years standing but who is on a sabbatical would find themselves outside this sample. Also, a lot of people don't bother paying dues except when taking initiation, and they would mostly be outside of this. How do we account for these?

For a better idea of retention rates, I think you will need to include both dues-current and dues-lapsed members, because being dues current is -- in my opinion -- not an indication of dedication, and there are many "inactive" members who are nevertheless part of the OTO current in some sense. With this in mind, the retention rate may be roughly in line with what you suggest, but for entirely different reasons.

Furthermore, I imagine (but don't know) that the requirement to stay dues-current in Freemasonry is better enforced (also, perhaps, the membership are wealthier). Therefore a dues-based comparison is pretty weak.

-- Ian Rons
[User Picture Icon]
From:[info]richard_kaczyn
Date: July 22nd, 2008 02:01 am (UTC)

Re: Retention

(Permanent Link)
Thanks for your comments, Ian. Note that a rate is a snapshot in time, and is necessarily, as it is called in the statistics business, right censored in that we don't know who else may leave in the future; nor does it account for who else may join in the future. This is equally true for Masonic retention figures as it is for O.T.O. Similarly, the data are also left censored in that not everyone has been a member for the same length of time. This is pretty standard in survival analysis and proportional hazards models, and is not a limitation since a rate, by definition, is only concerned with the a specific time period. (Medical studies can only follow patients for a finite length of time, yet this doesn't prevent them from reporting rates and risks).

Mathematically, p=x/n is the most straightforward point estimate we have for a rate, where x is the # of objects of interest (in this case, current membership) and n is the sample or population size (all people ever initiated). This is called a point estimate because almost any attempt to measure anything in the world is imperfect, and is regarded as an approximation of the real-world value. Statistics are based on the notions of variability and probability: Any observed phenomenon is a combination of a True Score and Random Variation, the latter being due to either unmeasured influences or less than alpha=1.00 level reliability in the measurement instrument. This may be expressed mathematically as y = βx + μ + ε.

All things being equal and in the name of scientific rigour, the Masons and the O.T.O. would operationalize retention in exactly the same way. The risk in this approach, however, is imposing an arbitrary definition of membership that does not reflect reality in either organization (for instance, an O.T.O. member on sabbatical may reactivate in the future, but during their leave they have no rights or privileges as a member, e.g., s/he can attend public events but may not officiate in any official rituals). Accepting how each organization defines "current membership" is more respectful of their structural differences.

We don't know if the Masons and O.T.O. define retention the same way, but I think they're similar: The Masonic definition speaks about how many drop out after their first year. Similarly, a Minerval initiate pays for one year's dues at the time of initiation, and is, by definition, on the books for their first year, too. Thus, the O.T.O. statistic also looks at who drops out after an equivalent time period (i.e., one year).

In naturalistic and quasi-experimental research, it is often impossible to measure two groups by exactly the same criteria. It is true in this case, where I didn't have access to raw data for either group, I merely had their published figures. Such approaches to research are certainly valid, and in the scientific community one acknowledges these limitations and readers can consider the findings in light of those facts. (Which is why I never believe any statistic I see published in a newspaper or popular magazine: Too much of the detail gets omitted.)

The significance test for the two retention rates is a one-sample z-test for proportions. I used a one-sample rather than a two-sample test because I did not know either the numerator or denominator for the Masonic sample. Therefore, I used the Masonic figure as the population parameter or gold standard, and compared the O.T.O. figure to it where my null hypothesis was H0: pMasons = pO.T.O.. The equation is z = (pO.T.O.-pMasons)/SEM, where SEM is the standard error of the mean. In other words, the size of the difference between proportions is scaled relative to the amount of variation (or standard error) in the distribution of scores in the general population. z is expressed in standard deviation units, and 99.9% of all observations fall within +/- 3 SD of the mean. Thus, when I say that the difference between the Mason and O.T.O. retention rates is 19 or more standard deviations from the mean, we're talking a tremendous discrepancy between what we expected (H0) and what we observed.
Re: Retention - (Anonymous) Expand
Re: Retention - (Anonymous) Expand
[User Picture Icon]
From:[info]shepjoe
Date: July 22nd, 2008 12:18 pm (UTC)

50

(Permanent Link)
I have "nothing" to add I just wanted to be the 50th person to post something
( giggle)
[User Picture Icon]
From:[info]richard_kaczyn
Date: July 22nd, 2008 02:30 pm (UTC)

Re: 50

(Permanent Link)
DING DING DING! We have a winnah!
[User Picture Icon]
From:[info]azael93
Date: July 22nd, 2008 01:33 pm (UTC)
(Permanent Link)
Another factor that is difficult to track is those who may go inactive, but still sort of hang around the periphery of the Order. Alot of these are folks that maybe took a Minerval a long time ago and just never went any further or kept up with their dues. But maybe they come to Masses more or less regularly and maybe attend public events that a local body thows, like public classes or parties or whatnot.

Judging from my own anectdotal experience, I would think this is a statistically significant number of people. Obviously, it would be disengenious to count those folks for purposes of membership statistics, but they still exist as part of the community which the Order serves in one way or another.
[User Picture Icon]
From:[info]richard_kaczyn
Date: July 22nd, 2008 02:33 pm (UTC)
(Permanent Link)
You're absolutely right: Most O.T.O. bodies have a mechanism that allows dues-lapsed members (of any degree, really) to events by paying the per-event admission price extended to the general public. Counting those people would, in many minds, be cheating..but if we did, that would make our retention even better. ;o) Even if we wanted to, we have no mechanism for counting these people.
(no subject) - (Anonymous) Expand
[User Picture Icon]
From:[info]kakurowski
Date: July 24th, 2008 02:30 am (UTC)
(Permanent Link)
100