The two largest sources of reviews for MBA admissions consultants, GMAT Club and Poets & Quants, have over 3,000 reviews altogether. It’s not hard to understand why a company might want to amass those sorts of reviews: the average consulting client pays $5-6k for services and, depending on the firm, refund policies can be draconian (if they exist at all).
And, just like when you’re purchasing toothpaste or socks, the savvy consumer looks to third-party cues like reviews to save themselves the heartache of a raw deal. If a consultant was terrible, the thinking goes, some jilted client would’ve taken to the internet to sound the alarm, right?
Counting the stars in review distributions
For the most part, this thinking works on Amazon:
By & large, customers are satisfied with their purchases. Of course, some folks end up with lemons, and some are simply impossible to please, so there’s a lot of action in negative reviews. All said & done, the reviews in this chart average out to about 3.90 stars.
So, how about admissions consultant reviews? Are they as effective at protecting you from lemons as Amazon reviews? Take a look at a distribution of a randomly sampled ~1400 reviews from our database:
Wow-what happened? How’d the stars align so uniformly in favor of… well, just about every consultant ever reviewed? And, more importantly, how the hell is this supposed to help you make a decision?
4 problems with 5-star reviews
We’ve done a lot of research into the thrilling world of admissions consultant reviews, and we’ve got the skinny on what we suspect went wrong.
Problem 1: Personal connections
An admissions consulting relationship runs deep. This isn’t someone fixing your plumbing or making your cuticles look fly–your consultant’s intimately aware of nearly every aspect of your personal story and, ideally, invested in your success, both professionally and emotionally. So, when the rejection letter comes (as it does in the majority of cases, let’s not forget), it’s often a blow to both the applicant and the consultant. In our experience, this shared disappointment has a softening effect on whatever evaluation an applicant might’ve written, likely adding a bit of padding to the rating.
Problem 2: Pay to play
Don’t forget that nearly every site you visit (including this one!) in the MBA admissions world relies on admissions consultants for some portion of their revenue. This isn’t a bad thing necessarily, and it definitely doesn’t mean reviews can’t work, but it’s worth examining the potential conflicts these mutual dependencies might create.
Take, for example, the reviews on GMAT Club. At the time of this post, the “Admissions Consultants” section has review data on around 20 different firms. admitbrain’s directory of admissions consultants, however, lists over 70 different firms. What gives?
It turns out that hosting reviews is a very premium service. We have a lot of respect for the venerable forum (after all: we learned math from Bunuel like so many others), and so we won’t list the fees, but suffice it to say that there’s a good reason ~70 % of consultancies don’t have any reviews available in GMAT Club’s archives.
Now, again, there’s nothing about this arrangement that’s unseemly, and indeed, since Poets & Quants doesn’t charge, it’s unlikely that fees are the lone culprit in the 5-star-epidemic. But, understanding this structure is critical to understanding the next point…
Problem 3: Conflicts of interest
In at least 3 major consulting firms, consultants are compensated for 5-star reviews received on one of the two platforms. We suspect this is tied to the cash outlay firms have to make in order to have the reviewing service available.
This means that, for the portion of consulting clients who don’t frequent GMAT Club or Poets & Quants, the idea of writing a review might not occur to them unless prompted by their consultant. Since the consultant has financial incentive to prompt (but only for 5 star reviews), we suspect a large portion of dissatisfied customers aren’t pointed to the review box.
We’ve got some experience as consultants ourselves and, we can tell you, it’s very clear which clients are likely to be dissatisfied and which aren’t long before the engagement’s over. As a result, there’s simply no way this doesn’t introduce some selection bias into these results.
Problem 4: Confirming identities
Last but not least, we have this idea of “confirmed” reviews. Because neither review platform is involved in the consultant/applicant transaction, verifying that the reviewer actually worked with the consultant takes some leg work (another reason hosting fees make some sense).
Both Poets & Quants and GMAT Club do their best to verify identities, but at the end of the day, the consultancy will almost certainly have some up-or-down say in whether or not the client was ever actually a client. We believe both platforms do a great job at verifying identities (a hard task to be sure), but from a structural standpoint, this arrangement at least creates the risk of impropriety, and thus bears mentioning here.
How do we fix it?
The good news is: we have several recommendations and one big idea for making reviews matter. The bad news is: you’ll have to tune in to the next post on this topic to learn about it :/ (this is 80% because we’re not finished with it, and 20% because we’re sleazy marketers trying to get you hooked on our sweet, sweet data-driven journalism).
You can subscribe to our email updates (of modest frequency) below, or you can pretend it’s the application status page & just click refresh for the next week (we humbly recommend the former).