Social proofiness: spotting digital potemkin numbers

If you still think 'follower count' is meaningful indicator of anything, we badly must rehabilitate you

Scene from Arrested Development where they build a fake “potemkin neighborhood” that looked real from afar to create investor confidence

The concept of digital social proof is not a new one. It’s been around since the 90’s as a quantitative data point such as number counts for posters on forums and boards (users used to have the “number of posts” they had under their name on a forum or board).

Post count would help act as a proof point for long-time community members and show their dedication to the forum. It wouldn’t necessarily indicate that person is trustworthy, just more active in that community (trust can’t be put into a neat number in an automated fashion). And of course, qualitative proof points have been around forever, think: “as seen on TV” or “as seen in the New York Times.”

On my (acquired) former blog, I wrote several posts outlining how people abuse vanity metrics online in order to justify the sale of expensive products or services. Those types haven’t changed (if anything, we’ve just spawned more since). But what has happened is the number of people that are attempting to game social proof (and associated metrics) such as snake oil agencies, consultants, stock and crypto promoters and even some brands.

The thing about social proof is it has to be legitimate for smart people to take it seriously. And I think the web as a whole has gotten a lot smarter. But that hasn’t stopped many from building entire businesses around Potemkin numbers/faked social proof and exploiting those still easily manipulated. We use numbers as shortcuts to quickly understand if we should buy a product (5 star rating on Amazon) or if it’s worth attending a conference (10 years running). These are good things. But the fact that it is a known signal or shortcut our brains use means it’s also exploited by bad actors. And make no mistake, even very smart people, who do not have the right mental defenses in place, can also occasionally be taken in.

The Potemkin number concept as described in Charles Seife’s book Proofiness: the dark arts of mathematical deception, is the perfect descriptor of falsified or fake social proof.

What exactly are Potemkin numbers? The name is from a legend about Russian Prince Potemkin. The prince wanted to convince the empress that the Crimea was a vital, thriving area. So he built Potemkin villages, elaborate facades in the shape of villages and towns that appeared real from a distance. But they were fake and insubstantial. Potemkin numbers are the same thing—meaningless numbers designed to look real or authoritative.

As far as false social proof goes, I thought I’d list some of the items I’m inherently skeptical about. Please note, not all of these may be Potemkin numbers on their own, and of course not everyone is willfully ill intentioned, but read through the descriptions on how each could be gamed and think critically if a person or brand is legitimately deserving of your trust via social proof signals or not. It’s usually pretty easy to pull back the curtain with some digging.

Easily gamed (be skeptical until proven otherwise)

Number of followers in any single platform

You can automate the growth of these with tools or via outsourcing. Platform specific followings (probably the most classic of which are Twitter followers) used to be and still are extremely game-able and so many have. The major platforms seem unwilling or unable (am rolling my eyes as I typed that, they definitely could address) to do anything about this. Some sites have made it more difficult but I seriously question anyone who used automation to build a low quality following in the first place. That act itself signals you’re trying to bullshit your way through life and definitely hurts someone’s reputation more than helps. Nothing more embarrassing than someone with 200K followers and zero engagement on their ideas. They might as well hang a flag they’re a grifter.

“As seen in” citing known media brands

This one used to be more difficult, but with a sprawling tail of media brands and blogs, even popular sites like Forbes are allowing guest content from anyone. It wouldn’t be terribly difficult to fake this and in fact many do. If a person or brand is using this, look for legitimate editorial endorsements from trusted writers who understand how to conduct digital due diligence. This should be obvious, but you’d be surprised.

Influencer scores

The premise of an influence score is inherently flawed. It is usually wrapped in marketing language to make it sound “official” but it is no more or less silly than those old sites that tell you how much your website is worth. Or HubSpot’s very old attempt at giving your website or blog an SEO or social media “grade” (they “graded” many sites that mint $ poorly, as they ignore you can’t understand performance without analytics access …but of course they didn’t care, these ‘tools’ were just exercises in link generation for themselves).

An influence score or automated grade for a human being online just doesn’t work. It is attempting to take something that’s actually more qualitative and reduce it to quantitative, but not all data needs or should fit into quantitative. In a world obsessed with numbers it’s a brilliant grift, and I’m sure more Klout clones will be built that make lots of revenue being a score in marketing software, providing something quick and easy to add to reports. That’s what previous generation marketers want. But is it meaningful or actionable to any truly data-driven marketer focused on outcomes and conversions? Please.. my friend Alex has the math, but I don’t even think you need it.

Tougher to game (but still certainly possible)

Subscriber counters to encourage you to subscribe to sites through FOMO

To fake a large numbers via the chicklet counters like those in Aweber or Feedburner which shows off subscriber count is possible but would be more difficult. Besides, if you’re going to game this why even bother with the widget when you can make up a number of subscribers and show it as text.

Legitimate comments

Real comments from a community — the kind that are well thought out by intelligent readers — would be difficult to game. Comments can be used as part of a brand’s social proof about a product or even to show authenticity and quality of an author to sell a book. Do check links/profile bios and make sure comments are not just smart, but left by real people.

Alexa Rank, Compete.com / Quantcast traffic, any sampled web traffic data

OK, so these aren’t the most reliable metrics in the first place, but certainly you can use them as indicators to gauge if a company is successful. Note it’s possible to inflate these with social traffic (i.e., a brand that makes Reddit bait purely for traffic is not necessary trustworthy) or spammy ad buys so look for longer-term trends rather than spikes.

Really difficult to game (so frequently, but not always, better trust indicators)

Speaking at credible industry events over many years/decades

It would be pretty difficult for a single individual or brand to gain consistent speaking slots across major industry events in any category. Too many chances to be called out by attendees or organizers interested in keeping a high quality event. Now – that doesn’t necessarily mean a talking head is qualified to, say, provide consulting for you. But it does show they have put forth the effort to be credible at the industry level. Still vet these people’s actual talent and case studies as you should anyone if you intend to hire them.

Multiple, consistent citations by media (not just guest content)

Through serendipity of social or the power of search, one or two citations as a trusted source by media are certainly possible for anyone. But an ongoing, consistent list of quotes in a variety of publications (national / across verticals beyond their own) would be quite the feat to achieve without being a legitimately qualified individual behind a subject. Also goes for brands too. It’s unlikely even a high quality PR firm could fake such relationships long term: it would get noticed / called out.

Included/quoted in research from major analyst firms (like Altimeter, Forrester, Gartner, etc)

Having facilitated many briefings with analysts over the years for brands / employers and personally contributed to several research reports, I’ll say the best ones always do their homework. They’re in the business of having a high quality reputation so they’re unlikely to risk quoting charlatans as this is potentially a reputation management problem that could cost them significant business. I have mixed opinions on some of these analyst firm’s actual content and ideas, but at the very least they definitely are thoughtful who is included in reports as these brands live and die by their reputation (and including grifters would be an easy way to torch it - just never worth chancing).

Consistent, long-term maintained blog, podcast or other in depth publishing of focused ideas

I still think it’s worth it to have a blog, especially those in marketing or tech, but increasingly important for areas like finance too. Yes, be on other platforms too (we don’t live in a 1-thing world) but a blog is platform agnostic, plugs into the rest of the web (social, mobile and email too) and demonstrates a high degree of dedication.

Putting aside the mechanisms of why a blog is still a great platform for a second, the story my friend Eric Friedman shared prior to being hired by Union Square Ventures in NY illustrates why in a simple example:

One of the things that stands out to me the most during a pivotal second round interview I had at Union Square Ventures. I sat down with one of the partners Brad Burnham, and presented my resume. He told me to hang on to it and he just wanted to chat.

When I pressed him as to why, he responded with something I will never forget which went something like this; “You can work really hard on crafting a well written, organized, resume with bullet points of accomplishments – but you can’t fake 500 blog posts.”

I was struck by this because I had never thought of things this way before. He was more interested in how I viewed the world (and subsequently the companies in it) rather than my list of things I think went well. Furthermore, he had already done his own homework seeing my previous employment history on the web (on LinkedIn)

One note here: if you can accomplish the above in shorter form arenas, more power to you. The semantics on type of publishing platform are of course not as important as what you actually do with it.

Wrapping up

In some cases any of the numbers above could be true indicators of success or indication of trust. But everyone should be adept at seeing through obvious digital Potemkin numbers. It’s not that difficult and if you don’t have this ability chances are you’re going to be manipulated, or be publicly seen following/endorsing ideas from people the rest of the world knows are charlatans. Which from an optics perspective, you’d personally never wish to happen.