We hope you will like the new Relevance and find it more intuitive and actionable. You may want to check the ‘scores’ tab in the Analytics Suite where influencers are divided in four quadrants. These are now easier than ever to interpret because they allow you to cluster people in groups. For example, anyone in the ‘high Reach+Resonance / high Relevance’ quadrant would be considered “experts with a large audience”.

Stay tuned for more improvements this year. We are always working on improving the way we compute our metrics to make them sharper than ever.

If you have any questions or concerns on this Relevance change, please contact your Account Manager.

Want to be a part of changes like these? You could be! We are now hiring developers, check out more details here.

The post “No Silver Bullet to Measure Online Influence”, as well as Traackr’s presentation at Third Tuesday Measurement Matters Conference in Toronto, has generated interesting discussions both on- and off-line about the ‘right way’ to approach the challenge of measuring online influence.

In this post, I’d like to share the approach we have taken at TRAACKR that is somewhat of a departure from others in our space (that is influencer discovery and measurement).

If you’re in the business of measuring online influence, you’re faced with 2 very basic challenges:

1. Clearly define what you’re trying to measure: what is influence and how does it manifest itself in quantifiable ways? Brian Solis and team created a whole project just around framing online influence for PR professionals. You can imagine that if influence means different things to different people within PR, things get much more complicated as you expand into other industries…
2. Figure out how to measure it: each site and media type has its own way of measuring things, with limited cohesiveness. So you’re left trying to compare friends and likes on Facebook, reviewer ranks, feed subscribers and votes on Amazon, views, channel subscribers, comments, and favs on YouTube, etc.

Now bring these two propositions together: you don’t know what to measure or how to measure it. Encouraging, isn’t it?

Most have tackled this challenge by doing 3 things:

1. limiting their scope of study to a universe somewhat uniform and controlled (be it a site like Twitter, a type of online communication like videos or blogs, or a closed community)
2. making assumptions on what influence is and how it manifests itself in their controlled environment
3. building a mathematical model that, with the support of technology, measures ‘influence’ in this limited universe

Though the approach is appealing because it makes life easier to those doing the measuring, there is a glitch: if you reduce the size of what you study to make it manageable and more linear and then make a series of assumptions on what influence means so that your mathematical models and computation work, you have most likely stripped out any possible interesting insight from your results even before you even started processing data.

As an example of this, Science Magazine has been covering a research project to measure influence on Twitter conducted by 2 PhD candidates at HP Social Computing Lab, that concluded: 1- influence is not popularity (and by that they mean having the most followers doesn’t mean you’ll get the most retweets) 2- Mashable, CNN, ESPN, The Onion, and the BBC are all very influential Twitter feeds. Well ok but maybe you could tell me something I don’t know…

At TRAACKR, we have taken a very different approach: instead of limiting our universe, we have extended it; instead of dogmatically defining influence, we have built a process to progressively discover what drives influence.

Our discovery-driven technology solution is architected and built on the premise that our process, data set and scoring are all bound to evolve. Here are the few rules we applied to make this possible:

1. Harvest all online data we can put our hands on. Analyze the data, look for patterns and correlations, expose the data to the users and observe usage patterns. This approach allows us to lower the threshold to add a new data type or data source and start collecting information even before knowing how we’ll be able to use it.
2. Build a modular scoring system (ref. graph above) to fetch, normalize, and score influencer data. This approach is essential for us to be able to add new data sources, alter scoring computation, and even test new scoring algorithms without disrupting the overall scoring system itself.
3. Let our users determine what defines success for them (influencer engagement), correlate success patterns with our data collection and scoring, adapt scoring.  We do this in 2 ways: 1- letting users weigh in on the quality of the data presented 2- tracking success metrics per campaign. This feedback loop enables us to constantly improve our scoring methods.

This empirical, discovery-oriented approach to measuring online influence makes some of our mathematician friends uneasy (as they favor a more purist theory-driven model as described earlier). As we ask them for forgiveness, we remind them of the wise words of Jim Sterne, one of the godfathers of web analytics more recently turned to social media measurement: “online marketing has been suffering from a delusion of precision and an expectation of exactitude.”  Imposing scientific rigor on influence measurement results in stripping findings from most of their potential value. Traackr has focused on preserving and exposing the value of the influencer data while building a solid foundation to measure influence.

Welcome to the first post of our series on measuring online influence. As I mentioned in the prelude to this series, we have been working on this topic for quite some time now and have learned a few things along the way. We figured we’d share some of them here, starting with the set of principles that lays the foundation of our scoring system. Each of these principles have shaped our scoring system and, in many ways, our business.

1- A definition for online influence

It turns out the dictionary has a pretty good definition of influence:

Influence, unlike popularity, is action-driven. Someone’s level of influence is gauged by others’ actions and reactions.  It’s important to recognize that we all influence each other’s decisions all the time, as mutual influence is at the core of our social fabric. For the purpose of our work though, we have focused our attention on those who exert a disproportionate influence on others.

We have also limited our field of study to online influence. What we mean by that is we only process and score those participating in online conversations in some way.

2- People influence people

This simple statement may be the most fundamental decision we made when we first started the company: behind every blog post, tweet, video, there is a person. This person – not his or her channel – is the one influencing their audience, network, or community.

We built Traackr around the idea that we needed to find individuals, not bloggers, YouTubers, or Tweeters.

This principle came with a set of challenges and benefits. A couple of the challenges were finding the individuals behind multi-author channels and reconstructing their full digital footprint as well as normalizing data from very diverse sources. The benefits kicked in once we solved these challenges and the richness of the data collected made our analysis and scoring much more reliable, therefore improving our ability to deal with missing data points.

3-  Influence is always contextual

Context is everything when it comes to measuring someone’s influence. Context is in many ways a proxy for expertise and trust. Getting people to make decisions based on a third party’s judgment requires that they trust this third party. Facebook will tell you that your Facebook friends constitute your trust network. We challenge this notion. Our data shows that your social tie to a person is only one element (and not the most reliable) of trust. Expertise vetted by the community on a specific topic, aka context (or relevance), is a much stronger candidate for trust.

The fact that we only measure influence in context is one of the founding principles of Traackr’s scoring system: the better our users define the context of what they are trying to accomplish, the more accurate the results of our influencer search will be.

4-  Influence becomes accurate when measured over time

Our scoring system is set to predict future patterns of success (influence) using historical data to determine these patterns. As we gather more historical data over time and are able to track influencer scores over time for a specific search, our algorithms become smarter and better able to accurately measure future results.

Measuring influence over time has also taught us that influence around a specific issue or conversation is never static and keeps changing. So unless you are only interested in a topic at one point in time, it’s important to keep your pulse on your influencer list, as you’ll see people coming in and out of fashion week after week.

The necessity to track all the stats and content of influencers over time has led us to major technical architecture decisions towards the adoption of big table data technology. To give you a sense of the size of what we’re talking about here: we’re tracking over 10MB of data per influencer (avg. 1,000 posts, 35 different stats across 9 platforms).

5-   No one size fits all when it comes to influencers

I’m probably one of the few people out there who doesn’t worship Malcolm Gladwell (though I did read all of his books..) but I have to give him credit for the way he defined influencer archetypes in the Tipping Point: the salesman, maven, and connector are a very simple way to remind ourselves that influence is multi-dimensional. Attempts to measure influence that don’t recognize the diversity in the way people influence others is bound to fail.

Right now, we’re using 3 separate metrics to ‘triangulate’ our influence score. Reach (salesman), Resonance (connector), and Relevance (maven) represent our 3 scores. We’re not done though. There will be more dimensions we’ll add to our scoring system to further represent the richness of what we’re attempting to quantify.

These five principles constitute the foundations of our scoring system and even though they have each evolved and matured as we have become smarter, they have each been a fairly stable base for us to build upon.

So, let’s begin this conversation. What is your baseline for influence measurement?

My team has been nagging me for weeks (ok, maybe months) to write a post explaining our scoring methods. As I’m finally starting to write, I’m asking myself why I have been skillfully resisting this moment until now.

Several companies in our space have been very vocal lately claiming to have cracked the nut of online influence and have shared with their audience a mix of very high level information and a formula magically spitting out an influencer score. In some ways, I think this is what my sales and marketing team was expecting from me: share with the world Traackr’s silver bullet to calculate online influence. I may disappoint them today because it’s not what I’m about to do.

I have too much respect for our readers, clients, and anyone truly interested in our field of study to deceive them by sharing platitudes about influence, a seemingly complex algorithm, implicitly asking readers to trust that if we can agree on the words, surely the formula must be accurate…

Right?

At Traackr, we have probably done more data collection, crunching and analysis, work building and iterating on our algorithms, and more testing and refactoring of our assumptions than anyone out there. Yet, I’d be the first to say that our measure of online influence is still an approximation (though getting better as we collect more data and become smarter on how to process it).

So, here is the disappointing truth for those in search of a silver bullet to measure online influence: it doesn’t exist, there is no one formula that will reveal the secret of online influence. Our algorithms are multi-dimensional, they are complex and they keep evolving. The data that goes through the grinder (aka our scoring engine) is massive, noisy, and sometimes incomplete. Our scoring system takes charge of the filtering, normalization, and processing of this data. As our Director of Product, a former scientist at the European Synchrotron*, would attest; “there is nothing simple about our scoring system that can be summarized in a couple of sentences and a shiny formula.”

Just like for any top-notch technology (call it Google Search or the Mercedes SLS), our goal is to make the input and output of our technology so intuitive for our users that we can leave the complexity of the work to scientists and technologists.

Now, don’t get me wrong here; I’m not saying that we shouldn’t discuss Traackr’s scoring for online influence because it’s too complicated. I’m actually suggesting the exact opposite: I would like to initiate a series of posts and hopefully spark a conversation on measuring online influence. The topic is very rich and raises many questions. We have worked on this more than most and will be bringing our point of view on the issue, but want to welcome other perspectives, experiences, and reactions.

If this series generates interest from the community, we’ll expand the conversation and create a “geek corner” to discuss machine learning, adaptive algorithms, analysis of big table data, and other tools that science and technology have brought us to become smarter and more accurate in our measurements.

My plan to get this going is to start by writing a piece on the underlying principles behind Traackr’s scoring system to measure online influence. If you have topics in mind you’d like to be discussed in the context of this conversation, please share and we’ll see if we have something interesting to contribute.

* The European Synchrotron is a research institute conducting cutting-edge science on photons

Just when you thought you had mastered the subtleties of the Traackr score, we’re turning the tables on you and displaying the score as an absolute value rather than a percentile.

Here’s why.

The score helps you improve your performance by comparing your score to other users’ and see how you can emulate what they do; but also by looking at your score over time and seeing how your promotion experiments pan out.

This is where the score expressed as a percentile is not doing you a favor: because we have many new users signing up every day, everyone’s percentile score is impacted, just because new users automatically change percentile rankings. If there’s more people, it’s harder be in the highest percentiles.

As the user base stabilizes, we may bring back percentiles, because, after all, they’re good measure of great results.

Meanwhile, we will stick with absolute values. Tell us what you think…

The Traackr Team