It's what people do, not what they say

I've lived long enough to have intuitively learned that the obvious is worth repeating. Then, the other day, while recording a podcast for the

Product Guru's

(highly recommended if you understand Portuguese),

Paulo Chiodi

(the host) shared a great insight on how to think about it:

Every day new people join the market. That's why we need to keep reiterating the basic things that matter, despite many possibly already knowing it.

Such a great point! And here's me risking to state the obvious that many may know, while hoping it to be useful either as a reminder or a new insight for someone truly new to product development.

A lot is said about talking to users and stakeholders, and by all means we should do that. The litmus test, though, to validate whether something we provided is fit-for-purpose is not what they say, but what they do (i.e., how they behave).

If you have been working around software for some time, maybe this simplest of the stories will resonate with you. A team delivers something after thoughtful consideration of what it should do (with proper interview or whatever means of talking to potential users), to then realize people started using it in a sort of left field manner and eventually a (critical) bug is born. As the meme goes: it's rare, but it happens a lot!

A somewhat older story from the market might help to land the concept even better. Should Facebook, back in the day, have listened and acted upon to the immediate feedback after launching the timeline feed as we know and are used now, and we'd probably not have it today! That's right: the backlash was immense at the time, with people worried with their privacy being exposed (for the youngsters out there: the original implementation expected people to visit each other's profile to only then see what they were posting).

Here's a recent example from my own work context, a story I heard from one of the PMs in the team. He worked with the development team to better understand the usage of the different reports and dashboards the team has delivered over tine. All of them had a reasonable initial reason for being. They found out, though, that a bunch of reports and dashboards have not been accessed by anyone else than the development team itself over the last +6 months.

I can almost bet with you that should the development team reach out to users and stakeholders suggesting to decommission those unused reports and dashboards, and there will be at least someone (maybe more than one) that will provide feedback that they still intend to use it. Likely to then continue not doing so – it's human nature, we are creatures of habits and attached to what we have.

The fundamental insight is this: 

While working software, just like the agile manifesto (in the principles section) states, is the primary measure of progress, that's only half of the story. It's in fact when all truly starts, and software is finally in the hands of users. Now we can start monitoring behavior until we have enough insight on the success of what we have delivered.

Think about it this way (with an analogy): can a football team claim to be successful for often creating chances with attempt of goals, for being so good at building up plays? I think you'd tell me that's not enough, and that only when actual goals are scored and matches are won they can truly claim success.

The same is true for product development. And just like in sports, a real friend of ours should be to develop a deep analytical understanding of what we do.

• Building-in measurability: because it's easier to get that in from the start than retrofit it.

• Defining, measuring and monitoring metrics: preferably those that are powerful ideas, despite the challenges in measuring them, rather than sticking to what we can measure perfectly.

• Taking actions: just as how we got here, if what we measure doesn't become actionable and actioned, we'd be again falling into the original trap of going with what we say rather than we (should) do.

On the theme of reiterating (the obvious), stick to this: the actual measure of success is what people do, not what they say. A simple and powerful idea which (from my experience and observation) gets too often sort of "dumbed down" to user/stakeholder feedback (e.g, "yes, that seems like what we requested" kind of thing…).

By Rodrigo Sperb, feel free to connect, I'm happy to engage and interact. If I can be of further utility to you or your organization in getting better at working with product development, I am available for part-time advisory, consulting or contract-based engagements.