Monday, February 8th, 2010

Quality, a User Problem

By Chris Dary

Bobby had just created something to be proud of, a real article of creativity. He had been working on his song for days, hunched over guitars, computers and microphones. A laborious process, but after he had his result, he was satisfied. It was an artifact worth the effort he had put into it.

The only thing left to do was to get it out to the world. In the music industry this would probably be more painful than creating the music itself, but luckily for Bobby he wasn’t in it for the money – so he does what any other modern musician would do nowadays. He posts it on social networks. Namely, Facebook.

Bobby's Status Update on Facebook

After submitting it (and breathing a sigh of relief), he moves on to browse elsewhere for awhile. In the back of his mind, though, is the constant thought – are people listening? Are people enjoying my work? Creativity is largely a social act, and Bobby is in the final stage of it—validation.

He checks back every few minutes, and watches his update begin its inevitable trudging down the feed wall into oblivion. His work is slowly overtaken with gems such as:

A lame, anonymized status update on Facebook.

This is a tragedy. And Facebook knows it.

The Problem of Worth

The problem here is simple: the gap between the quality of these two posts is very high. One of them is a genuine article of effort – someone truly creating something novel and sharing it with the world (whether your tastes align with the content or not). The other is half-hearted mumblings at best. Even the authors themselves wouldn’t deny the difference in quality.

Facebook has acknowledged this problem recently, with their introduction of the “News Feed” versus the “Live Feed”. The News Feed is meant to be a subset of the Live Feed – the things that are “important”, an algorithm primarily based on number of comments and likes. Both the implementation and the naming are a bit clumsy, but the effort is noted: Facebook understands the noise problem, and that it has only been exacerbated with the prevalence of third-party application notifications. They’ve tried to solve it, but haven’t really nailed a way to determine the quality of a post.

Quality is a User Problem

“Quality”, in this case, is a very nebulous concept. It’s not really measurable in machine terms – this makes it a particularly hard problem to solve by filtering. “Likes” are not exactly newsworthiness, for example. You could even make the argument that this enters the realm of Strong AI—machine intelligence that matches or exceeds a human’s—due to its requirement of very human characteristics like taste and emotion. And Strong AI is a very long way off from reality. This is simply not a problem made for machines to solve.

So if it’s not a machine problem, it’s a user problem. That means we have to rely on the authors, and their readers, to figure out the quality of their posts for us.

One way to implement this is below – a simple “update volume” slider, that asks the user “How loudly do you want to say this thing?”

A mockup of 'status volume' when posting a new update in Facebook.

Depending on the author’s choice (and a few other factors like frequency of posts), the update would be displayed more or less prominently on their readers’ update streams.

The authors themselves aren’t ignorant; by and large they know the worth of their own submissions. Putting the problem into their own hands to solve may seem a bit strange at first, but it provides us with a very valuable data point to begin from: Are your words worth anything?

This is the hard part; this is the user problem. From this bit of data, we can begin to solve the simpler problems programmatically: Is a user being arrogant and loud by posting high-volume content too often? Automatically scale their volume based on post frequency. Are application notifications annoying, but still potentially useful for some users? Set them as the lowest volume by default, but allow a reader to specify individual modifiers on specific Applications. These are all solvable problems once we’ve pushed the intractable one of Quality into the authors’ hands. This single data point provides us a lot of ground to stand on when making otherwise-difficult decisions about the worth of a post.

I’d be interested to see how this would play out in the Facebook ecosystem. It seems a particularly more noisy system than other social streams like twitter, due to third party applications posting junk. In the end, we’ll either need better quality control, or fewer friends.

14 Responses

  1. 2/8/2010
    Justin Jackson Said:

    An interesting idea. I am guilty of mindless updates, but I am also a producer of published “gems” that I would gladly “turn up the volume” on.

  2. 2/8/2010
    Mike Grace Said:

    Interesting and well thought out thoughts on this problem. I like the approach of having the author of the content help solve the problem and react accordingly if a particular user is misusing the tool to solve the problem. We could aso have the consumers validate the volume choice of the author and have checking built in that way also. I have been thinking about a similar problem with Twitter and looking to implement a solution using a Kynetx application to help solve it using consumer side validation and ranking. Look forward to more posts from you.

  3. 2/8/2010
    Apreche Said:

    The problem is real, and does need a solution. However, I believe that your solution will obviously not work. Look at how people use priorities on e-mail at work. Everything is urgent, even when it’s not. People are not a very good judge of how important their own things are. They made it, so it’s always incredibly important to them.

  4. 2/8/2010
    Marty Said:

    IMHO, Volume isn’t really the right metaphor.

    Rather than a quantitative mechanism, we need broader qualitative choices than “post” to describe what we are trying to do: speak. inquire, share, show, etc.

    Also, we need better tools to self-regulate our “friends”, not technology to to guess on our behalf.

  5. 2/8/2010
    Chris Dary Said:

    @Apreche – of course people will do that. People are arrogant. But this is a tractable problem, as I mention above. If they are being too noisy, you can easily drop them down to the lowest level volume no matter what they specify. Essentially, they are the Boy Who Cried Wolf. Treat them as such.

    @Marty: Interesting, but I think a different discussion. Whether it’d be useful from a reader perspective to know whether a user is trying to “show” or “ask” or “inquire” is arguable – shouldn’t the content itself explain that? Also, regarding self-regulation/segmentation of friends, I sure hope Facebook is hard at work at that – it’s sorely needed, and something that occurs in real life all the time. It’s a no-brainer.

  6. 2/8/2010
    Levi Said:

    This is ALWAYS a problem in digital media, including email due to the “linear-ness” of how the information is presented currently. Important messages get buried in spam. I have seen solutions with this where priority is used as a currency; for example, there is a finite amount of “urgency” doled out to users and if they feel they are sending a message of high worth, they can assign all of their “urgency currency” to it in hopes that it rises to the top.

    But then again I have a bunch of douchey friends and twitter folks I have had to unfollow and un-friend because they are always whining or inviting me to play farmville.

  7. 2/8/2010
    Tim Meaney Said:

    One facet of this problem is that the answer of how loud do I, as the author, want a post to be is contextually related to the receiver – a post on my lost voice might be really important to me, fairly important to my coworkers that are depending on me for a meeting tomorrow, and really important to my mom (since she cares how I’m feeling), but total noise to others.

    So ‘how loud do you want to be?’ might vary by receiver.

  8. 2/9/2010
    Scott Wiersdorf Said:

    The real problem is that people are terrible at gauging their own competency:

    http://apa.org/journals/features/psp7761121.pdf

    So any solution to detecting quality has to consider the audience:

    http://www.paulgraham.com/goodart.html

    Which makes me think that I as a consumer of information should be able to turn up/down the volume on the feeds I receive, not the other way around. If FB could aggregate *that* kind of data, we’d have a much better overall picture of who produces the quality posts, and other members would have some incentive to do a little more thinking before they hit ‘Post.’

  9. 2/9/2010
    Chris Dary Said:

    Scott/Tim, that’s the other side to this coin, and I certainly agree – the reader having more fine grained controls on each user would be very useful. I think both methods could be used in tandem to good effect.

  10. 2/9/2010
    Rich Ziade Said:

    I actually had a similar thought to this in the context of Twitter. It’s detailed out here:

    http://www.basement.org/2009/03/twitter_observation_2_the_boy.html

    This is one of those things that you just sort of throw out there and see what social patterns/norms materialize. Who knew that yellow lights would make people go faster?

  11. 2/9/2010
    Michael Rehse Said:

    There is also the problem about people being too quiet – those people who create things that are wonderful but have no confidence in them.

    Granted, if they are sharing them online, this probably isn’t an issue. However, the problem of “too soft” still exists as well.

    This could be partially addressed by reader “volume,” as has been recently discussed is this thread – those people who are naturally “softer” and don’t want to be “rude” by being “too loud” will have their volume increased by those people who want to listen similarly to how volumes would be decreased on the loudmouths (Yes, there are far too many quotation marks in that sentence).

  12. 2/9/2010
    April Said:

    This is a good idea… It would be great if the user receiving the feed could adjust volume of posters. Just as some think it’s fine to share trivial info, others will think it’s fine to share trivial info at deafening volumes. Volume controls on both sides (poster: posts, reader: repeat-offender posters) would be helpful.

  13. 2/9/2010
    Bobby Ziade Said:

    I like the idea of a volume knob, on both ends (poster and reader). Ever the tween shall meet. It was really disheartening to put it up there and have it farted on by Farmville updates and whatever nonsense musings a depressed half-friend who can’t find a job or sleep well has to say. As an aspiring-real-world object-designer, musician, etc – I find myself *staring* at Facebook and Twitter more than I use them.

  14. 2/10/2010
    Marty Said:

    >>”whether a user is trying to “show” or “ask” or “inquire” is arguable – shouldn’t the content itself explain that?”

    If content could explain itself, we wouldn’t really have this problem would we? The content /doesn’t/ explain itself because the content is based on a technical metaphor of a ‘post’ – meaning some arbitrary string of characters.

    Assigning an arbitrary number (volume) to those string of characters to represent a self-assessment of importance seems crude to me. But maybe this would help as part of a general improvement.

    I think we need to provide distinct types of messaging. One example I thought of (and there are not many) is in email we have both the “message” and the “calendar” invite.

Leave a Comment