Pilot Project Critical Success Factors

Originally published 9/29/2010

Over the last few blog entries I’ve written a lot about how *not* to do things.  This week I thought I would share some advice I’ve given in the past on how to do a successful pilot project.

It happens with any new technology: after some early adopters report some initial success, organizations often want to dip their toe in the water by implementing some sort of pilot project using said technology. (Someone once observed that every organization wants to be the absolute first to implement a proven technology.)  As with most things, there is a right way and a wrong way to approach a pilot project—often called a “proof of concept.”  Back in the day when I taught software development methods, these are some of the hints we used to give people on critical success factors when considering a pilot project.

Criteria One: Pick a project that is large enough to be interesting.  And by interesting I mean have economic benefit. Your pilot project can’t be something that executives wouldn’t notice even if you complete it: the pilot should have some kind of measurable impact on the bottom line, either by increasing revenue, decreasing costs, or some combination of the two. Forget “intangible” benefits: make them tangible and measurable.  If the end result of the pilot is a new system that’s supposed to make customers happy (intangible) then it should also make them buy more (increase revenue), churn less often (reduce costs), recommend your company to friends (increase revenue) or in some other measurable way make an economic impact to your organization.  Bottom line: If no one notices that your pilot was implemented, it’s hard to declare success.

Criteria Two: Pick a project that is small enough to be doable. “Build the next Facebook” may be your ultimate goal, but it’s not a good pilot (not even for Facebook).  Again, the reason is somewhat self-evident: the biggest, meanest, ugliest and messiest projects are the ones that are more likely to fail—for a variety of reasons.  Small projects similar in nature and size to those your team has done before are more likely to be successful. At the very least, they are easier for team members to intellectually justify trying a new technology on: if at any point the new technology proves to be unusable, the team knows it can fall back on methods used on prior projects and still get the job done.

Criteria Three: Pick a project that can be a stepping stone to a larger project.  If the pilot project is a “one-off” that has no relationship to other more mission critical projects, even a completed implementation may not ensure that the pilot is successful (if the end goal of the pilot is to not only get the project done but prove the value of the new technology).  The best pilots are those that might be considered “phase one” of a multi-phase mission-critical system, rather than some stand-alone project that can’t be leveraged in the future. They can have value as a stand-alone deliverable, but they might have even more value as part of a larger and more comprehensive system.

Criteria Four: Get help. We don’t train craftsmen by handing them a book and telling them to go build a project.  All mature tradecrafts have the notion of some kind of apprenticeship program, where people skilled in the trade can watch over newcomers and take corrective action when necessary to keep the project from being an utter disaster.  The same is true with new technologies like social media. Organizations that go buy a COTS or open-source social media package and try to implement it themselves *might* be successful, while organizations that work with a knowledgeable partner increase their chances of success by a wide margin.

Project Titanic

Originally posted 9/22/2010

These days I work with a bunch of very smart people that know a lot about how to implement social media solutions (you can even chat with some of them here).  But back when dirt was new, I used teach software development methods and project management. My old friend and colleague at the time Ken Orr had observed that a lot of failed projects had similar profiles, and came up with a template for failure he called “Project Titanic.” We used it at the time to illustrate how *not* to do projects. I dig it out every now and then to see how far we’ve come as an industry since the mid 1970’s.  Judge for yourself.

Step 1: Assign a due date and a budget. The two things you know the least about; actually it turns out this is an important step.  It announces to all involved with the project the exact time and place when failure is to occur.

Step 2: Assign the project an acronym. Either three or four letters; should include one of the following words: “Management”, “Information”, “System”

Step 3: Select a Project manager. A classic catch-22: anyone who applies for the job is instantly disqualified, since anyone who has ever been the project manager on this type of project is smart enough not to apply.

Step 4: Develop a Requirements Document. You know: the 10 pound requirements specification developed after talking to the user about their requirements for a few hours.  Quantity is important.  Use lots of graphs.

Step 5: Get User Approval. Actually this isn’t too hard.  If the requirements document has used up 60% of the budget, the user community is unwilling to stop the project without having anything to show for it.

Step 6: Hire Systems Analysts.  Someone needs to do the following work.

Step 7: Develop the Input Forms. It’s important to do this early, otherwise the three-part perforated color forms won’t be back from the printers in time for the projected start date.

Step 8: Hire Programmers. Someone else needs to do some real work.

Step 9: Develop Edit Programs. Results in a minor feedback loop since it inevitably results in changes to the input forms.  Hope that the printer hasn’t gotten too far along with the job.

Step 10: Develop Conversion Programs. After all, we wouldn’t want that old bad data to just reside in the old system.

Step 11: Develop Data Files. You have to have some place to store the old bad data.

Step 12: Develop Update Programs. You have to be able to add new bad data at some point.

Step 13: Hire More Programmers. It’s usually around this time in the project that people realize that things aren’t going well.  The project is obviously behind, but no one wants to admit it, so we hire more programmers.  Now if programming is like digging a ditch, four hundred people can get a ditch dug faster than four people; unfortunately programming is more like digging a well: four hundred people can’t dig deeper faster than four people, you’ll just have people stepping all over each other and the well will be a lot bigger around when you are done.

Step 14: Begin Work on Outputs. Finally someone begins to ask embarrassing questions like: what is someone going to want to do with this system.  Unfortunately the answer usually invalidates all the input forms, edit programs, update programs, data files, and requirements specifications.  Now is a good time for all members of the project team to update their resume.

Step 15: Slip the Schedule. Call the user back.  “Remember the system that was supposed to be delivered in January?  Well, it’s going to be more like June….of next year.”

Step 16: De-Commit. Call the user back.  “Well, Okay, we’ll meet the revised June date, but the system won’t quite have all the features we originally talked about.  The new payroll systems won’t be able to actually print checks for six months.  If it’s all that important perhaps we can throw something together to print them off in hexadecimal for the interim.”

Step 17: Panic. The fan and the excrement at last collide.

Step 18: Search for the Guilty. This goes without saying. Someone must be responsible for this mess.

Step 19: Punish the Innocent. The classic audit function: come in after a battle and shoot the survivors.

Step 20: Promote the Uninvolved. Actually not a bad idea; if the person was smart enough to stay away from the project, they might do a better job next time. 

And since this is an “unstructured” project, the last step is of course...

Step 21: Go to Step 1.

How NOT to Do Social Networking

Originally posted 9/16/2010

Early in my career a computer hardware vendor (who shall remain nameless) tried to get me fired after a seminar I had conducted.  These are the true events of this story: My company had arranged a technical education day for small and medium businesses.  In the morning, a session conducted by a colleague focused on computer hardware—what computers were good for, what computers could do, what type of things to look for in a computer (this was before the PC, when computers were the size of desks and were only owned by businesses), and so on.  During the lunch hour we had arranged for several hardware vendors to display their computers.  Attendees could walk around, see and touch an actual computer (wow!) and speak to the various hardware reps.  In the afternoon my session focused on software, and my message was pretty simple: I told people that it was most important to focus first on what you wanted to *do* with the computer.  Once you knew what you were going to use it for, go out and find the best software available for that application and buy whatever computer it ran on (this was back in the day when most software was only available on certain proprietary systems). 

You can probably guess where this is heading: One of the hardware vendors called my boss the next day, complaining that I had trashed his hardware company (I hadn’t) and that he’d had five computer orders cancelled after the seminar because of what I had said. Fortunately, my boss had been at the seminar and knew the real story. He knew that the orders had been cancelled because no one at those organizations had ever thought about what the computer was going to be used for.

Fast forward a few years. 

In the mid 90’s, I was managing application development for one of the major telecommunication providers in the US (who shall also remain nameless).  This was about the time the Internet was just beginning to sweep through the halls of industry as “the next big thing.”  At one meeting, the head of our division issued a mandate to--and I quote--“do something with the Internet.”  Well, with direction like that it’s no surprise that the only thing we were able to do with the Internet was fail!

Fast forward to today. 

Social networking is now the new “next big thing” for many organizations.  Executives are familiar with Facebook and Twitter (at least enough to ban use of them from company computers) so certainly there must be some opportunities there.  So more and more CI’s and IT department heads are being told to “go do something with social networking.”  Sound familiar?

Unfortunately this leads us right back where we started with this post.  Many enterprise social networking initiatives are doomed to failure almost from the word, “Go” because companies go out and invest in a social networking solution without giving a lot of thought to how they are going to use it.  Do you have specific, measurable goals set for your social networking strategy? As the old saying goes, “If you don’t know where you are going, it’s going to be hard to tell when you get there.”

Social Networking Security

Originally posted 9/9/2010

On the internet, no one knows you’re a dog.  That famous New Yorker cartoon by Peter Steiner from 1993 illustrates a fundamental issue faced by today’s social networking sites: are the members who they say they are?

We’ve learned a lot about participation in social networks via the internet over the last few years. We know from the good old days of altnet newsgroups (don’t worry if you’re too young to remember those) that anonymous participation in social networking sites is always problematic.  People post under multiple identities, they post under false identities (which is why Twitter has a “verified” flag for celebrities), and they can be incredibly disruptive by posting argumentative, obnoxious, inflammatory, obscene and generally vile remarks.  After all, when you’re anonymous, no one knows you’re “that” dog.

One solution to the problem is to try to tie a member identity in the community to an actual person.  Most sites accomplish this by requiring unique user names tied to an email address: that way when you sign up, they can at least send you an email to confirm that you are a real person and that you own that particular email address.  Of course, this doesn’t prevent people from signing up with multiple email addresses, but it helps.  By making members identifiable (rather than anonymous), most of the problems mentioned above go away (or the user is rapidly ejected from the community for being a jerk).  If a social networking site doesn’t require an email address for registration, or if they don’t make new members verify it by clicking on a link in an email sent to them, the site runs the real risk of degenerating into a de facto anonymous network. 

The flip side of requiring a unique user name and a valid email address is that it limits member participation.  Someone using a work computer, for instance, has a problem joining with their personal email address, since their personal email may be unavailable on their work computer (or vice versa).  Or people may fear that the social networking site is collecting their email addresses for nefarious purposes like spam or phishing, and refuse to provide it (and thus be barred from participating in the network). 

This is the classic tension found in systems requiring security: The ultimate secure system lets no one in, assuming that everyone poses a threat to the system; an open system lets everyone in assuming they have a right to be there.  In the real world, security and openness is a balancing act: Make security too hard and lots of people that you want to participate won’t; make security too simple and lots of people that you don’t want to participate will.

Digg and Reddit’s Excellent Adventure

Originally posted 9/1/2010

Popular social networking sites Digg and Reddit had some interesting things happen to them over the last few days.  And if nothing else, their experiences clearly illustrate that social networks are inherently unpredictable.

Let’s begin with Reddit.  Reddit was approached by supporters of California’s Prop 19 (a November 2010 ballot proposition to legalize marijuana in the state) to run ads supporting the proposition.  Reddit’s parent organization, Condé-Nast (who also owns several traditional magazines like Vogue, Glamour, and Vanity Fair as well as Wired, Ars Technica and Reddit) told them in no uncertain terms that they could not accept Prop 19 revenue.  End of story, right?  Not exactly.  Within hours, Reddit members had voted several different links for Prop 19 ads to Reddit’s front page, for which Condé-Nast receives no ad revenue (ok, that was probably predictable).  Reddit administrators announced that since Condé-Nast wouldn’t allow revenue from Prop 19 ads on the site, they were going to be running Prop 19 ads for free, and did so for several hours before they were apparently removed.  Reddit members had worked themselves into such a lather that Ben Huh, owner of FAIL blog and I Can Has Cheezburger among other sites, made a public offer to buy Reddit from Condé-Nast.  Where this all ends has yet to be decided, but for now the ads are gone from the site.

Meanwhile around the same time, Digg rolled out its much anticipated Version 4 user interface.  The result was a resounding thud.  Many Digg users were unimpressed, and spent several days complaining loudly and often that the new interface seemed to favor corporate news stories rather than user-generated ones.  After a couple of days, Digg members had apparently figured out enough about how the new page rating algorithm worked to game the site into displaying a large number of links on Digg’s home page from “corporate” news aggregator…Reddit (ok, that was also probably pretty predictable).  As of this writing, Digg was apparently contemplating a roll back of the user interface to the prior version, although it was possible they would stick with the new release and tune it more to the member’s liking.  In the meantime, Digg’s front page was (as of this post) still dominated by links to Reddit and its corporate sister site Ars Technica.

I’m waiting with baited breath for round 2 of this match, when Digg decides to start running Prop 19 ads.

Fish, Phones and Crowdsourcing

Originally posted 8/26/2010

I began my career many years ago developing and teaching software system development methods.  One of the principles my colleagues and I stressed is still valid today in the context of social networking but often ignored: data is only as good as it’s most frequent usage.

Why does frequency of use effect data quality?  Data that is frequently used is apt to be good data, since people are looking at it regularly and using it for some meaningful real-world purpose:  If an error is present in the data to start with, or if the data changes, someone notices it quickly and it gets corrected.  Data that is infrequently used is typically of lower quality for the same reason: It isn’t used as often, so when errors or changes occur they take longer to notice and fix.  Consider phone numbers: if you build a web site and require visitors to enter phone numbers, how many of them are valid to start with? How many initially valid ones are still be good after a year? After five years? Unless you are regularly using those numbers to call people, the quality declines with time.

Social networking sites often seem to forget that data, like fish, can go bad.  And I’m not referring to the fact that lots of sites try to collect profile information that will never be used (like asking for my phone number when you have no intention of ever calling me).  I’m referring more to crowdsourcing: social networking sites set up to collect user generated content (UGC) intended to help solve some kind of problem.  While some UGC is long-lived (“how do I turn on/off a setting in a popular application,” for instance), much UGC has a much shorter shelf life (“where’s the cheapest place to buy gasoline this week,” for instance).  Everyone has had experience wandering into a forum, discussion group, or blog where there has been no activity for many, many months.  And if you are like me, when I wander into such I site I tend to wander right back out again. 

Like stale fish in your refrigerator, stale UGC on your social networking site can spoil other sections by driving away visitors. So a word of advice to organizations setting up crowdsourcing applications: keep the freshness of data in mind.  Throw out new interesting topics frequently to keep content fresh and retire old topics when they’ve stopped attracting new content.  Move old content to archives so that users don’t accidentally confuse fresh content with older content.

And while you are at it, stop asking for my phone number.

Life Events and Social Networking

Originally published 8/18/2010

Google’s CEO Eric Schmidt has some interesting ideas on privacy and social networking.  Noting that people (especially young people) have a habit of posting private or potentially embarrassing information on public social networking sites, he predicted that “coming of age” in the future might have to include the creation of a new online persona to distance oneself from youthful online indiscretions. If that’s true, I wonder why would we limit reinventions of one’s online self to just a single coming of age life event.  What about getting married: should the bride and the groom be able to change their names to disavow social networking posts made when they were single?  Should divorcing couples be allowed to change their names to distance their online personas from when they were married?  Should parents be allowed to change their names after they have children? What about switching political parties? Moving to a new city? And so on and so on.

How would such a trend apply to our professional personas?  Increasingly, employees are asked to join social networks as online representatives for their company. If I post positive reviews of my current employer’s products and services, for instance, won’t I need a new persona if I change careers and go to work for a former competitor?  Or if I post a competitive analysis casting another company’s product offerings in a negative light, won’t I need a new name if my company and that company merge?  As it stands today, you certainly don’t have to look far to find lots of examples of statements made by corporate executives that they’ve come to regret after their business environment changes.

I doubt seriously if creating new online personas for each different life event becomes formalized anytime soon (informally, of course, people do it all the time with email and user name aliases).  Instead, I expect governments to get involved soon (some already have) to legislate that when a person leaves a social network, the owner of the network must irrevocably delete all content that person created while a member.  It remains to be seen how those privacy laws will effect 3^rd party content aggregators and search engines.

As online privacy laws stand now, it should give people pause when participating in social networks. It should also be a big consideration for organizations thinking about getting into social networking. How much involvement can you expect of members on public social networking sites when it finally sinks in that anything they say “can and will be used against them” in the future?