Survey Results, Q2-3: Interest and Usefulness of PS Renewal-related Websites

There was an unusually high level of traffic on my blog yesterday, so I guess the first results of my little survey are drawing some attention. Not a single visitor has posted any comment though, which is ironically a good lead-in for this posting.

Today I am presenting the results to a few more questions from the survey, more specifically:

Q2. What is the level of usefulness of the following websites? Q3. What is your level of interest in the following websites?

1. The PS Renewal page on the Canada Public Service Agency intranet site
2. The site An Inconvenient Renewal
3. The group Bottom-Up Renewal
4. The blog Contrarian Thinking
5. The blog CPSRenewal.ca
6. The Facebook group on PS Renewal

The results are as follows (click on image to enlarge):



The good news is that respondents rated 5 of the 6 sites just above “medium” for both usefulness and level of interest (the only exception being the Facebook group on PS Renewal).

Something puzzles me though. While 5 of the sites' usefulness and interest comes at least in part from their interactive features (comments, input from users, or in some cases capacity to share links or attach documents), the level of participation on the majority of these sites is almost anaemic:

• The site An Inconvenient Renewal has received no comment whatsoever;
• The group Bottom-Up Renewal has received a single entry apart from my own (which not surprisingly comes from the guys at CPSRenewal.ca);
• The blog Contrarian Thinking and CPSRenewal.ca have shown some moderate signs of activity from readers (comparable to the Facebook group on PS Renewal), but the majority of comments come from a handful of users.

Part of the explanation undoubtedly lies in the inequality factor (thanks Nick for the link!), which states that “user participation often more or less follows a 90-9-1 rule:

• 90% of users are lurkers (i.e., read or observe, but don't contribute);
• 9% of users contribute from time to time, but other priorities dominate their time;
• 1% of users participate a lot and account for most contributions: it can seem as if they don't have lives because they often post just minutes after whatever event they're commenting on occurs.”

On my blog, I have received approximately 3200 visitors in the past 12 months, or 267 visitors per month. It is not too bad given that I have averaged just over one post per month in the last year (by comparison, CPSRenewal.ca practically features a new post every day and a column every week!). But I have also averaged just over 1 comment per post, when according to the 90-9-1 rule I should have expected 15 to 20 times. So there must be something else…

I am raising this issue because the implications are broader than you might think. As you may know, I do quite a lot of work in staffing, and one of the comments I hear the most often coming from managers is that they want “more tools and more opportunities to share best practices” around staffing and the new PSEA. Now here we are with PS Renewal, hearing similar demands, and for once the tools are available and the opportunities to share “best practices” are better than they ever were. But no one is making use of the tools, and few people are sharing anything. Why is that?

Could it be that the repeated demands for more tools and opportunities to share best practices are in fact “cop out”? A thinly veiled excuse for not taking responsibility for the change, yet not being blamed for the lack of progress?

I can already see employees pointing the finger at managers, managers pointing the finger at senior executives, and senior executive pointing the finger at central agencies. “It's not our fault if we are not getting positive results with regards to {insert problem of the day here, i.e. PSEA, staffing, recruitment, succession planning, PS Renewal, etc.}! We need more tools! YOU (whoever this may be) must give us tools like websites where we can access information, repositories for sharing best practices with other Departments, and discussion forums to encourage dialogue!”...

Hum...

Troubling isn't it?...

And perhaps a little painful...

Let's get back on track before denial kicks in...

So far, I have only been able to come up with a few other explanations for the low participation rate:

• Restricted access to these sites from work;
• Equivalent or better sites available behind departmental firewalls;
• Limited time to comment and provide input;
• Discomfort or lack of familiarity with the medium and/or the technology;
• Language barrier (i.e. sites in English only);
• Fear of getting in trouble;
• Learned helplessness.

If you can think of other reasons why participation is so low – especially when we keep hearing that people want more opportunities to share ideas and best practices – please comment below. I will use your input for my next poll on this blog.

Tomorrow... Well, tomorrow is a special day. I'm saying no more, it's a surprise!

Survey Results - Q1: Access to Web 2.0 and PS Renewal-related Websites

The first question of the survey was inspired by an idea from New Zealand’s public service blogger Jason Ryan in a posting where he referred to the Stop Blocking! campaign. I instantly felt an urge to rate the Public Service of Canada’s Departments and Agencies, but the undertaking was a little ambitious. So I compromised for a single question that would provide an imperfect but very telling picture of the current state of site blocking in the public service.

The question was: “Which of the following websites are you able to access from your work computer? Select all that apply:

1. The PS Renewal page on the Canada Public Service Agency internet site
2. The PS Renewal discussion guide on the Canada Public Service Agency intranet site
3. The site An Inconvenient Renewal
4. The group Bottom-Up Renewal
5. The blog Contrarian Thinking
6. The blog CPSRenewal.ca
7. The Facebook group on PS Renewal (or just the Facebook home page if you don’t have a Facebook account)
8. Prime Minister Stephen Harper’s channel on YouTube
9. Randy Pausch's Last Lecture on YouTube
10. The "Canadian Civil Service" page on Wikipedia
11. The Prime Minister's photo gallery on Flickr
12. The "Emails, Emails and More Emails" presentation showcased at the Canada School of Public Service
13. Business Week's video on "being entrepreneurial"
14. Ottawa Citizen's reporter Kathryn May's articles on the federal public service and PS Renewal-related initiatives
15. The 2007 Manion Lecture audio recording.”

My intent with this question was to get an sense of just how many public servants have access to PS Renewal-related Web resources. While some of the links above are not directly related to PS Renewal, their content either lends itself well to the spirit of PS Renewal, or constitutes information that you would expect all employees to be able to access in an organization operating on principles of trust, transparency, and openness.

The selection of some of the sites was also governed by their popularity. As of October 31, 2008, Alexa’s ranking of the top websites in Canada included: Facebook (#3), Youtube (#5), Wikipedia (#9), Blogger (#10), and Flickr (#19). The Government of Canada held the 11th spot.

I was specifically interested in knowing what percentage of public servants didn't have access to those sites and resources. The methodology I used to compile the results was the following. By asking people which of the sites listed they could access from their work computer, I was able to determine which ones they couldn't access. I then compared the answers of respondents from the same departments to see if their responses were consistent. When there were inconsistencies, such as it can be expected in a large decentralized departments (e.g. Service Canada) where different offices may impose different access restrictions to websites, I applied the following rule: in order to conclude that the employees of the department don't have access to a specific website, all the respondents from that department must have no access to that website”. This logic obviously skews the results in favour of the sites that are accessible (the more respondents from a given Department, the greater the odds of concluding the site is accessible), however it gives me a greater degree of confidence with the claims I make later when I say that a certain percentage of the public service doesn't have access to a given website. For instance, let’s say I have 4 respondents from the same department: 3 say they can’t access a given site, 1 says he can. I will conclude that the employees in this department can access the site, even though in reality 75% of them may not have access. By applying this rule, I may be embellishing the actual level of accessibility to certain sites, but at least it reduces the odds of exaggerating the level of inaccessibility to those sites.

The table below presents three sets of data:

1. The 34 Departments and Agencies represented in survey (which accounted for 85,4% of all public servants);
2. The 87 respondents;
3. An extrapolation of those results to the whole Core Public Administration (i.e. 107 Departments and Agencies totaling 263,118 public servants), factoring the size of each Department and Agency.

Percentage of federal public servants who can't access PS Renewal-related websites from their work computer

Sites	34 Departments and Agencies Represented in Survey	87 Respondents	Entire “Core Public Administration” (Extrapolation)
The Facebook group on PS Renewal (or just the Facebook home page)	50%	61%	73,50%
Business Week's video on "being entrepreneurial"	35%	53%	60,80%
Randy Pausch's Last Lecture on YouTube	41%	59%	60,60%
Prime Minister Stephen Harper’s channel on YouTube	38%	56%	55,30%
The "Emails, Emails and More Emails" presentation showcased at the Canada School of Public Service	35%	57%	54,70%
The Prime Minister's photo gallery on Flickr	32%	48%	46,90%
The blog Contrarian Thinking	24%	34%	43,00%
The 2007 Manion Lecture audio recording.”	15%	38%	18,10%
The blog CPSRenewal.ca	12%	25%	17,90%
The group Bottom-Up Renewal	12%	21%	4,70%
Ottawa Citizen's reporter Kathryn May's articles on the federal public service and PS Renewal-related initiatives	9%	22%	1,40%
The PS Renewal discussion guide on the Canada Public Service Agency intranet site	6%	15%	0,90%
The "Canadian Civil Service" page on Wikipedia	6%	17%	0,90%
The PS Renewal page on the Canada Public Service Agency internet site	3%	6%	0,70%
The site An Inconvenient Renewal	0%	14%	0,00%

I don't want to discuss the results too much – in fact I hope that the readers will discuss them using the “Comment” feature below and therefore spark an interesting conversation – however I will point to some of the highlights I see:

• Although Facebook is the top Web 2.0 site in Canada, an estimated three quarters of all public servants can't access it from work– not even the Facebook group on PS Renewal. (Wow! Those CBSA recruits really scared government officials last year…)
• Over half of public servants can't even access the Prime Ministers’ channel on YouTube, and just under half can’t access his photo gallery on Flickr! (Who do we work for again?)
• In 2008, nearly 18 % of public servants still can’t download a legal MP3 file from the Canada School of Public Service website. (I guess we still have a long way to go before becoming a true “learning organization”!)
• Approximately 43% public servants can't access my blog, despite the fact I use the Blogger platform – the 10th most popular site in Canada. Nevertheless, it is an improvement compared to what it used to be. A year ago, none of my friends at PWGSC – which accounts for nearly 5% of all public servants – could access my blog, but in this survey most said they could.
• The figures are better for my friends at CPSRenewal.ca who, despite the fact they use the same platform as I do (Blogger), are able to be reached by an additional 25% of public servants (that means 65,000 more people!). That is a relief, since I think their blog is the single best source of news on PS Renewal. But why is my blog blocked more than their's? I don't know...
  
• I am actually amazed by the level of access to my sites An Inconvenient Renewal and the group Bottom-Up Renewal. But then it makes me wonder why the participation on those sites in non-existent (I will come back to this in tomorrow's posting).

The methodology I used is obviously flawed, and I won’t argue with that. The numbers presented above probably reflect a best-case scenario. Given the relatively small number of responses I received, the results of the survey need to be interpreted with great caution; so please don’t go around quoting these numbers as facts!

With the recent launch of GCpedia (accessible to federal public servants only), these results promise to generate quite a bit of discussion. Coincidentally, Jason Ryan and Mike Kujawski just made some postings related to the use of social networks and Web 2.0 in government.

Please feel free to discuss the findings below.

More results coming tomorrow!

PS Renewal Survey Results: Stay Tuned!

From September 1 to October 31, 2008, I have been conducting a survey on PS renewal and Web 2.0, using the platform QuestionPro. I originally announced the initiative by sending an e-mail to a long list of selected public servants who I thought may have more at stake in PS Renewal: middle managers, young public servants, etc.

87 people from 34 federal departments and agencies responded to the 10 questions survey. These 34 departments and agencies represent 85,4% of the 263,118 public servants working for the 107 organizations composing the Core Public Administration (FAA Sch. I and IV). Given the size of the public service of Canada, it is quite obvious that this sample of respondents constitute an extremely small fraction of the total population I was targeting.

In fact, I encountered a few obstacles with this initiative:

• In compliance with the direction I had received from my department, I administered the survey entirely on my own personal time, using my own personal email address. Consequently, many recipients who don’t know I am a federal public servant responded by asking "Just who exactly are you?”. As a result, I suspect many people chose not to fill the survey and not to forward the link to their friends or colleagues.
• Some recipients also refused to forward the link on the grounds that surveys in general consume too much of employees’ time, and people were concerned that this survey would be one too many given the upcoming launch of the 2008 Public Service Employee Survey.
• A few recipients indicated that the firewall of their department or agency prevented them from even accessing the survey (!).
• Finally, while the initial announcement was bilingual, the reminders I sent afterward as well as the survey itself were only in English. For that reason, I must stress that the survey’s results may not reflect the perspective of all public servants.

Consequently, I make no claim about the representativeness or accuracy of the results. These results are just what they are: the perspective of 87 individual public servants. Regardless, I know some readers will see the value of this exercise and will find the results interesting.

Rather than presenting all the results of the survey in a single posting, I will spread them in multiple postings over the course of the next week. Each posting will present the results of a few questions, i.e.:

• Access to Web 2.0 and PS Renewal-related Websites;
• Interest and Usefulness of PS Renewal-related Websites;
• Clarity around PS Renewal;
• Getting Involved in PS Renewal.

Stay tuned!