92. Design experiments for local democracy

Photo credit

The notwestminster work we have been doing is seriously great and the folks involved have achieved a lot.  There have been two brilliant events, many ideas generated and lots of new connections made.  From the stuff done so far we have settled on a set of local democracy design challenges to work on - a list of things we want to change.

While this is all good, the next step is to do something practical; to make some stuff.  We had a brilliant Maker Day in February but we need to take it up a notch.

The method I'm suggesting is design experiments.  Here are some notes and first thoughts.

Design Experiments

Recently I have been reading ‘Nudge, Nudge, Think, Think’ by Peter John and others.  A fascinating book - you should read it!  It’s primarily about techniques to encourage behaviour change (e.g. nudge, think) in areas of civic concern such as recycling or volunteering.  It is also about how you test and develop innovations in public service settings.  Specifically it looks at how randomised control trials and design experiments can be used to explore new ways of engaging with citizens.

Stoker and John describe a design experiment like this:

‘…researchers work with practitioners over a period of time to design, implement, evaluate, and redesign an innovative intervention. The group receiving treatment is compared to a comparison group. The aim is to perfect the intervention over several iterative cycles until it is the best it can be [1].

One example in the book is of a video being used to get the voices of the ‘seldom heard’ into the debates at a council's area committee meetings.  The research team asked local practitioners to produce a video that was then shown to different area committees.  Observations at each meeting led to changes to the video and, at the end of the experiment, conclusions were reached about the effectiveness of the approach and how it could be made more effective in future (turns out that the context of the meeting was as important as the video itself).

Along similar lines are some of the experiments being done by Nick Taylor.  See this simple voting machine to encourage community engagement, for example.  Again, a small intervention designed to test the idea that lowering the bar to access improves community consultations (it does).

The design experiment method comes from education where classrooms - a relatively controllable environment - serve as laboratories for experiments.

In the same way the formal process of local democracy in local councils can provide a great laboratory for democracy experiments.  It is a relatively stable and controllable environment and, where you have multiple meetings of the same type, it can support comparisons between interventions and non-interventions.

Design experiments of this type can only tell us what works (or doesn't work) in a particular context and it is important to recognise this limitation. What they can provide, however, is the starting point for experiments on a wider scale in in different settings that might point to more general conclusions.

Advantages

I think there are some real advantages to using design experiments to take forward the #notwestminster design challenges.

1.  It is a manageable approach  
Of course councils don’t have extra resources to dedicate to this type of work but, given that the experiments would be small and hopefully fit with local work that might have been done anyway, I think design experiments are a reasonable proposition.

2.  It will test our assumptions
In our design challenges we have a number of assumptions about how people will behave if different aspects of local democracy are redesigned (people would be better informed if we did this…, people would get more involved if we did that…).  Wouldn't it be great to have some evidence to back this up?  Design experiments, if done robustly, could provide evidence to support our assumptions (or force us to think again).

3.  We can make the most of our network
One of the things I love about Notwestminster is the way it brings together councillors, practitioners, academics, techies and citizens.  Design experiments are a great was to bring people together in small teams to make something new.  We can also share experiments in progress across the network getting valuable input as we go along.

4.  It will give us something to share
By reporting experiments and their outcomes we can contribute to a growing body of knowledge about civic innovations that will be of use to practitioners and researchers alike.  What’s not to like about that?

5.  We can make something worthwhile
Last, but not least, wouldn't it be great to actually make something that makes a difference to a local council and its citizens?  Wouldn't it?

Challenges

In developing the method and approach there are also some challenges to be overcome.  Sarah Cotterill and Liz Richardson [3] have highlighted some of these as they relate to working with local government and a couple are particularly relevant here:

1.  Measuring the difference
If we are serious about research we need to be clear about what ‘outcome measures’ will tell us what we need to know.  Ideally we will want ‘objective’ measures (e.g. voter turnout) but sometimes we will need to rely on people’s perceptions.  The classic concerns about validity and reliability apply.

2.  Mixed methods
If we want a rich picture about what is happening around a particular experiment we will need to invest in a range of sources of evidence.  What research techniques should be used? Do we need a 'tool kit'?

3.  Organisational commitment
Cotterill and Richardson point to organisational difficulties as a reason why many experiments in local government fail.  How can we ensure that experiments won’t get neglected when other pressures and priorities come into play?

Design experiments also require a different way of working and participants need to be clear about that at the start.  As John et al suggest:

'Design experiments favour small-scale innovation, in a relatively controlled environment, where the dialogue can take place with a small range of policy-makers and workers, all of whom have signed up to a new way of doing business and to intense researcher-practitioner interactions.' [2] 

But what should this commitment look like and how can we make sure that it sticks?

4.  Good governance
Design experiments in councils would be managed by ‘design teams’ involving councillors, practitioners, researchers, practice advisors and others.  Being clear about roles and how decisions are made will be important – but how should this look exactly? How should the different experiments be linked together and managed as a whole (if at all)?

So, some initial thoughts.  Let’s see what can be done with them.


[1] Stoker, G and John, P (2008) Design Experiments: Engaging Policy Makers in the Search for Evidence about What Works, Political Studies, Vol 57 (2).
[2] John, P et al (2013) Nudge, Nudge, Think, Think, Experimenting with Ways to Change Civic Behaviour, Bloomsbury, London.
[3]  Cotterill, S and Richardson, L (2010) Expanding the Use of Experiments on Civic Behavior: Experiments with Local Government as a Research Partner, the Annals of the American Academy of Political and Social Science, Vol 628

49. A Library of Local Politics Case Studies

I’ve just come across a paper by Rod Rhodes.  It’s called ‘The Lost World of British Local Politics?’ and was published in 1975 in Local Government Studies.  It essentially provides a mapping of what was the then emerging study of local government.

The thing that caught my eye was a table showing the local government case studies that had been completed by academics up until 1974.  A couple of them can be found on my page of local government classics.  Here is a selection of the county boroughs to give you an idea:

I have no idea how many new studies have been completed in the nearly 40 years since then (or even if more recent maps have been produced) – but I do think it would be great to have an up to date version, say as an online database, which could be used to find all the academic case study work that had been done on any given council.

This would obviously be a really handy research tool but I also think it would have real benefits for local councils.  Local government is not good at preserving its history and such a database would help councils to understand their own story.  Case studies provide rich pictures of days gone by that some or all of the organisation may have forgotten about.  Often we are not talking about that long ago.  

These case studies can also provide useful external perspectives.  They will include the conclusions and critical reflections of independent researchers and they might just offer some useful challenge.  Who knows, they may even shed light on current problems or provide some relevant and useful evidence.
Project anyone?

48. A Journal of Welsh Public Service

Shouldn't Wales have its own academic journal dedicated to public service?  Well I think it should.

I know that there are sector specific publications such as the Welsh Housing Quarterly but, as far as I know, there is nothing that covers the Welsh public service as a whole. 
Such a journal could focus exclusively on Welsh concerns and have peer reviewed academic papers (although not too many) alongside contributions by practitioner and early career researchers.    It could provide a focus for relevant debates and facilitate dialogue between academics and practitioners.  It could be open source, allowing free access to everyone with an interest.  It should also be bilingual; promoting the Welsh language within wider academic circles.

Specifically it could provide:

• A forum for critical reflection on public policy and service delivery
• Critical evaluation of new initiatives
• A focus for cross sector learning
• A means for disseminating relevant research to a practitioner community
• An opportunity to debate and develop public service challenges and innovations in the Welsh context

On this last point I would argue that Wales provides a natural community for such a journal.  As the Beecham review argued, Wales has the potential to be exemplar small nation governance.  It has a devolved government with an interest in evidence and research, it has national bodies with similar preoccupations and it also has university departments dedicated to public management and administration.  It even has its own think tanks and pressure groups.    Surely such an important research community deserves its own journal?

Update:  looks like there is a Croatian model (CCPA) that this could be based on.  Below are some nice tweets I had on this post.  I partuclarly like the SHU site highlighted by Tamsin Stirling:
 

@localopolis #Localgov Great idea Dave! Running and editing a journal is hard work. Meanwhile, you can submit papers to #CCPA :-)
— Dr Vedran Đulabić (@vdulabic) October 4, 2012

@localopolis that's an excellent idea. You just need a group of people/ institutions to help achieve it.
— Filipe Teles (@FTeles) October 5, 2012

@crischarliepaul @localopolis @tamsinstirling1 @catherinefarre we need better connect btwn research & practice; does journal = way to do it?
— Megan@KafkaBrigade (@Megan_KafkaUK) October 5, 2012

@tamsinstirling1 @megan_kafkauk @localopolis @catherinefarre Online woud work well- have many ideas actually!
— chris paul (@crischarliepaul) October 5, 2012

Research and practice - I like the SHU on-line approach @localopolis @crischarliepaul @tamsinstirling1 @megan_kafkauk @catherinefarre
— Catherine Farrell (@CatherineFarre) October 5, 2012

Photo credit: http://www.flickr.com/photos/bslavin/2214558473/

46. Learning and Research Communities

In this post I argue that public sector practitioners and academics need to be investing in semi formal communities in order to bridge the gap between their two worlds.  Such communities can foster the interpersonal networks needed to generate the trust, common understandings and shared priorities on which effective co-learning and co-research can develop.  

Bridging the Gap between Practice and Academia

As a full time practitioner doing a part time PhD I often find myself hopping between the worlds of local government and academia.  I also like to visit some of the places that exist in-between.  The Public Administration Committee conference, for example, is an annual theory and practice forum that I have personally found extremely useful as a place to develop my academic understanding and confidence (I sketched out the ideas for this post on the train after the last one).

The problem is, however, that events such as the PAC conference are the exception rather than the rule.  I read with great interest this piece by Kevin Orr and Mike Bennett on the hidden politics of 'co-production' in research which confirmed my belief that one of the biggest challenges in this context is indeed ‘the distance between the academic and practitioner worlds’.  This is something I picked up in a small way in this paper I did on local government attitudes to research degrees (pdf) that I presented to the PAC conference in 2009.  In a similar vein this is my Prezi to a workshop for the annual Centre for Public Scrutiny conference in 2010.  My general argument is that while local government doesn’t really ‘get’ academia, at an individual level it something we often value.  Differences therefore relate more to misunderstandings that to something more fundamental. 

This distance between the two worlds is a big missed opportunity in my opinion.  While academics are increasingly concerned about the impact of their research so public sector practitioners are concerned about wider outcomes in the community. While academics are being asked to become more ‘real world’ by their funders and institutions so practitioners are under pressure to employ evidence and critical thinking more than they ever have done before.  While austerity has severely limited local government’s ability to spend money on training, courses or events so those involved on the academic side live with uncertainty about whether they fit with the priorities of their institutions and wrestle with the future relevance of their disciplines. 

Learning and Research Communities

How then to build better bridges between practitioners and academics?  I think the answer is to invest in the setting up and support of Learning and Research Communities. 

This idea of community is borrowed very much from the world of social media (see my previous post on Social Bureaucracy).  It implies something informal, fluid and flexible, something that has a focus on relationships between individuals rather than between institutions and social rather than contractual interaction.   Communities can be porous, overlap and interact.  They imply a sense of shared identity, purpose and belonging.  These agile forms of working allow work to happen across organisational, departmental and disciplinary boundaries – something which is a challenge in both worlds.  I think Patrick Dunleavy illustrates the way the world is changing in this respect from an academic perspective rather brilliantly in this presentation on the Republic of Blogs.

Learning and Research Communities could provide a new type of environment in which learning and research, whether formal or informal, would take place.  The biggest difference would be support for networking and interaction outside of those more traditional activities.  

Supporting a community would mean:

• Providing the community its underpinning support,  with a home organisation (lending a sense of credibility and legitimacy) as well identifying the individuals who will perform community roles e.g. admin, facilitator etc
• Establishing a clear purpose and identity around a defined theme for the community such as ‘Welsh Scrutiny’ or ‘UK Directly Elected Mayors’.  The important point is to have something that is meaningful and important both to academics and practitioners.  
• Determining the basis for membership - this might be through an application or may just be open to anyone with an interest.  It also means defining the responsibilities for members and describing how they contribute to the purpose of the community – again this need not be anything grand, simply a short statement of expectation.
• Identify the way in which interaction will take place; online and off.  Social media sites such as the Knowledge Hub, Yammer or even twitter can do this.  Face to face events are also important whether traditional conference style events or unconferences such as localgovcamp.

It is the community, and particularly the networking and sharing elements that provide the context for other activity to take place.  For practitioners these would be spaces where they can engage with academia without cost or commitment in the first instance, where they can gain awareness and find out about opportunities.  Perhaps most importantly spaces where relationships and trust can be fostered leading perhaps to joint projects.  Similarly for academics communities should provide a space for them to gain a more nuanced understanding of the needs, wants and concerns of those engaged in practice and to be able to adjust their own ‘products' accordingly, to be able to explain what they have to offer and to adapt quickly to changes.

Of course the cost of resourcing these communities will be an issue particularly as membership should be free wherever possible.  For me this requires working from a different business model, one where supporting communities is in some sense a loss leader for the other, more traditional, transactions that come after.  This is the way that many web 2.0 services now operate; a free basic membership with the option to migrate to premium charged services later on.

The piece by Orr and Bennett is a reminder that if we want to foster co-learning and co-research initiatives then first we need to have in place relationships of trust and understanding between individuals.  This cannot be done by a top down planning but requires putting our faith in something much less planned - something much more organic.  There are, no doubt, many examples of informal joint working already out there that can be used to inspire confidence that this is the way to go.  I suspect there are also already examples of Learning and Research Communities in action.

In summary, I think there are really exciting opportunities to be had if we take advantage of new ways of organising to fill the space between the town halls and the ‘ivory towers’ of academia.  I am also going to suggest a name; the University of Public Life.


Photo credit: http://www.flickr.com/photos/lselibrary/3833726778/in/photostream/