Pen and paper…

Thoughts, ideas, questions, experiences

All things FOI and data journalism related…. May 6, 2010

Filed under: Online journalism — Kellie Maddox @ 11:35 am
Tags: , , , , , , ,

Top of yesterday’s ‘to-do’ list was not ‘attend a one-day session with the MA students on FOI and CAR (computer assisted reporting)’ but that’s actually what my day entailed. And I’m really glad I went now.

James Ball came to uni to spend the day with MA Online Journalism students talking about FOI and data journalism – two areas we have looked at in Online Journalism this semester. Having found it difficult to a) yet find something worth pursuing with an FOI request and b) work out what exactly to ask/avoid/point out/demand etc, I found James’ talk on the in’s and out’s of FOI really interesting and useful.

So – what did I learn from James?

  • There’s certain information you shouldn’t ask for
  • Names of people: some data will be given anonymously but senior civil service workers are identifiable by name as are directors od local organisations
  • In many cases – such as crime stats from the police – they will often give responses like “less than 3” or “less than 10” (their reasoning being that you could work out the identity of such few people therefore confidentiality is at risk)
  • That said – you can often identify people without the data so it undermines the argument anyway!

Two things to be aware when filing FOI requests, of according to James:

  1. Cost limits – the cost of someone dealing with an FOI request is  £25 per hour (so  for small organisations like councils 18 hours works out at £450). Organisations can only charge for ‘staff time’; they can’t charge for passing it between departments or up/down the ladder.
  • It’s harder to dispute cost on time grounds, but it is worth checking.
  • Think of FOI as a conversation – you can negotiate well to get the information you need most quicker. E.g. if you asked for 3 years worth of data across all ‘areas’, be flexible and ask what would make the process easier – less data, fewer areas etc

     2.   Commercial confidentiality – organisations should only use this reason for refusing FOI requests when an agreement has been signed in    connection with the ICO (Information Commissioner’s Office). You should ask to see a copy of the agreement in such cases.

Other useful tips and hints:

  • Crowdsourcing FOI requests work nicely – make it easy for people to request FOI’s by giving sample letters/helpful pointers/FAQs to help them engage in the process
  • SPECIFY THE FORMAT you want the data to be in – if it’s in a PDF you can’t extract it easily to work with so make sure you specify Excel/Access or a tab delimited text file
  • FOI requests work best when you have a clearly defined topic and question
  • If the question you ask doesn’t match up with the organisation’s system – ask them to call you to work out how you can get the data you want
  • Ask for a few more numbers than you need
  • Send a test request when asking more than one company for data – attach a model answer to show exactly what you are looking for

All FOI’d out…..onto data journalism and CAR.

James said that data journalism was ‘catching on’ and was starting to get more attention; but he added that it’s still hard to pitch stories based on data. Although the word ‘data’ may instill the dread of reams and reams of boring numbers to crunch, James assured us that it can actually be fun and lots of quirky stuff can be found like a heat map of the cleanest and dirtiest restaurant in the area.

James’ tips for using data:

  • Learn some basic stats – it’s good to at least know some of the jargon (mean, median and mode) and identify the mistakes you are likely to make
  • Check to see if the dataset cuts off – if it does and you don’t realise you’ll only have calculated a proportion of the data so it will be inaccurate
  • If you interpret the data wrong, it’s your fault and you could be reported to the PCC for doing so
  • Learn how to spot anomalies in data and check them before you start working with them
  • Read up: Huff’s ‘How to lie with statistics’ and Gilovich’s  ‘How we know what isn’t so’ are good places to start

James then went on to talk about CAR and gave the example of the London Weekly investigation which he worked on with others. The ‘case’ itself is really interesting just because it seems totally crazy but the way in which James et al were able to investigate the story using tools such as Whois, wiki history and google maps shows how useful CAR can be in a situation where a company refuses to pick up the phone and ‘face the music’.

(See Roy Greenslade’s blog for the first edition commentary and contributions to the investigation via HelpMeInvestigate)

To round off the day, we then learnt a bit about Excel and the magical things it can do with data (fear not……it’s better than GCSE ICT let on). This included calculating averages, percentage differences, sorting data and creating pivot tables (which really are as fabulous as James claimed!).

And finally…James’ tips for Excel:

  • NEVER WORK ON THE ORIGINAL COPY – save a new one so if it goes wrong, you still have the original
  • Save spreadsheets before you sort – use a consistent labelling system e.g. by date, time or version number
  • Don’t use the ‘spangly and ever-so-attractive 3D charts’ – they make whatever’s closest look biggest
  • Go back and look at the ‘raw’ data and find stories worth chasing

All-in-all a great day, learned lots of new and useful stuff and hopefully I’ll remember it all when I come to do some *fancy* data stuff in the future….