Aug 6, 2015

Takeaways from the CAJ conference, Part 3

This is what five years worth of data looks like.
David Weisz of the Toronto Star, who is a co-instructor at the University of King's College data school, led a session at the June CAJ conference titled, Dirty Data and Common Mistakes.

From the annual King's College Data School website, this is what data journalism is about:

Data journalism encompasses a lot of things these days, from the data analysis skills that have traditionally been known as computer-assisted reporting, or CAR, to computer programming to developing news applications...CAR skills can range from using a spreadsheet to re-order and make sense of a list of large salaries, to using a database program to crunch through a large inspection database to designing maps that compare poverty and crime in your community.

One example where a large dataset might be analyzed in a spreadsheet program (typically people use Excel or a program like it), is this story about how eleven nurses at the Nova Scotia Health Authority are earning twice their salary, while "dozens more are earning tens of thousands of dollars more in overtime." It raises important questions about why nurses are working so hard, and if it's safe for them to do so.

I am trying to work through a huge dataset right now (pictured above!) so I decided to go over my notes from David Weisz's session to remind me about good data practices. A few pointers:

-All data sets are dirty. There will always be technical errors.
-Make sure you have a strong index. That means, make sure each individual record has its own unique identifiers. This avoids duplicates and makes connections within the data set.
-Check to see that each individual identity number only comes up once.
-Relying on scraped data is a great way to be wrong.
-Don't be afraid to check issues that seem wrong with other sources such as communications staff, FOI staff, annual reports.
-If you feel you may be wrong about the conclusions you are drawing from your data, embrace your fear. You may well be wrong.
-Outliers: do they catastrophically affect the story? Has someone screwed up somewhere? If you can work around it, do. Give ranges instead of exact figures, if necessary to remain accurate.
-Know the weaknesses of your data and be prepared to defend them.
-Get and give as much context for the data as possible.

Jun 24, 2015

Takeaways from the CAJ conference, Part 2: How to tell your stories with maps

The map below was created using a program called ARCGIS, a mapping service offered by Esri Canada, a company that specializes in geographic information. Try each of the different tabs (Global Ports, The U.S., Southern California) for a new map. 

Paul Voegele presented on behalf of Esri at the CAJ conference, teaching a course on how to use the mapping service through free accounts offered to journalists. Esri profits come from other paid services. The reason it offers the technology to journalists, Voegele explained, is because Esri hopes to build exposure as journalists embed their maps within news sites. All their maps have an Esri watermark built in. 

Here are a few examples of story maps built using ARCGIS. The links are embeddable and shareable. You can upload data from a .CVS file for quick mapping. The maps are very customizable. You can also layer in real-time conditions collected by Esri, including things like traffic conditions, building footprints, and even Twitter comments made by location.

Esri is also linked to Environics demographic information, which could be quite powerful if you're trying to map something like household debt or income by city.  

Jun 11, 2015

Takeaways from the CAJ conference, Part 1: Dean Beeby and FOI for Dummies

On June 5, I went to the first day of the Canadian Association of Journalists' annual national conference. I wasn't able to attend both days of the event, but the sessions I did go to were fantastic. In the next few posts I'm going to lay out what I learned, session by session. This is part one: Dean Beeby and FOI for Dummies. 

It was a great treat to meet and talk for a bit with Dean Beeby. He spent almost all of his career at the Canadian Press, and just last year he went to the CBC's Ottawa bureau. To give an example of his reporting, Dean broke this story about the Harper government leaving $97 million dollars unspent on social services.

That story couldn't have been done without certain key documents. Governments generate millions of documents, and the majority of them will never be seen by the public. Access to information stories give people an insight into ways governments work that we would not otherwise have, Dean explained. 

It costs a $5.00 application fee to use freedom of information (FOI) and access to information/privacy (ATIP) legislation. Legislation exists at the federal and provincial levels, and some municipalities. Federal is the easiest to navigate and the best place to practice requests. Some major federal departments such as the department of national defence are now accepting online applications and payment by credit or debit card. This is good, because the department is required to respond with certain time limits and online applications expedite the process.

Here's what comes with the $5.00 application fee: email responses from the department, a CD of electronic copies of the documents (if requested), 125 pages of photocopies, and 5 hours of processing or research time by department staff. Sometimes we hear about FOI applications costing ridiculous amounts of money, but Dean said that by targeting his requests and making multiple requests (each new request gets 5 more hours and 125 more photocopies), he rarely pays anything more than the application fee. 

Your identity as a requester is supposed to be confidential and stay within the ATIP unit - but a minister can overrule this. Also, any applications marked "journalist" as opposed to private citizens may be flagged to the minister's office. There is an option where you may "decline to identify." 

Somebody in the audience asked if it would be helpful to have a non-journalist make the requests. Dean said he didn't think that would work very well, and in a climate where it is very hard to get an answer from the federal government we should use the legislation fearlessly. Be polite, be persistent, demand good service and respectful treatment.

Some documents are privileged and excluded from ATIP requests. But some of those, such as cabinet confidences, will be declassified after 20 years. Then they'll come under the act, and if they still have relevance to what's going on today, it could be worthwhile to request. 

Six months is about the maximum useful timespan to request a series of documents (e.g. requesting documents generated between January 2014 and June 2014). If you request more, you may end up with a pricey bill or more information than you can handle. You can also split up your requests (e.g. one request for January-June, a second request for July to December).

Make it a habit to request lists. You can then look over the lists and see if there are items on the list that deserve a deeper ATIP. E.g. at the beginning of the month, Dean requests a list of briefing notes to ministers and deputy ministers. All he receives are the titles of the notes, but then he looks over the titles and does a second ATIP on anything he thinks is intriguing. He encouraged all of us to pick a ministry and try this next month.

Also, keep your wording in requests general. "Briefing notes" might be too specific. "Briefing material, including briefing notes, memos, Powerpoint material, house cards, etc" is better. Etcetera is a good word to include. 

If you are not sure about useful wording, you can take a look at the federal database of completed ATIP requests. Any of those completed requests can be ordered without paying the $5.00 fee, so you can see what you might get back.

Try things like checking the Canadian Merx tendering website. Maybe a department is ordering an interesting consultants report, to be due on a particular date. You could make a note in your calendar and ATIP that report when the date comes around.

Any citizen can use the ATIP/FOIPOP legislation for themselves. It was put in place to give citizens better access to their government. Only about 15 per cent of requests come from reporters, Dean said. The rest come from opposition parties, trade unions, lobby groups, and many other types of requesters. Dean said many reporters never learn to use this legislation but he sees it as a duty. 

Here is Dean's quick guide to ATIPs, posted on the CAJ website. Follow him on Twitter @deanbeeby.

Apr 3, 2015

I was in Costa Rica in March, which was a relief from this endless winter. We rented a car and drove around the country, from the wet green mountains to the dry heat of the Pacific coast.

Good moments:

Zip-lining down mountains and rappelling down waterfalls.

Lying on the beach watching the surfers.

Soaking in a hot tub underneath a volcano, watching the green fireflies wake up in the dark.

Ceviche for dinner.

Green hummingbirds outside the window when I woke up.

Stopping to drink tea at a tiny cafe in a village on top of a mountain all wreathed in cloud. The cafe was run by three generations of women who somehow made sense of our broken Spanish. They made the meals on a brown Aga cooker and we sat at the counter and watched while they washed the dishes. Then we watched a football match next door.

The evening our Costa Rican host brought us shots of homemade moonshine made from sugarcane, pineapple, and corn.

Driving through a small village on Sunday morning when everyone was coming out of church carrying long green spiky things, and realizing that it was Palm Sunday and people had carried actual palm fronds to service.

Feb 25, 2015

On story structure

I was thinking this week about story structure. There is a rhythm to news stories, and when you produce them every day it becomes mechanical. Clunk, clunk, clunk.

So you have to force yourself to think about things like story structure sometimes, especially for an exceptional story.

Here's one. It is a story about an old man, lost in a snowstorm, who abandons hope and prepares himself to die. He is rescued at the last moment by an ex-politician who mistook him for a seal, then carries him to safety.

There are a few different ways that you could structure that story for broadcast. Somehow, you have to bring out that marvellous moment when we realize: that's not a seal, that's a man.

This lesson from This American Life's Ira Glass was in my mind. Glass is a master storyteller, and he makes a case for the power of anecdote. There's a great force behind the question, "what happened next?"

You want to present a piece of bait, Glass says. Constantly be raising questions and answering them. What is that dark shape lying on the ground?

"You can feel through its form, that when you have one thing leading to the next, to the next," Glass says. "You can feel inherently, that you're on a train that has a destination, and that he's going to find something."

I say, we have to begin with Charlie Parker, the former MLA with the seal question. In news we usually start with the most important information, but you can't have a big reveal if you give it all away at the top. It's rather unusual to find that the main character of your story is, himself, the big reveal, but this is why it's good to think about story structure sometimes.

Here are a few more thoughts on story structure:

Anyone who writes (particularly for broadcast) should listen to Ira Glass on storytelling.

Totally worth dissecting the essays of Joan Didion. In Blue Nights she describes writing in a way that sounds more like musical composition - putting down marks on the paper to determine the rhythm of the words. “We tell ourselves stories in order to live," she wrote in The White Album. "We interpret what we see, select the most workable of the multiple choices. We live entirely... by the imposition of a narrative line upon disparate images, by the 'ideas' with which we have learned to freeze the shifting phantasmagoria — which is our actual experience.”

From the Nieman Foundation for journalism: "Stories give shape to experience."

Kurt Vonnegut gives an amusing talk on story structure.

Jan 15, 2015

Havana Nights

Last week I was in Havana, where it is warm and breezy and men smoke fat cigars in shirt sleeves while directing garbage trucks.

Ah, Havana. And good timing too, wasn't it, since the trade and travel restrictions loosen up tomorrow?
My most vivid memory of Havana was when we went dancing one night to the Hotel Florida. We had drinks in the tiny, smoky salsa bar, where the tables at the front are reserved by the Cubans and the people in the back can't see the floor because of the crowd of bodies. I danced until I could feel my shirt sticking to me with sweat. 
When we left the bar it was raining hard. We were hurrying to get to Central Park where we'd have a better chance of picking up a taxi. But before we got to the park, the sky started spewing rain so hard we had to stop under a shop awning. There were several Cubans standing underneath the awning as well, and we started talking to two of them. They were an older woman named Maria who was missing a tooth and holding a shawl over her head, and a young man named Marco who was wearing a suit jacket and carrying a briefcase. 
With broken English-Spanish we told them we were from Canada and we learned they were going to a party at a place called the Guantanamera. But we were tired and wanted to go home, so all of us decided to run for Central Park. When we got there, we stopped under an arcade and Maria spotted a man she knew, standing there with an inside-out umbrella in his hand. 
Don't take that other taxi, she said in Spanish. This man has a car, and he'll take you where you're going for six pesos (with a little extra tip in Maria's hand too, of course!). The man's name was Daniel, and his car was a battered blue sedan with a low roof and a bare bit of cloth covering the springs of the backseat, but we jumped in.
You've seen those beautiful pictures of the shiny 1950s boat-cars with the fins and the chrome? Daniel's car was nothing like that. It was 50s, all right, but it was the small car of a large family that made fierce economies. There was no panelling inside at all, just the bare metal body of the door, with some big metal screws where the door latch and the window handle should be. One of the windshield wipers was missing. The one that remained slid over the glass so slowly it did nothing to sweep away the rain. 
What that car did have was a completely modern digital sound system that glowed red from the dashboard and blasted salsa music from two vibrating speakers in the back. 
By my talented dad
So picture us speeding down the Malecon in the middle of the night. The storm waves are rolling in over the seawall and onto the road, the salsa music is blasting, we are completely unable to see anything in front of the car except the lights of on-coming cars. Daniel reaches out the driver's window twice to reattach the lonely windshield wiper. It was quite impossibly perfect. 

Dec 4, 2014

What I'm reading

For the last week I've been "reading" a book on audiotape. It's Wild by Cheryl Strayed. It's about Cheryl's trek along the Pacific Coast Trail from California to Oregon and Washington state. She had no experience as a long distance hiker, and took it on as a project to heal from the death of her mother and a divorce from her husband. I like this passage from a part of the trail when she's walking through the Mojave Desert, among the Joshua trees and the rattlesnakes.

"It was a deal I'd made with myself months before and the only thing that allowed me to hike alone. I knew that if I allowed fear to overtake me, my journey was doomed. Fear, to a great extent, is born of a story we tell ourselves, and so I chose to tell myself a different story from the one women are told. I decided I was safe. I was strong. I was brave. Nothing could vanquish me. Insisting on this story was a form of mind control, but for the most part, it worked. Every time I heard a sound of unknown origin or felt something horrible cohering in my imagination, I pushed it away. I simply did not let myself become afraid. Fear begets fear. Power begets power. I willed myself to beget power. And it wasn't long before I actually wasn't afraid.”