Computational & Data Journalism @ Cardiff

Reporting. Building. Designing. Informing

You are here: Home / Archives for Aidan O'Donnell

Some APIs for journalism

by

This month we find ourselves digging up data with the help of APIs. While there are oodles of APIs for different things (there’s a Star Wars API and an ISS API and many many others), I wondered which endpoints might be interesting for journalists. So here is a list of some of them — we’ll add to it as we find more — starting with government and moving on to business, health and … where you can charge your electric car.

* means an API key is required, ** means an API key plus extra authentication is required

Government

 

Covid, weather etc.

 

US

 

Media

 

Tim Harford’s lethal bathtub

by

Tim Harford’s books are on the reading list for journalism students at Jomec and we are big fans of More or Less. And this month he supplied us all with a great case of numbers going wrong, in a piece for the Financial Times.

You can listen to him explaining it on Radio 4’s The World at One (segment at 17′ 52″).

The thinking — about how dangerous UK life is during the Covid pandemic — goes like this:

  • Every day in the UK about 40 people out of a million get the virus (ONS).
  • How dangerous is it if you’re one of the forty? If you’re aged 60, you have roughly a 1% chance of dying if you catch it.
  • 1% of ’40 in a million’ gets you to almost a 1 in 2 million chance of dying. So, if you are 60 and live in the UK at the moment (and are exposed to the typical risk in the UK) there’s a 1 in 2 million chance Covid will kill you.
  • Or make that a one in a million chance if you include ‘serious injury’ since another 1% of the ’40 in a million’ who catch it are left with health problems.

Everything, Tim Harford says, is fine up to here. But then he looked for other things that had a one in a million chance of death / serious injury. One of them, he explained to The World at One, was “taking a bath”.

“So when I discovered this I thought ‘oh, I wonder what else is about that risky?’ […] So when I wrote this all up for the Financial Times I just — as an afterthought, having worked so carefully to get all my Covid maths right — I just said ‘it’s a bit like riding a horse, riding a motorbike, going skiing, or taking a bath'”.

This is the error. The risk of dying in the bath is one in three million every year — not every time you take a bath. As Tim Harford remarks “Covid is no more risky than you thought. And taking a bath is much safer than you thought”.

Nonetheless, “That is the most shared thing I’ve ever said because it’s the most interesting thing I’ve ever said […] because it happens to be wrong”.

It is, as he observed, an instructive case of how mistakes happen and what newspeople pick up on.

His full account of it is on twitter.

 

Our course after 6 months of Covid-19

by

The Covid-19 pandemic shut down our schools at the end of March and sent staff and students alike home to work on their laptops. This meant MSc students finished their group projects using online platforms and started dissertation projects while trying to get back to their home countries, or while stuck in Cardiff.

Although the Summer months are probably the right time to be stranded here when we get more sun than usual but less than in hotter parts of the world.

A new cohort of students will be arriving in Cardiff next month. Our course this year will run both online and in classrooms for the first semester. The computer science courses will be taught online, while most of the journalism work will take place in classrooms.

There is of course a huge amount of data and data-related stories that have been published in recent months because of the pandemic. And, it appears, the data and the effects of the pandemic on societies around the world will keep coming for a while yet. So it is a good time to be working on this kind of material.

And the Americans are planning an election, which should keep us busy in November and the weeks before.

 

The Clwstwr news projects — update

by

Clwstwr is a five-year programme in south Wales — run from Cardiff — that was started to encourage the development of original screen-related projects. ‘Screen’ here means anything that involves creative or technological industries in a broad sense. Since it was set up in early 2019, it has allocated funding and development support to 23 different projects to allow for original research and development.

Many of the projects have been underway for close to a year at this stage (a full list of the projects is here) and a few of them are of particular interest to us since they are working on news:

Artificial Intelligence in the newsroom

This project is investigating how to put the resources of the deep web at the disposal of working journalists, by using artificial intelligence. It’s run by the Cardiff team of Amplyfi, a company that uses tech for business intelligence, and the project aims to  develop technology that will identify new entities that are emerging in the deep web, and especially new relationships between those entities.

Extracting court information for the press

The team behind the Caerphilly Observer are running this project, which will deal with court information (who’s appeared in court, who’s due to appear) that is often either unwieldy or downright inaccessible for journalists. The plan is to gather all this information for Magistrates Courts in Wales and make it available to journalists through a searchable database, which would greatly aid press coverage of local courts.

New ways of telling news stories

What’s the best way to tell a news story? This project is trying to answer this question by looking firstly at how people understand and response to stories in general, and then by designing new journalism techniques that will allow the press to tell stories in the most effective way possible. It’s a radical re-evaluation of a journalistic storytelling tradition that has long worked just on the basis of ‘that’s how we’ve always done it!’.

News in school

This project will design “a pilot for regular news service delivered to pupils within school hours”. The idea is that teachers can use this service to complement their teaching and that a new generation of young people will be introduced to the idea of staying informed.

Journalism by Numbers — 2019 [Virtual] Summer School

by

With Cardiff University buildings closed since March because of the Coronavirus pandemic, the Summer School for the public moved online in June, and included a one-hour session on what datajournalists do.

The Summer School comprised a week of workshops that ranged from radiography and earth sciences to building design and writing for business.

In our rapid run-through the data journalism world, we touched on classic go-to number stories like A&E waiting times and party-political donations as well as how journalists dig up the data in the first place (FOI, web scraping and so on). We looked at visuals done with colouring pencils, graphing cleaner air in Cardiff during lockdown and the ongoing questions around who keeps an eye on the algorithms.

People appeared online for our Journalism by Numbers workshop from around Wales and the UK, but also from Pakistan, Sweden and Nigeria.

Other workshops during the week covered ethics in Artificial Intelligence, copywriting and Google analytics. There was also a session on the ever-interesting Pharmabee project (which launched the Spot-a-bee app this year as part of their bee-mapping project).

Capturing OSINT flags with Cardiff’s Cybersoc

by

Cardiff University’s Cyber Society gave us all its Capture the Flag challenge earlier this year and now has over a thousand players on its leaderboard, many of them sitting on the maximum score 0f 15,000.

The challenges are organised into three streams: ten introductory questions to get you warmed up, 18 tasks for online intelligence gathering and finally a dozen challenges centred around some fictional characters and their online life.

There are no pre-requisites for attempting it — it starts with a “What is OSINT?” question, so beginners are welcome — but it should test most players’ “resilience” (i.e. can you keep playing even though you’ve run out of ideas, patience and any sense that you once knew anything about online intelligence gathering?). At least one of our Computational Journalism students has made it successfully through all the challenges.

The challenges were featured by We Are OSINTCurious on its webcast in March.

SELECT * FROM a day of SQL…

by

This month our students survived a full-day workshop on SQL, moving from the very basics of the syntax to querying datasets or working through some of the better tutorials.

First up was the excellent Select Star tutorial by Zi Chong Kao, which is based on a dataset of US prisoners executed since 1976.

We then looked for newslines in a sqlite database of US babynames (via the command line) and wrote queries in Carto to map a dataset of protected Welsh monuments.

There was more sqlite with a database of shooting incidents involving Dallas police officers, this time via a notebook. And we finished with the Knight Center’s fine SQL-based murder mystery.

Enough there to get you started (or refreshed) with your SQL syntax.

NHS Hack Day returns to Cardiff

by

At the end of semester two our 2019 students set off (armed with a full semester of javascript) for the Cardiff NHS Hack Day.

This regular event moves around the UK and brings together health specialists, technologists and anyone at all who’s got a suggestion about improving any aspect of healthcare.

There’s an overview of the Cardiff event by one of the judges available here and an account of several of this year’s projects here.

Our Computational Journalism students spent two days working on a app called “Can I eat this?” which allows users with dietary restrictions to scan barcodes on packaging see if a product is safe to consume — their app and their presentation to the judges is available on the Hack Day website and the code is here.

 

Above photo by https://www.flickr.com/photos/paul_clarke

Election data — the UK’s December vote

by

Elections are a special meeting of journalism and data. They generate lots of both! So the morning after the long night of vote counting, we got this year’s students working on the results for a full day. Four student groups were each given one of the four UK nations. Each group also got a Welsh constituency to analyse; after consulting with our pol corrs on the MA-News programme we decided the interesting Welsh battles would be in Cardiff North, Ceredigion, Caerphilly and Vale of Glamorgan.

The main difficulty with analysing the results was not having the XML feed from PA that UK news organisations had been relying on (and had been testing for weeks). We didn’t have the raw data flowing in as soon as a count was announced. But that’s where the BBC came in — they published results for each constituency in a standardised url, supplying 650 webpages for the UK’s 650 constituencies.

This meant that it was enough to draw up a few lines to grab each page and the corresponding batch of results. If only all large-scale scraping was as clean and consistent!

We were able to publish csv files with full results for the four nations by the end of the day. Now of course you can get them from lots of sources but right after the election, and with results still being declared throughout Friday, we were able to get started on analysing the results once we had these tables.

The people at Flourish provided very helpful templates ahead of the vote. So hex maps, animated bar charts and Sankey diagrams were all ready and waiting for numbers.