Computational & Data Journalism @ Cardiff

Reporting. Building. Designing. Informing

You are here: Home / Archives for Aidan O'Donnell

Sharing Jupyter notebooks online

by

You have a brilliant Jupyter notebook you’ve been working on.

If only you could share it with the world! Here’s are some options for getting your great notebook online.

 

  • If you just want to show a notebook to people without them running the code, nbviewer does the job by showing the cells and their output and supplying a link where your notebook will always live. If your visitor likes what they see, they can immediately launch a functioning version via a Binder link, or download the .ipynb file. Here’s a simple example of what you see.

 

  • If your notebook is in a github repo, you can to go https://mybinder.org/, supply the repository url and it will serve up all the repo files, with the notebook cells showing output. If the notebook needs modules that aren’t builtins, you can add a requirements.txt file to the repo, so that Binder knows they’re needed when it builds your online notebook environment.

 

 

 

A github repo with Github Pages enabled can run as a webpage using a package called nbinteract but I’ve found it has trouble loading widgets, as seen in some of the tutorial pages.

Of course, Jupyter notebooks are not the only option: Kaggle, Google Colab, Iodide, Azure and many more. There’s an episode of the podcast Talk Python To Me which features the authors of a paper that reviewed 60 (!) of them.

What we did in 12 weeks of data journalism

by

Now that we’ve finished a first semester of data journalism work, I’ve put the module details online. It’s not an online course but it does have our Reading List, a running list of Interesting Datasets for practice (or work) and outlines of what we did each week in the Great Academic Year Of The Pandemic (some in great detail, others less so).

The course ran over 12 weeks and covered … “everything”.

If you want to see more, there are some catalogues online of what people are doing when they teach data journalism: this one started a few years ago by Dan Nguyen, this from the IJEC and this list by Jonathan Gray.

 

 

 

Some APIs for journalism

by

This month we find ourselves digging up data with the help of APIs. While there are oodles of APIs for different things (there’s a Star Wars API and an ISS API and many many others), I wondered which endpoints might be interesting for journalists. So here is a list of some of them — we’ll add to it as we find more — starting with government and moving on to business, health and … where you can charge your electric car.

* means an API key is required, ** means an API key plus extra authentication is required

Government

 

Covid, weather etc.

 

US

 

Media

 

Tim Harford’s lethal bathtub

by

Tim Harford’s books are on the reading list for journalism students at Jomec and we are big fans of More or Less. And this month he supplied us all with a great case of numbers going wrong, in a piece for the Financial Times.

You can listen to him explaining it on Radio 4’s The World at One (segment at 17′ 52″).

The thinking — about how dangerous UK life is during the Covid pandemic — goes like this:

  • Every day in the UK about 40 people out of a million get the virus (ONS).
  • How dangerous is it if you’re one of the forty? If you’re aged 60, you have roughly a 1% chance of dying if you catch it.
  • 1% of ’40 in a million’ gets you to almost a 1 in 2 million chance of dying. So, if you are 60 and live in the UK at the moment (and are exposed to the typical risk in the UK) there’s a 1 in 2 million chance Covid will kill you.
  • Or make that a one in a million chance if you include ‘serious injury’ since another 1% of the ’40 in a million’ who catch it are left with health problems.

Everything, Tim Harford says, is fine up to here. But then he looked for other things that had a one in a million chance of death / serious injury. One of them, he explained to The World at One, was “taking a bath”.

“So when I discovered this I thought ‘oh, I wonder what else is about that risky?’ […] So when I wrote this all up for the Financial Times I just — as an afterthought, having worked so carefully to get all my Covid maths right — I just said ‘it’s a bit like riding a horse, riding a motorbike, going skiing, or taking a bath'”.

This is the error. The risk of dying in the bath is one in three million every year — not every time you take a bath. As Tim Harford remarks “Covid is no more risky than you thought. And taking a bath is much safer than you thought”.

Nonetheless, “That is the most shared thing I’ve ever said because it’s the most interesting thing I’ve ever said […] because it happens to be wrong”.

It is, as he observed, an instructive case of how mistakes happen and what newspeople pick up on.

His full account of it is on twitter.

 

Our course after 6 months of Covid-19

by

The Covid-19 pandemic shut down our schools at the end of March and sent staff and students alike home to work on their laptops. This meant MSc students finished their group projects using online platforms and started dissertation projects while trying to get back to their home countries, or while stuck in Cardiff.

Although the Summer months are probably the right time to be stranded here when we get more sun than usual but less than in hotter parts of the world.

A new cohort of students will be arriving in Cardiff next month. Our course this year will run both online and in classrooms for the first semester. The computer science courses will be taught online, while most of the journalism work will take place in classrooms.

There is of course a huge amount of data and data-related stories that have been published in recent months because of the pandemic. And, it appears, the data and the effects of the pandemic on societies around the world will keep coming for a while yet. So it is a good time to be working on this kind of material.

And the Americans are planning an election, which should keep us busy in November and the weeks before.

 

The Clwstwr news projects — update

by

Clwstwr is a five-year programme in south Wales — run from Cardiff — that was started to encourage the development of original screen-related projects. ‘Screen’ here means anything that involves creative or technological industries in a broad sense. Since it was set up in early 2019, it has allocated funding and development support to 23 different projects to allow for original research and development.

Many of the projects have been underway for close to a year at this stage (a full list of the projects is here) and a few of them are of particular interest to us since they are working on news:

Artificial Intelligence in the newsroom

This project is investigating how to put the resources of the deep web at the disposal of working journalists, by using artificial intelligence. It’s run by the Cardiff team of Amplyfi, a company that uses tech for business intelligence, and the project aims to  develop technology that will identify new entities that are emerging in the deep web, and especially new relationships between those entities.

Extracting court information for the press

The team behind the Caerphilly Observer are running this project, which will deal with court information (who’s appeared in court, who’s due to appear) that is often either unwieldy or downright inaccessible for journalists. The plan is to gather all this information for Magistrates Courts in Wales and make it available to journalists through a searchable database, which would greatly aid press coverage of local courts.

New ways of telling news stories

What’s the best way to tell a news story? This project is trying to answer this question by looking firstly at how people understand and response to stories in general, and then by designing new journalism techniques that will allow the press to tell stories in the most effective way possible. It’s a radical re-evaluation of a journalistic storytelling tradition that has long worked just on the basis of ‘that’s how we’ve always done it!’.

News in school

This project will design “a pilot for regular news service delivered to pupils within school hours”. The idea is that teachers can use this service to complement their teaching and that a new generation of young people will be introduced to the idea of staying informed.

Journalism by Numbers — 2019 [Virtual] Summer School

by

With Cardiff University buildings closed since March because of the Coronavirus pandemic, the Summer School for the public moved online in June, and included a one-hour session on what datajournalists do.

The Summer School comprised a week of workshops that ranged from radiography and earth sciences to building design and writing for business.

In our rapid run-through the data journalism world, we touched on classic go-to number stories like A&E waiting times and party-political donations as well as how journalists dig up the data in the first place (FOI, web scraping and so on). We looked at visuals done with colouring pencils, graphing cleaner air in Cardiff during lockdown and the ongoing questions around who keeps an eye on the algorithms.

People appeared online for our Journalism by Numbers workshop from around Wales and the UK, but also from Pakistan, Sweden and Nigeria.

Other workshops during the week covered ethics in Artificial Intelligence, copywriting and Google analytics. There was also a session on the ever-interesting Pharmabee project (which launched the Spot-a-bee app this year as part of their bee-mapping project).

Capturing OSINT flags with Cardiff’s Cybersoc

by

Cardiff University’s Cyber Society gave us all its Capture the Flag challenge earlier this year and now has over a thousand players on its leaderboard, many of them sitting on the maximum score 0f 15,000.

The challenges are organised into three streams: ten introductory questions to get you warmed up, 18 tasks for online intelligence gathering and finally a dozen challenges centred around some fictional characters and their online life.

There are no pre-requisites for attempting it — it starts with a “What is OSINT?” question, so beginners are welcome — but it should test most players’ “resilience” (i.e. can you keep playing even though you’ve run out of ideas, patience and any sense that you once knew anything about online intelligence gathering?). At least one of our Computational Journalism students has made it successfully through all the challenges.

The challenges were featured by We Are OSINTCurious on its webcast in March.

SELECT * FROM a day of SQL…

by

This month our students survived a full-day workshop on SQL, moving from the very basics of the syntax to querying datasets or working through some of the better tutorials.

First up was the excellent Select Star tutorial by Zi Chong Kao, which is based on a dataset of US prisoners executed since 1976.

We then looked for newslines in a sqlite database of US babynames (via the command line) and wrote queries in Carto to map a dataset of protected Welsh monuments.

There was more sqlite with a database of shooting incidents involving Dallas police officers, this time via a notebook. And we finished with the Knight Center’s fine SQL-based murder mystery.

Enough there to get you started (or refreshed) with your SQL syntax.

NHS Hack Day returns to Cardiff

by

At the end of semester two our 2019 students set off (armed with a full semester of javascript) for the Cardiff NHS Hack Day.

This regular event moves around the UK and brings together health specialists, technologists and anyone at all who’s got a suggestion about improving any aspect of healthcare.

There’s an overview of the Cardiff event by one of the judges available here and an account of several of this year’s projects here.

Our Computational Journalism students spent two days working on a app called “Can I eat this?” which allows users with dietary restrictions to scan barcodes on packaging see if a product is safe to consume — their app and their presentation to the judges is available on the Hack Day website and the code is here.

 

Above photo by https://www.flickr.com/photos/paul_clarke