How to develop a data project without panicking / losing your mind

If you are an NGO, an international organization, a company, a government or just someone with access to an interesting amount of data, anyone can start a data project. Okay, so how to start? We’ll share four lessons we learned working with data so you don’t panic at the first attempt.

Key #1: “Let go”. On how to build a dataset

As Marie Kondo says, order is magical. Letting go of what is not needed in a database will allow us to have a clear and ordered dataset and prevent us from wasting time when trying and using different tools to visualize data. It will also reduce the margin for inexplicable errors ahead.

Some advice:

• First things first: Data alone is not a dataset. The first task is to convert it into a dataset, unifying the type of file. For example, transform and transfer all data into an excel spreadsheet format.
• But pay attention! Not any Excel file is a dataset. A dataset only has data: it does not look cute, and most likely it is not understood at first glance (it is made to be understood by an application or program, not humans). In a dataset there are no colors, no borders, no merged cells.
• It is very important to separate the information, and split any data that is not necessarily related. For example, instead of having two tables in the same Excel file, it is better to create a new file or a new sheet for each table.
• Now, this is between you and the table. Take out any extra metadata or info that is not directly related to the table. If possible, even remove titles and sources, you can save them on another sheet.
• Last tip: make it simple. Review everything again, maybe it is possible that one column can be split into two different data columns. For example, if the same column includes the sex and age of a person, it is best to separate that data into two columns so that they can be processed and crossed independently.

2. Go beyond the numbers

Some data is quite complex to understand, but there is always a way to make data interesting and understandable for someone who doesn’t know anything about an issue.

One way is to tell stories born within the data.

How do you find these stories? It’s useful to have a talk with someone who doesn’t know anything about the subject but is interested; tell him what we researched, what we know, what the data is about. This mix between the informed perspective and the one that’s maybe naive but interested, usually brings up findings and insights that can easily go unnoticed by someone that is too familiar with the data.

For each goal of the Government Commitments project, we matched the progress with a text that explains what the commitment is about, why it is important and what will be done to achieve it. In addition, we included videos, animated gifs and infographics to add context, so that you can understand what’s most important of the commitment at a glance.

3. Many tools available to visualize data

A few years back, only specialized professionals could use tools to create a nice visualization, but now, new and easy-to-use tools appear every day. Today it is possible to produce a dataviz piece quickly and easily. Infogr.am or the free versions of Tableau and Highcharts are great allies to create our own visualizations. In most cases, it’s just about uploading one of the datasets we prepare and choosing the best way to display them. While this has lots of advantages, it does come with some disadvantages.

4. Open data

Opening the data is, without a doubt, the part that generates more resistance and is most conflictive, although it’s also the most rewarding one.

Open data means publicly offering the datasets we generate and have so anyone can download them, analyze them and use them in any way they want. For governments, this is increasingly an obligation. Exposing the details of the government management increases its transparency before its citizens, it enables them to verify if what is said matches what is done and, thus, reducesthe probability of corruption. But you cannot open data in any way. There are standards that data must meet to be considered truly open. For example, it is not the same to upload information in a pdf, a format that is almost an image and cannot be copied or pasted, than to upload that same information sorted in a table and in an open format file (.csvs or .xml), a friendly format for data processors. In the case of the Government Commitments of the City of Buenos Aires, each graphic is accompanied by its dataset, which can be downloaded in .csv.

For the rest of us mortals, individuals, companies or organizations, opening data is not an obligation, but it can be a very smart way to relate to the world and put the data to work.

By Ana Soffietto. Illustrated by Marcelo Morán.