Civic Data Education Series
  • Civic Data Education Series
    • About
    • License
    • Instructor Guide
    • Feedback
  • Segment 1: Introducing Civic Data
    • Module 1.1: Introduction to Open Civic Data
    • Module 1.2: The Lifecycle of Open Civic Data
    • Module 1.3: A Critical Approach to Open Civic Data
  • Segment 2: Understanding Civic Data Ecosystems
    • Module 2.1: Identifying Civic Data Intermediaries
    • Module 2.2: Introduction to Civic Data Ecosystems
      • Module 2.2 Activities
    • Module 2.3: Assessing the Civic Data Needs of Communities
  • Segment 3: Preparing Libraries for Sharing their Data
    • Module 3.1: Selecting and Sharing Open Data
      • Module 3.1 Activities
    • Module 3.2: Protecting Privacy
    • Module 3.3: Metadata for Open Civic Data
    • Module 3.4: Data Documentation for Open Civic Data
      • Modules 3.3 & 3.4 Activities
    • Module 3.5: File Formats for Open Civic Data
  • Segment 4: Community Engagement through Civic Data
    • Module 4.1: Defining Data Visualization
    • Module 4.2: Telling Stories with Data
    • Module 4.3: Outreach on Open Civic Data
  • Segment 5: Using Civic Data
    • Module 5.1: Asking Data Driven Questions
    • Module 5.2: Answering Data Driven Questions Computationally
Powered by GitBook
On this page
  • Introduction
  • Instructional Materials
  • Video
  • Script and Slide Deck
  • Activities
  • Activity 1: Exploring File Formats
  • Activity 2: Exploring Data Portals and File Formats
  • Resources
  1. Segment 3: Preparing Libraries for Sharing their Data

Module 3.5: File Formats for Open Civic Data

PreviousModules 3.3 & 3.4 ActivitiesNextSegment 4: Community Engagement through Civic Data

Last updated 2 years ago

Introduction

This module introduces the concepts of open and machine-readable file format. We build an understanding of the importance of using open and machine-readable formats when sharing civic data and the benefits to data users.

Guiding Questions

  • What are open file formats?

  • What are machine-readable file formats?

  • How do these formats support use?

  • What are examples of open file formats?

  • How can we make data available?

Vignette

The librarian is working with their local open data portal to make the Wi-Fi data available to the public. When the IT department provided the data to the librarian, it was in an Excel spreadsheet format. The manager of the open data portal explains that open file formats are preferred and requests that the librarian make the data available in the portal as a .csv file.

Instructional Materials

Video

Video Files

Script and Slide Deck

Script and Slide Deck Files

Activities

Activity 1: Exploring File Formats

Overview: Open file formats are those file formats with published and open documentation. This open documentation allows for developers to create software that can read the file format. This means that the file format is not reliant on a single or limited set of software programs to read it.

In this exercise, we will examine the digital files on one of our own devices – a laptop, phone, tablet, desktop. We will consider the file formats that we create and interact with and how they may support or impede access and reuse.

This activity can be done individually, with time to debrief and share after completion.

Supplies:

  • Device with files

  • Pen or pencil

  • Paper for note taking

Time: 30 minutes

Activity:

  1. Access your files

For this exercise, you will need access to a device with files that you have created and downloaded. To narrow your focus, you can choose a folder or subset of your files on your device. You will want to be able to identify the file format of your files, using the file extension (e.g. .csv, docx, .mp4).

2. Inventory your files

Inventory the file formats on your device or in the selected folder on your device, noting:

  • The file extension (set of characters at the end of the filename that support us in recognizing the file format)

  • Which file formats you recognize and which are unfamiliar to you

  • The program that you would use to interact with the file

3. Classify your files

This comparison will support you in identifying which file formats you interact with are “open” and which we would characterize as proprietary or restricted in nature.

Notate open versus proprietary versus uncertain with a symbol (e.g. star for open format).

4. Reflect on your inventory

Consider: If you were to provide a hard drive with your files to a friend or colleague, would they have any difficulty accessing the file formats because of software dependencies?

5. Reconvene (if working in group setting)

If completing this activity in a group setting, reconvene together. Discuss what you observed about the nature of the file formats you interact with.

Activity Files

Activity 2: Exploring Data Portals and File Formats

Overview: In this exercise, we will examine the file formats found through data portals. This activity will build our understanding of the standards used for sharing open civic data and the expectations for file formats that data portal managers communicate to data publishers. This activity can be done individually or in pairs.

Supplies: Computer or Internet-connected device Pen or pencil Paper for notetaking

Time: 20-30 minutes

Activity:

  1. Select Data Portal

2. Examine three open civic datasets

Browse the data portal and locate three datasets and the accompanying data files. Inventory the data formats associated with the data files, using the file extensions as an indicator.

  • Are the same data files available in more than one format (e.g. both a csv file and an Excel (.xls or xlsx) file format)?

3. Explore the data portal guidance

Now, look to see if the data portal has guidance for data publishers on preparing data for sharing. On some portals, this guidance may be under an “About” tab. Consider:

  • Is there guidance about preferred file formats for data sharing? If so, what are the preferred formats?

  • Are these recommendations reflected in the data files you reviewed in the portal?

4. Debrief [if working in a group setting]

If completing this activity in a group environment, reconvene to share observations about file formats used to share open civic data in portals. What file formats did your group observe? Did the data portals have guidance for data publishers regarding file formats? If so, what were the recommended file formats?

Activity Files

Resources

Compare your file format inventory to the .

Individually or in pairs, locate an open data portal from your local city, region, state, or another location of interest to you. , a searchable catalog of open data portals, can be a helpful starting point for locating a data portal for this exercise. You can return to a data portal that you’ve explored for another module activity.

Are the formats open or proprietary file formats? If you’re uncertain, use the and the format tables available on the

The European Data Portal provides access to data shared in EU-based open data portals. To support open data initiatives, the European Data Portal created a series of modules on open data, including a module on choosing file formats. The module creators emphasize the value of the csv file format for tabular data.

list of open file formats on Wikipedia
DataPortals.org
Wikipedia list of open data formats
FOSS Open Standards/Comparison of File Formats resource
European Data Portal Module: Choosing the Right File Format for Open Data
41MB
Module3.5_CaptionedRecording.mp4
Module 3.5 Captioned Recording: File Formats for Open Civic Data MP4 File
38MB
Module3.5_UncaptionedRecording.mp4
Module 3.5 Uncaptioned Recording: File Formats for Open Civic Data MP4 File
361KB
Module3.5_Script.docx
Module 3.5 Script: File Formats for Open Civic Data Word Document File
106KB
Module3.5_Script.pdf
pdf
Module 3.5 Script: File Formats for Open Civic Data PDF File
8MB
Module3.5_Slides.pptx
Module 3.5 Slides: File Formats for Open Civic Data PowerPoint File
2MB
Module3.5_Slides.pdf
pdf
Module 3.5 Slides: File Formats for Open Civic Data PDF File
357KB
Module3.5 _Activity1.docx
Module 3.5 Activity 1: Exploring File Formats Word Document Format
94KB
Module3.5 _Activity1.pdf
pdf
Module 3.5 Activity 1: Exploring File Formats PDF Format
357KB
Module3.5 _Activity2.docx
Module 3.5 Activity 2: Exploring Data Portals and File Formats Word Document File
106KB
Module3.5 _Activity2.pdf
pdf
Module 3.5 Activity 2: Exploring Data Portals and File Formats PDF File
LogoAnalyzable Data in Open Formats (Principles 5 and 7) - Open Government Data: The Book
Joshua Tauberer unpacks two principles from the 8 principles of open data, which were published in 2008. In this resource, Tauberer describes the meaning of "machine-readable," which he frames as "analyzable," and non-proprietary. He provides examples of data file formats that align with these categories. Citation: Tauberer, Joshua. “Analyzable Data in Open Formats.” In Open Government Data: The Book, 2nd edition (2014).
https://opengovdata.io/2014/analyzable-data-in-open-formats-principles-5-7/
Module 3.5 Captioned Video: File Formats for Open Civic Data
File Formats
The Open Knowledge Foundation offers a useful introduction to the value of open file formats and an overview of common file formats used in open data portals. Citation: Open Knowledge Foundation. "File Formats." In Open Data Handbook.
http://opendatahandbook.org/guide/en/appendices/file-formats/