Reading & Parsing JSON Data With Python: Tutorial

JSON is a common standard used by websites and APIs, and even natively supported by modern databases such as PostgreSQL. In this article, we’ll present a tutorial on how to handle JSON data with Python. Now, let’s start with the definition of JSON. 

What is JSON?

JSON, or JavaScript Object Notation, is a format that uses text to store data objects. In other words, it is data structures for representing objects as text. Even though it’s derived from JavaScript, it has become a de facto standard of transferring objects. 

This format is supported by most popular programming languages, including Python. Most commonly, JSON is used to transfer data objects by APIs. Here is an example of a JSON string:

{
   "name": "United States",
   "population": 331002651,
   "capital": "Washington D.C.",
   "languages": [
      "English",
      "Spanish"
   ]
}

In this example, JSON data looks like a Python dictionary. Just like dictionaries, JSON contains data in key-value pairs. However, the JSON data can also be a string, a number, a boolean, or a list.

Worth noting that JSON data can also be parsed in numerous ways. One of them is to parse nested JSON. Doing so allows you to manipulate data elements that are otherwise nested within complex data structures in JSON. Luckily, Python has a built-in json module; therefore, parsing nested structures becomes a significantly more manageable task.

Before JSON became popular, XML had been the common choice to represent data objects in a text format. Here is an example of the same information in the XML format:

<?xml version="1.0" encoding="UTF-8"?>
<country>
   <name>United States</name>
   <population>331002651</population>
   <capital>Washington D.C.</capital>
   <languages>
       <language>English</language>
       <language>Spanish</language>
   </languages>
</country>

As evident here, JSON is lightweight. This is one of the primary reasons why JSON is so popular and why JSON syntax is such an efficient data exchange format. If you want to read more about the JSON standard, head over to the official JSON website. 

JSON in Python

Python supports JSON data natively. The Python json module is part of the Standard Library. The json module can handle the conversion of JSON data from JSON format to the equivalent Python objects such as dictionary and list. The JSON package can also convert Python objects into the JSON format. Crucially, JSON also allows you to write custom encoders and decoders. As a result, it becomes easy to parse JSON responses from APIs.

The json module provides the functionality to write custom encoders and decoders, and there is no separate installation needed. You can find the official documentation for the Python JSON module here.

In the remainder of this tutorial, we will explore this package. We’re going to convert JSON to dictionary and list and the other way round. We’ll also explore how to handle custom classes.

Converting JSON string to Python object

JSON data is frequently stored in strings. This is a common scenario when working with APIs. The JSON data would be stored in string variables before it can be parsed. As a result, the most common task related to JSON is to parse the JSON string into the Python dictionary. The JSON module can take care of this task effectively, thus making this specific parsing JSON in Python task rather simple.

1. Importing JSON. The first step would be importing the Python json module. This module contains two important functions – loads and load.

Note that the first method looks like a plural form, but it is not. The letter ‘S’ stands for ‘string’. 

2. Using the load function. The helpful method to parse JSON data from strings is loads. Note that it is read as ‘load-s’. The ‘s’ stands for ‘string’ here. The other method load is used when the data is in bytes. This is covered at length in a later section.

Let’s start with a simple example. The instance of JSON data is as follows:

{
   "name": "United States",
   "population": 331002651,
}

3. Storing JSON string. JSON data can be stored as JSON string before it is parsed. Even though we can use Python’s triple quotes convention to store multi-line strings, we can remove the line breaks for readability.

# JSON string
country = '{"name": "United States", "population": 331002651}'
print(type(country))

The output of this snippet will confirm that this is indeed a JSON string:

<class 'str'>

We can call the json.loads() method and provide this string as a parameter.

import json

country = '{"name": "United States", "population": 331002651}'
country_dict = json.loads(country)

print(type(country))
print(type(country_dict))

The output of this snippet will confirm that the JSON data, which was a string, is now a Python dictionary.

<class 'str'>
<class 'dict'>

This dictionary can be accessed as usual:

print(country_dict['name'])
# OUTPUT:   United States

It is important to note here that the json.loads() method will not always return a dictionary. The data type that is returned will depend on the input string. For example, this JSON string will return a list, not a dictionary.

countries = '["United States", "Canada"]'
counties_list= json.loads(countries)

print(type(counties_list))
# OUTPUT:  <class 'list'>

Similarly, if the JSON string contains true, it will be converted to Python equivalent boolean value, which is True.

import json
 
bool_string = 'true'
bool_type = json.loads(bool_string)
print(bool_type)
# OUTPUT:  True

4. The following table shows JSON objects and the Python data types after conversion. For more details, see Python docs.

Now, let’s move on to the next topic on parsing a JSON object to a Python object.

Converting JSON file to Python object

Making Python read JSON files to parse JSON data into Python data is very similar to how we make Python parse the JSON data stored in strings. Apart from JSON, Python’s native open() function will also be required.

Instead of the JSON loads method, which reads JSON strings, the method used to read JSON data in files is load(). 

The load() method takes up a file object and returns the JSON data parsed into a Python object.

To get the file object from a file path, Python’s open() function can be used.

Save the following JSON data as a new file and name it united_states.json:

{
   "name": "United States",
   "population": 331002651,
   "capital": "Washington D.C.",
   "languages": [
      "English",
      "Spanish"
   ]
}

Enter this Python script in a new file:

import json

with open('united_states.json') as f:
  data = json.load(f)

print(type(data))

Running this Python file prints the following:

<class 'dict'>

In this example, the open function returns a file handle, which is supplied to the load method.

This variable data contains the JSON as a Python dictionary. This means that the dictionary keys can be checked as follows:

print(data.keys())
# OUTPUT:  dict_keys(['name', 'population', 'capital', 'languages'])

Using this information, the value of name can be printed as follows:

data['name']
# OUTPUT:  United States

In the previous two sections, we examined how JSON can be converted to Python objects. Now, it’s time to explore how to convert Python objects to JSON.

Converting Python object to JSON string

Converting Python objects to JSON objects is also known as serialization or JSON encoding. It can be achieved by using the function dumps(). It is read as dump-s and the letter S stands for string.

Here is a simple example.  Save this code in a new file as a Python script:

import json

languages = ["English","French"]
country = {
    "name": "Canada",
    "population": 37742154,
    "languages": languages,
    "president": None,
}

country_string = json.dumps(country)
print(country_string)

When this file is run with Python, the following output is printed:

{"name": "Canada", "population": 37742154, "languages": ["English", "French"],
 "president": null}

The Python object is now a JSON object. This simple example demonstrates how easy it is to parse a Python object to a JSON object. Note that the Python object was a dictionary. That’s the reason it was converted into a JSON object type. Lists can be converted to JSON as well. Here is the Python script and its output:

import json

languages = ["English", "French"]

languages_string = json.dumps(languages)
print(languages_string)
# OUTPUT:   ["English", "French"]

It’s not just limited to a dictionary and a list. string, int, float, bool and even None value can be converted to JSON. 

Refer to the conversion table below for details. As you can see, only the dictionary is converted to json object type. For the official documentation, see this link.