Introduction to NoSQL and document data model

Relational databases have a long-standing position in most organizations. Based on a tabular data model composed of rows and columns, relational databases – designed back in the 1970s and popularized in the 80s – became the default way of managing data.

The relational model pairs well with traditional client-server business applications that inherently operate on structured data. Classical relational databases follow the ACID property. That is, a database transaction must be Atomic, Consistent, Isolated, and Durable. In a nutshell, this guarantees consistency; every modification on data will transfer the database from one consistent state to another consistent state. 

However as developers build applications for today's digital economy, they are facing demands that stretch the limits of what’s possible with relational systems. These demands include:

• Accelerating developer productivity to keep pace with a constant stream of ideas from the business, compressing application release cycles from months to days and weeks.

• Storing and querying rapidly changing data that arrives in all shapes and sizes – structured, semi-structured, and polymorphic – where defining a flat, tabular schema in advance is not practical.

• Distributing data across multiple servers and regions for application resilience, scalability, and intelligent geo-placement.

Meeting these challenges has driven developer adoption of non-tabular, sometimes called “NoSQL” databases over the past decade. There are many flavors of non-relational databases defined by the data model they support : key-value, wide-column, and graph. But it is document databases, based on JSON-like documents, that have risen [1] to become the most popular and widely used alternative to traditional relational systems.

How the Document Data Model is Different

The document data model offers a number of properties that make it flexible and dynamic, but which also enable you to maintain similar levels of schema control that relational databases have traditionally afforded.

Firstly the document model is polymorphic – fields can vary from document to document within a single collection. For example, you can enforce that all customer documents contain the customer ID, name, address, and the date they opened their account. But you may have additional attributes for only some of your customers such as their social media handle, interests, or location data from a mobile app. Documents make modeling such diverse attributes easy for developers, elegantly handling data of any structure.

Secondly, there is no need to declare the structure of documents to the database – documents are self-describing. Developers can start writing code and persist objects as they are created.

Thirdly, when you need to make changes to the data model, the document database continues to store the updated objects without the need to perform costly ALTER TABLE operations, update a separate ORM middleware layer, and coordinate all of these changes across multiple developer, DBA, and Ops teams. Documents allow multiple versions of the same schema to exist in the same table space. Old and new applications can co-exist.[2]

Through these advantages, the flexibility of the document data model is well suited to the demands of modern application development practices. The document model works well with use cases such as catalogs, user profiles, and content management systems where each document is unique and evolves over time. Document databases enable flexible indexing, powerful ad hoc queries, and analytics over collections of documents.

JSON Documents are everywhere

JSON stands for JavaScript Object Notation. Lightweight, language-independent, and human readable, JSON has become an established standard for data interchange and storage. It is used across the application stack – from frontend web and mobile services, to inter-service messaging, streaming and APIs, through to the backend server and data storage. You don’t have to transpose, serialize and deserialize your data between different formats in each layer of your application stack, making development much more fluid.

Major non-relational databases that can easily store, query, and index JSON data are : MongoDB, Amazon DocumentDB, Amazon DynamoDB and Microsoft Azure CosmosDB.

Case Study: Using JSON APIs in a webpage

We will use JSON API for covid-19 cases and API for weather to add the latest information in this webpage. Every time this page is loaded, it will fetch the APIs to get the latest covid cases in US and the latest Seattle weather. Let’s look at the API data sample for covid cases. Understanding the structure of the key-value pairs in the JSON is crucial to index the required value. Various resources like http://jsonviewer.stack.hu and https://prettydiff.com can format the json data in a form which is easy for analysis. If we see the API data below (acquired from api.covidtracking.com) which is formatted from the two resources, we can analyse the structure easily. Here the key-value pairs are inside an array, so we’ll index them accordingly in our code.

Similarly we’ll access the required values for weather (acquired from weatherapi.com) by analysing its JSON structure.

The code below includes the javascript for fetching JSON API data for covid cases in US and Seattle weather along with the basic html script to add everything in the webpage.

The final output from the script above is as follows :

Endnotes

[1] https://db-engines.com/en/ranking_categories

[2] https://www.mongodb.com/nosql-explained

< Back to Data Warehousing