Storage Size Estimation for Schemaless Big Data Applications: A JSON-based Overview

Abstract

Numerous technologies have been proposed for storing big data on the Cloud platform. However, choice of these technologies is always application specific. Determining a strong model is a perplexing task, which makes it necessary for the architects and designers to review the requirements and choose a solution. This paper presents 14 data models available in the market. Above all, there are more than 45 database solutions available in the market, which can be categorized into one of the data models each of which is applicable to its own set of use cases (However, there are few products, which could not be categorized into any of these 14 data models). Contributors have figured out that while storing schema-less information, the size of data stored in the database is higher than the original size. Metadata information and physical schema are the two responsible factors for such a high amount of storage requirement. Mathematical models and experimental evaluations conducted show that MongoDB requires storage space many times more than the original size of data. A storage space estimation equation for JSON based solutions has been suggested, which can compare the storage requirement size using space required by CSV as a base. This may be used to decide an approximate amount of storage space required by the application, before buying a storage space on the Cloud environment.