Why Traditional Databases Aren't Going Away Anytime Soon

|

Data volumes continue to explode, creating both opportunities and challenges for higher education. On the one hand, big data analytics can provide new insights that improve decision-making across operational areas, from recruitment to academic advising to facilities and foodservice. However, many schools are grappling with how to store and manage all that data, much less analyze it in real time.

More than 80 percent of the data stored by the typical organization is unstructured or semi-structured, and the types of data being processed and stored are becoming increasingly diverse. The difficulties associated with managing unstructured data is driving increased interest in next-generation database technologies such as NoSQL.

NoSQL doesn’t store data in columns and rows according to a fixed schema. Instead it uses various models — including document, graph, tabular and key-value — that allow you to add data on the fly. NoSQL platforms are designed to handle large volumes of data that change rapidly, and can offer better performance than traditional databases in certain contexts. In addition, NoSQL databases can be scaled horizontally across multiple servers, making it easier and more cost-effective to accommodate data growth.

Another alternative database technology is Hadoop, which is designed to extract and analyze information that does not fit neatly into traditional database schemas. Once a small, open-source project, Hadoop has become a popular analytics engine for large datasets, used by the likes of Facebook, Twitter and eBay to crunch data. It enables batch processing of structured and unstructured data at massive scale using commodity hardware.

According to a report by Allied Market Research, the NoSQL market in North America is expected to see a compound annual growth rate (CAGR) of more than 40 percent through 2020. MarketsandMarkets expects the Hadoop big data analytics market to see a 43 percent CAGR through 2021, much of it driven by consulting and development services as organizations begin to adopt analytics tools.

But while NoSQL and Hadoop are seeing increased interest, they account for just 3 percent of the overall database market, according to IDC. Traditional relational databases aren’t going away anytime soon, for several reasons:

  • Organizations have made significant investments in relational databases and underlying infrastructure. In addition, there are countless applications in production that are designed with a relational database backend.
  • IT teams have deep skill sets in relational databases and well-established processes for managing and supporting them. The move to a next-gen database represents a massive cultural change.
  • Most database workloads are still transactional, and structured data volumes are growing. In a recent survey by Datos IO, 59 percent of IT professionals said they expect the size of their organization’s database to double within the next two years.
  • Relational databases represent a mature technology that is proven to support mission-critical applications. NoSQL is still in its infancy, and the immaturity of Hadoop is restraining the growth of that market.
  • NoSQL and Hadoop are great for granular analyses of unstructured data. However, relational databases can be easier to use when analyzing aggregated and transformed data.

The upshot of all this is that organizations tend to add next-gen databases to their environments to support specific use cases. The relational database, that workhorse of the data center, keeps doing its job.