Big Data Architecture Stack Layers

The analytics layer interacts with stored data to extract business intelligence.

Big data architecture stack layers.

The big data architecture might store structured data in a rdbms and unstructured data in a specialized file system like hadoop distributed file system hdfs or a nosql database. Without integration services big data can t happen. Logical layers of a big data solution logical layers offer a way to organize your components. Increasingly storage happens in the cloud or on virtualized local resources.

The processing layer is the arguably the most important layer in the end to end big data technology stack as the actual number crunching happens in this layer. Multiple analytics tools operate in the big data environment. The security requirements have to be closely aligned to specific business needs. In this layer analysts process large.

In addition keep in mind that interfaces exist at every level and between every layer of the stack. To empower users to analyze the data the architecture may include a data modeling layer such as a multidimensional olap cube or tabular data model in azure analysis services. Therefore open application programming interfaces apis will be core to any big data architecture. Big data layers as you see in the preceding diagram big data architecture or unified architecture is comprised of several layers and provides a way to organize various components representing.

The layers simply provide an approach to organizing components that perform specific functions. They do not imply that the functions that support each layer are run on separate machines or separate processes. The goal of most big data solutions is to provide insights into the data through analysis and reporting. New big data solutions will have to cohabitate with any existing data discovery tools along with the newer analytics applications to the full value from data.

The layers are merely logical. Some unique challenges arise when big data becomes part of the strategy. The data layer at the bottom of the stack are technologies that store masses of raw data which comes from traditional sources like oltp databases and newer less structured sources like log files sensors web analytics document and media archives.