The Issue of Management of Big Data - Essay Example

Data Warehousing and Big Data With the data explosion in the recent years, the information and technology sector continues to confront the issue of management of big data. Big data refers to data that has a loose structure and has a distributed aggregation. In most cases, the data is usually incomplete and the accessibility is a big issue. Traditional methods have become inadequate to process big data, thereby creating an avenue for continuous development of data processing techniques. Large web companies such as Google have experienced a serious problem due to the big data that is generated on daily basis. This called for interventions such as the MapReduce meant to support distributed computing that involves the use of large amounts of data. Most of the activities that involve the use of big data are either for analytical or transactional applications. Data warehousing was the conventional method of helping manage data and provide information that helped in decision-making. Warehousing of big data has been questioned and it seems to be on the brink of inviting another data processing strategy. It is the wish of every organization to store up data and use the same data in decision making of the organization. Data mining entails the process of data retrieval and analysis in order for it to make sense for use in organizational decision-making. Data mining software have been developed to facilitate the analysis of data from a number of perspectives based on the needs of the users. In addition, they are able to categorize the data and develop summaries that will predict relationships. Data mining is a very important process since it helps in segregation of data so that an organization is able to keep a keen eye on the most important information. The essence of having the data mining software developed is based on the size of data that organizations struggle with; mainly in gigabytes and terabytes. Raw data is deficient and cannot be relied upon by business owners and the managements in their decision-making (Brown & Kros, 2003). This is because it has a lot of junk information that may not be necessarily necessary. Data mining has constantly evolved with time. its main essence over the years has been to obtain information from databases in order to facilitate the analysis of business progress. In the 1960s, the main concern by many business owners pertained issues to do with revenue. Data mining entailed mainly data collection where the technology under application was mainly the use of discs, computers, and tapes. With advancements in technology, data mining progressed to data access in the 1980s; in this case, many decision-makers focused on the unit sales and were facilitated by the development of computers with a bigger capacity and relatively cheaper. Preceding data access was the data warehousing technology, which was coupled with a decision, making support. This provided an opportunity for data analysis to be made at a more specific level in addition to online analytical processing (OLAP). Based on the use of better computing systems with advanced computer algorithms, the data mining technology is the latest advancement (Wu, Zhu, Chen & Wang, 2009). Data mining and OLAP are technologies advanced to facilitate business intelligence; however, they differ in some respects. OLAP is a business intelligence technology that is applied in the analysis databases that are multi-dimensional in nature. On the other hand, data mining is a technology that entails extraction of data that has some meaning and useful from raw data. Since businesses have been in need of understanding the market, data mining has been applied in order to get the trends in the market. In data mining, there are four processes that are involved, that of clustering data, classification of data, regression and the association. For the case of OLAP, there are systems that are designed in a way that they are able to answer multidimensional queries. As opposed to data mining, the main roles of OLAP include marketing, forecasting, and budgeting among other related functions. The operation on the data sets the difference between OLAP and data mining. While data mining provides ratios, influences and trends for a specific data, the OLAP deals with data aggregation which encompasses the analysis of multi-dimensional data in order to provide summarized data (Rupnik, Kukar, & Krisper, 2007). Through data mining, one is able to know important things while at the same time it is capable of foretelling what may happen in the future. Data mining uses modelling to tell the user of the data what is likely to happen or what they did not know. In this regard, the model uses the data that has known implications and uses the same to predict the answers to the scenario in question. Modelling ensures that big data is able to be handled effectively and used in decision-making for an organization. For example, it can allow for segregation of data based on various demographics such as age and gender among others. Various data mining technologies have been advanced. For the analytical techniques, most of the data mining has adopted the use of mathematical algorithms. Data mining has used tools such as the artificial neural networks; these are predicative models. Other types of data mining technologies include, the use of genetic algorithms, decision trees, and rule induction and nearest neighbor (Alexander, n.d). These data mining tools are very important in the process of data retrieval and use for decision making in an organization. These tools are significant for their input in analyzing historical data and using the same to develop the way forward for the organization. The data that is stored in the warehouses is sifted and used by financial analysts to identify relationships, trends, and any anomalies that is important for the evaluation of the organization. Using data mining tools, a business is able to track its customer loyalty, sales trends and develop marketing strategies among others. In particular, data mining is very essential in the detection of fraud, analysis of market share, market segmentation, and direct marketing input to the business. Data warehouses provide data storage, which has provided organizations with an opportunity to have their big data kept safely. These warehouses are able to consolidate data that comes from different locations so that it is capable of being stored in a common platform where it can be retrieved when required. Big data pose a challenge to the data warehouses creating an opportunity to develop data compression strategies. Through the storage of data in the warehouses, the management of an organization is capable of check on either the transactional or analytical data in order to assess the well-being of the organization (Devlin, 2011). Data mining raises some ethical concerns that should be addressed before engaging in the process. One of the major concerns is the privacy of the individuals and the organizations. An individual’s privacy may severely affected when the information that is stored in a database about them is retrieved (Alexander, n.d). For example, an employee of an organization may be badly exposed since data mining is able to show all the data that person has interacted with and all that they have posted. Companies have been on the outlook of information that will help them track the consumer trends, such infringes the privacy rights of an individual. However, this issue can be dealt with by developing technologies that protect individual privacy. This can be done through aggregation of data or through creation of algorithms that are able to follow on the patterns of data without revealing the identities of the individuals. In conclusion, data mining poses to be an important technology that will facilitate the management of big data in order for it to have some meaning to an organization. Through data mining trends that can be used by decision makers are identified and used. References Alexander, D. (n.d). Data Mining. Retrieved from http://www.laits.utexas.edu/~anorman/BUS.FOR/course.mat/Alex/ Brown, M. L., & Kros, J. F. (2003). Data mining and the impact of missing data. Industrial Management + Data Systems,103(8), 611-621 Devlin, B. (2011). Will warehousing survive the advent of big data? Retrieved from http://radar.oreilly.com/2011/01/data-warehouse-big-data.html Rupnik, R., Kukar, M., & Krisper, M. (2007). Integrating data mining and decision support through data mining based decision support system. The Journal of Computer Information Systems, 47(3), 89-104. Wu, X., Zhu, X., Chen, Q., & Wang, F. (2009). Ubiquitous mining with interactive data mining agents. Journal of Computer Science and Technology, 24(6), 1018-1027. Read More