Soumen et al (2009) agree that data quantity surrounding us is humongous and the amount of information bombarding us is increasing. Making sense of this increasing data volume requires data mining skills and techniques that have evolved with increase in computing power. An allied discipline is Competitive intelligence which is a discipline used for improving market standing, improving strategic thinking - seeing through morass of disinformation and market disruptions and interpreting events without getting emotionally swayed with "pregnant" data. It is about analyzing an opportunity or threat before it has materialized (Reviews 2007).
Finding patterns in data is a common way of analysis. Scientists want to discover the pattern and use the patterns for developing theories that can be extended beyond the concerned data in allied fields. This helps the scientists predict what will happen in newer situations. Intelligence is thus about using the available information in an efficient manner based on picture which may or may not be perfectly clear and exploiting the gleaned intelligence for making strategic decisions. Data mining calls for electronic data storage and using of specified search for pattern identification. Global data doubles by in every 20 months, and increased availability of machines that can digest and process such data have increased opportunities for data mining. Intelligently analyzed data may our only redemption in making sense of the growing data volume. Bits and pieces of information lead to understanding of the big picture. Data mining is using the existing data to solve problems and discovering patterns in data. Consumer shopping data might help in eliciting likely reason for customer loyalty and churn. Also data analysis on same database may identify the reason why customer may be attracted to other product or service thus allowing development of special targeted offers. Data is only useful if it can be analyzed or mined, intelligence is about knowing the identity and acting on signals that analyzed data portrays - it is also acting on information before everyone else sees the same picture (Soumen et al 2009).
Getting information from data
Data mining can be divided into two separate types including predictive data mining which using existing variables in the data set predicts the unknown variables while descriptive data mining focuses on finding patterns that can be interpreted by analysts (Chethan n.d.).
Data has to be cleaned for removing inconsistencies and extraneous information, often data requires consolidation from multiple sources and streamlined to provide pertinent data for the concerned task. Later the data may be transformed so appropriate mining techniques can be applied. This step is usually followed by evaluation and then presentation of gleaned knowledge. Classification of data is used to predict the class of data so that the mode derived can be used for derivation of attributes about the data. Classification data can be represented using IF-Then rules, decision trees or neural networks. Decision tree is like a tree with multiple branches each representing outcomes. Neural network consists of collection of neuron type processing units