Pattern Recognition Using Neural Network - Essay Example

Pattern Recognition Using Neural Network Introduction Technology is a journey that never solves a problem without creating many more and this is thebasic motivation for all sorts of innovations. Today we stand on a platform where ‘impossible’ means ignorance. Pattern recognition is one of the most challenging processes in today’s technology that includes character recognition, handwriting identification and facial image analysis etc. Above two are common letters of English alphabet. Task of a pattern recognizer is to characterize these two images as ‘a’ and ‘b’, no matter how they are aligned with reference axis unless and until they are not getting conflicted with other characters. Last line means to say that if the letters are placed in some tilted position, then also algorithm must tackle this problem. Pattern Recognition or Optical Character Recognition (OCR) is a pipelined process consisting of several stages in proper sequence. They are shown in figure 2. Starting from raw data till the output, at each and every stage data is processed and transformed from a low level sub-symbolic representation to a higher level symbolic representation. Each character is represented as a combination of pixels. All pixels together make a huge feature vector. Total number of pixels is equal to wh where w is the number of pixel in width side and h is the pixels present in height. Figure 3 depicts the way pixel forms one particular character. xi is the fraction of ink in pixel i. Classifier must be adaptive (generalize) in nature so that it can be able to recognize patterns encountering first time. A typical character image is 6464 pixels large and for each such pixel 256 grey values are required making feature space large. For training a recognizer hence, requires huge amount of data to fill this vast space. In order to reduce the dimension space Principal Component Analysis is mostly used which transforms into lower dimension space (Yeung & Ruzzu, 2001). OCR also should make a distinguishing between ‘O’ and ‘6’. Figure 4 shows one case example. If t/b comes smaller that means letter is ‘O’ otherwise ‘6’. A good algorithm must define the tolerance level (T) adequately. Other examples of such cases are letter ‘q’ and digit ‘9’. There are various algorithm or computer processes available for pattern recognition. One such example is Brian Sanderson’s Pattern Recognition (PR) Algorithm. Every patter is identified according to three systems of notation: 333 Conway Thurston Notation. P3 The International Union of Crystallography notation. S333 The Montesinos Notation First identify the maximum rotation number whether it is 1, 2, 3, 4 or 6. Then check any mirror is present or not (m). Is there any indecomposable glide reflection (g)? And finally whether there is any rotation axis on the mirror present or not. Other than this Genetic Algorithm also works as PR. Selection of patterns play an important role in PR process as it determines the accuracy of algorithm, its learning time, and the necessary number of samples. Best selection of feature plays important role at the time developing classifiers. The problem comes out more difficult when number of features become very large. Genetic Algorithm (GA) gives better result in that. As they are effective in rapid global search of large, nonlinear and sparsely spaced points, GA is applied for feature recognition problem. It combines different optimization problem into a single formulation problem (Morita). Most effective approach today for OCR is Neural Network based recognition. Neural Network: An Overview A Neural network is a massively parallel distributed processor made up of simple processing units, which has a natural propensity for storing experimental knowledge and making it available for use. It resembles the brain in two respects: 1. Knowledge is acquired by the network from its environment through a learning process. 2. Interneuron connection strengths, known as synaptic weights, are used to store the acquired knowledge (Aleksander &Morton, 1990). Benefits of Neural Networks (NN) are: Nonlinearity: Most of the signals encountered in physical world are nonlinear. Hence to tackle those signals it is necessary should behave nonlinearly. Input-Output Mapping: During training of NN, they are given several training samples consisting inputs and their corresponding outputs. Thus the network learns from the examples by input–output mapping. The same becomes most important factor after training when it is used for some particular task. In pattern Recognition case for example, after training when they are asked to determine one different shaped object then their previous mapping works out to find out actual pattern. Adaptivity: A NN can be designed to change its weights automatically in accordance with the surrounding environment. This Adaptivity property makes it robust independent of the specific environment where it was trained. As neural networks are in massive and parallel form hence it has the flexibility of getting implemented by Very Large Scale Integration (VLSI) technology. NN is somewhat fault tolerant. One example related to that statement is that, if by somehow one link or connection between any two neurons fails then due to its distributed nature of computing performance does not degrade much. Neural Network has several architectures each one being linked to on specific type of learning process. Its different classes of architectures are stated below: 1) Single-layer Feed forward Networks: Layers are nothing but the fragmentation of total network into several structure where each of neuron in one layer processes the signals at the same instant. In single layer feed forward network we have an input layer of source nodes and one output layer of neurons known as computation nodes. 2) Multilayer Feed forward Networks: In multilayer network, other than input and output there is also hidden layer(s). Their computation nodes are also called hidden neurons. It can perform higher order statistics after incorporating another layer of neuron. Multilayered neural network can be either of fully connected or partially connected type. In fully connected network each node in each of the layer is connected to all other nodes of adjacent forward layer. But in partially connected one that statements does not hold true. 3) Recurrent Networks: Only Feed forward connection structure is invalid in Recurrent Network. Here feedback paths are incorporated. It can either have hidden neurons or not but feedback must be there. Feedback property enhances its learning capability and performance. Figure below shows the forms of different neural network structure (Haykin 1999). Neural Network & Pattern recognition: Most essential requirement of an OCR is an efficient algorithm. Using neural network for character recognition gives higher accuracy as well as speed. For detecting single size and single font near about 3000 inputs are required with 20 neurons in fist layer and 94 neurons in the second or output layer (Hadar I, 1995). Network is fully connected and feed forward type. The nonlinearity of each neuron is specified by sigmoid function, For training Back propagation algorithm is used which is a steepest descent algorithm that adjusts its weight after minimizing error function (Rumelhart D. E, 1986). Where di and yi mean the desired and actual outputs of the ith neuron in the last layer in response to the pth input pattern. N is the number of neurons in the output layer. The data consisted of pre-segmented (though not all perfect) character images, scanned at 400 d.p.i. and in 8-bit gray scale resolution. Database is kept for training purpose. Data is comprised of 94 digitized character images with one-to-one correspondence between each training data and each member of the 12 point Courier font character set. From English library 1,072,452 character images are evaluated as testing data for NN. In order to reduce noise sensitivity and make it size and position independent normalization and preprocessing is done before feeding it to NN. Reduction of background noise sensitivity is also known as thresholding. After thresholding image is centered by positioning the centroid of image. Centroid of image can be obtained from following formula. After normalization is done next step comes is conversion from two dimensional space to vector by concatenating each row of two-dimensional pixel array. Another additional step is performed to dither the centroid of the two-dimensional input image. It is know as Centroid dithering. After centering and scaling, the input image is displaced randomly and independently in both the horizontal and vertical directions over the range of [-2, +2] pixels in each dimension; the image is shifted at random in one of twenty-five possible displacement positions. The resultant image is then converted into a vector, normalized, and fed into the network as previously described. It creates many images from a single one. Neural network is exposed to the same character at different displacement positions, making the recognition system invariant to input displacements. This approach also enables the network to tolerate width variations in character strokes. Training is one of the most important steps for one NN. Here initial learning parameter is kept at μ=10. Learning process is governed by error energy profile, MSE which can be given by, Where C is the cost function. The neural network is trained with 430,000 iterations, and the final M.S.E is approximately 10-6. Figure below shows one MSE profile during training. For a good algorithm design must be robust enough from environmental variations or any other segmentation error that may occur. Due to bad representation of image letters may get changed its original shape. For example ‘g’ may be treated as ‘9’. To avoid this sort of error NN must be designed with proper threshold so that it can take decision adequately. Abbey Reader & Other Application of NN Abbey reader works on the principle of OCR to convert scanned characters in jpg format to ASCII format Word document which computer can easily handle. Scanner has given us potential to access any document or image whereas the remaining important task is done by Abbey Reader where we can extract each pieces information in a scanned image, put them into words and then words into sentences, thus enabling us to access and edit the content of the original document. Fine Reader first analyses the structure of document image. It divides the page into elements such as blocks of texts, tables, images, etc. The lines are divided into words and then - into characters. Once the characters have been singled out, the program compares them with a set of pattern images. Program makes several hypotheses and after processing each of the probabilistic hypotheses finally it takes the decision about the actual character. Other than this the software has in built dictionary of 36 languages which gives flexibility to the users for character recognition. Other applications of Neural Network include (Sivanandam S. N, 2006): Bioinformatics: especially in protein sequencing and protein folding problem. Forecasting of business and financial market. Bankruptcy Forecasting Healthcare services: clinical diagnosis, image analysis and interpretation, signal analysis, and drug development. Intrusion detection to secure company networks, detect all real time intrusions and execute to stop the attack. Communication field, to detect any error is present in received data stream and if so then correct them. Conclusion Neural Network brought a revolution in technology field. It is an efficient tool that can do computation of any computable function. Anything that is represented as mapping between vectors spaces can be approximated to arbitrary decision by feed forward NNs. If they are properly trained they can be used in environment where the statistical properties change with time. In our daily life its impact bears a lot; starting from healthcare to business field, forecasting to pattern recognition usage of Neural Network is enormous. References Yeung K. Y., Ruzzo W. L. An Empirical Study of Principal Component Analysis for Clustering Gene Expression Data. May 2, 2001. Morita M et. al. Unsupervised Feature Selection Using Multi-Objective Genetic Algorithms for Handwritten Word Recognition. Aleksander &Morton, 1990, An Introduction To Neural Computing Van Nostrand Reinhold Co. New York, NY, USA Haykin S. Neural Networks: A Comprehensive Foundations. 2nd Edition. Prentice Hall India. 1999. John A. Bullinaria. Introdution to Neural Network 2004. Hadar I. et. al. High Accuracy Optical Character Recognition Using Neural Networks with Centroid Dithering. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 17, No. 2, February 1995. D. E. Rumelhart, G.E. Hinton, and R.J. Williams , “Learning Representation by Error Backpropagation,” In Parallel Distributed Processing, vol. 1, MIT Press, Cambridge, MA, 1986, Chap. 8, pp. 318-362. Sivanandan S. N. Introduction to Neural Networks using MATLAB 6.0. Tata McGraw Hill. 2006 http://www.abbyy.com/ Read More