Total views : 445
Quick Matching of Big Binary Data: A Probabilistic Approach
Given two sets of binary data, how can we determine if the data are dissimilar? The simplest technique is to simply subtract the two sets or to calculate the correlation between them. Both of these methods –as well as other methods– require some type of similarity operation to be applied to all points of the data. This implies that as the data becomes big, more processing time is required. In this paper, we present a novel approach to matching using a probabilistic model that requires a few number of points –and not all points – to be compared between two data sets to detect dissimilarity. Furthermore, the model is size invariant; big data can be matched just as quickly as matching small data. The similarity between the data can also be measured to a good degree by repeating the matching process several times.
Big Data, Binary Data, Binary Matching, Pattern Matching, Probabilistic Model.
- Montgomery DC, Runger GC. Applied statistics and probability for engineers. John Wiley & Sons; 2010.
- Barnea D, Silverman H. A class of algorithms for fast digital vector registration. IEEE Transactions on Computers. 1972 Feb; c-21(N2):179–86.
- Choi S, Sung-Hyuk C, Tappert CC. A survey of binary similarity and distance measures. Journal of Systemics, Cybernetics and Informatics. 2010; 8(1):43–8.
- Consonni V, Todeschini R. New similarity coefficients for binary data. Match-Communications in Mathematical and Computer Chemistry. 2012; 68(2):581.
- Hamming RW. Error detecting and error correcting codes.
- Bell System Technical Journal. 1950; 29(2):147–60. DOI: 10.1002/j.1538-7305.1950.tb00463.x.
- Sokal RR, Michener CD. A statistical method for evaluating systematic relationships. Science Bulletin. 1958; 38:1409– 38.
- There are currently no refbacks.
This work is licensed under a Creative Commons Attribution 3.0 License.