Deduplication algorithm based on condensed nearest neighbor rule for deduplication metadata

Deduplication algorithm based on condensed nearest neighbor rule for deduplication metadata

Blog Article

el reformador tequila anejo Building effective deduplication index in the memory could reduce disk access times and enhance chunk fingerprint lookup speed,which was a big challenge for deduplication algorithms in massive data environments.As deduplication data set had many samples with high similarity,a deduplication algorithm based on condensed nearest neighbor rule,which was called Dedup2,was proposed.Dedup2uses clustering algorithm to divide the original deduplication metadata into several categories.According to these categories,it employs condensed nearest neighbor rule to remove sophie allport bee curtains the highest similar data in the deduplication metadata.After that it can get the subset of deduplication metadata.

Based on this subset,new data objects will be deduplicated based on the principle of data similarity.The results of experiments show that Dedup2can reduce the size of deduplication data set more than 50% effectively while maintain similar deduplication ratio.