Optimizing Rolling Hash Computation Using Simd Vector Registers

This technique can be conditionally used in certain environments depending on the nature of the rolling hash and the CPU capabilities available. A large vector register is used to hold the sliding window bytes and specialized vector instructions are used to shift the register bytes and add and remove bytes in the register. This simulates the behavior of a sliding window entirely in a CPU register.

Publication Date
16 April 2013

single instruction multiple data vectorized hash function X86 Processor Vector Instructions streaming SIMD Extensions advanced vector expressions sliding window rolling hash content-aware chunking data de-duplication


Click here to download link on ip.com

Click to share this page via your favorite social network.

Learn more about defensive publications with our examples and frequently asked questions

What we are trying to do?

We are attempting to mobilize the creativity and innovative capacities of the Linux and broader open source community to codify the universe of preexisting inventions in defensive publications that upon publication in the IP.COM database will immediately serve as effective prior art that prevents anyone from having a patent issued that claims inventions that have already been document in a defensive publication. In addition to creating a vehicle to utilize this highly effective form of IP rights management for known inventions, it is hoped that the community will use defensive publications as a means of codifying future inventions should the inventors prefer not to make their invention the subject of a patent disclosure and application.