Welcome! Log In Create A New Profile

Advanced

Re: hash algorithm for nginx cache

Joe Bofh
April 21, 2009 07:22PM
FWIW, I got a follow up reply from Austin (the chap that created
murmurhash). He sez:

"MurmurHash2, MD5, Lookup3, and any cryptographic hash all have
statistically-correct collision behaviour - the average number of
collisions they create on a given keyset is the same as if the hash
function returned a totally random number. For comparison, FNV mixes
very poorly and tends to produce massively more collisions than expected
on "sparse" keys with only a few bits set, and SuperFastHash will tend
to produce a higher than expected number of collisions on keysets where
keys differ in only 2-3 bits - see
http://murmurhash.googlepages.com/statistics and the associated pages
for more info."

So it looks like collision-wise, it's comparable.

--J


Johan Bergström wrote:
> Hey,
>
> On Apr 21, 2009, at 17:46 , Joe Bofh wrote:
>
>>
>> I guess if you put in an option to specify the hash, that would work
>> too. FNV and Murmur have been adopted on various open source
>> projects as
>> optional hashs because of the hashing performance.
>
> For reference; libmemcached - another high performing project -
> nowadays include most current hashing algorithms. Fnv1a and murmur are
> two of them. See:
> http://hg.tangent.org/libmemcached/file/048b32d9d1ae/docs/memcached_behavior.pod
> (sorry for bad linkage, the site docs/manpage hasn't been updated
> for a while)
>
> Two my knowledge, murmur/murmur2 is the best pick for a high
> performance/low (but not zero) collision tradeoff.

--
Posted via http://www.ruby-forum.com/.
Subject Author Posted

hash algorithm for nginx cache

Joe Bofh April 18, 2009 02:27PM

Re: hash algorithm for nginx cache

Marcus Clyne April 18, 2009 04:31PM

Re: hash algorithm for nginx cache

Joe Bofh April 18, 2009 11:42PM

Re: hash algorithm for nginx cache

Marcus Clyne April 19, 2009 07:13AM

Re: hash algorithm for nginx cache

Igor Sysoev April 21, 2009 03:17AM

Re: hash algorithm for nginx cache

Joe Bofh April 21, 2009 11:46AM

Re: hash algorithm for nginx cache

Johan Bergstr April 21, 2009 12:23PM

Re: hash algorithm for nginx cache

Joe Bofh April 21, 2009 07:22PM

Re: hash algorithm for nginx cache

Igor Sysoev April 22, 2009 07:07AM

Re: hash algorithm for nginx cache

Joe Bofh May 15, 2009 01:29AM

Re: hash algorithm for nginx cache

Igor Sysoev May 16, 2009 11:38AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 154
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready