Nilsimsa Hash
Nilsimsa is an anti-spam focused locality-sensitive hashing algorithm originally proposed the cmeclax remailer operator in 2001[1] and then reviewed by Ernesto Damiani et al. in their 2004 paper titled, "An Open Digest-based Technique for Spam Detection".[2] The goal of Nilsimsa is to generate a hash digest of an email message such that the digests of two similar messages are similar to each other. In comparison with cryptographic hash functions such as SHA-1 or MD5, making a small modification to a document does not substantially change the resulting hash of the document. The paper suggests that the Nilsimsa satisfies three requirements:
- The digest identifying each message should not vary significantly (sic) for changes that can be produced automatically.
- The encoding must be robust against intentional attacks.
- The encoding should support an extremely low risk of false positives.
Subsequent testing on a range of file types identified the Nilsimsa hash as having a significantly higher false positive rate when compared to other similarity digest schemes such as TLSH, Ssdeep and Sdhash.[3]
Nilsimsa similarity matching was taken in consideration by Jesse Kornblum when developing the fuzzy hashing in 2006,[4] that used the algorithms of spamsum by Andrew Tridgell (2002).[5]
Several implementations of Nilsimsa exist as open-source software.[6][7][8][9][10]
References[edit]
- ^ cmeclax remailer operator (10 February 2002). "Nilsimsa v.0.2.4". Archived from the original on 7 July 2005. Retrieved 23 February 2014.
- ^ Damiani; et al. (2004). "An Open Digest-based Technique for Spam Detection" (PDF). Retrieved 2013-09-01.
- ^ Oliver; et al. (2013). "TLSH - A Locality Sensitive Hash". 4th Cybercrime and Trustworthy Computing Workshop. Retrieved 2015-06-04.
- ^ Jesse Kornblum (15 May 2008). "The Fuzzy Hashing Patent". LiveJournal. Archived from the original on 7 May 2016. Retrieved 23 February 2014.
- ^ Jesse Kornblum (2006). "Identifying almost identical files using context triggered piecewise hashing" (PDF). DFRWS. Retrieved 23 February 2014.
- ^ "py-nilsimsa - Python port of Nilsimsa locality-sensitive hash". github.com. Retrieved 2016-11-08.
- ^ "Nilsimsa". Nilsimsa.rubyforge.org. Archived from the original on 2013-06-15. Retrieved 2013-09-01.
- ^ "Digest::Nilsimsa". metacpan.org. Retrieved 2013-09-01.
- ^ "golang nilsimsa - implements nilsimsa fuzzy hash by cmeclax". hersensch.im. Retrieved 2018-04-25.
- ^ "node-nilsimsa - Node.JS port of Nilsimsa locality-sensitive hash". github.com. Retrieved 2023-09-09.