I didn’t realize that there was a exact match following the hashed match. I agree, my false positive concern is misplaced.

I am looking forward to reading your papers.

Josh

It also detects “near miss” clones which you can think of as parameterized code, e.g., if you made a macro out of block of code and replaced well-formed sections, you’d end up with a parameterized clone.

The present (2010) implementation operates somewhat differently than the 1998 paper, but the basic ideas are the same. A 2004 study (published in IEEE Transactions on Software Engineering) by Steve Bellon compared several detectors, and concluded that ours produce the smallest number of false positives, so I think your worry is misplaced.

You’ve observed that matching exact trees is “easy”. In fact, I agree. What isn’t easy is matching inexact trees to produce the near miss clones, and making this work at the 2 million line scale for multiple programming languages.

You can find a number of examples of clone detection runs on different languages at the website.

– IDB

I read your paper, though I have not tried out your implementation. If I remember correctly, the high level view of the algorithm was to compute a hash on the ASTs and sub-ASTs, and then compare hashes for clone detection. I think that one advantage to my approach is that when a clone is reported there is a higher degree of certainty that it is valid. (I think like 100% certainty, though I haven’t proven that or anything.) The ‘fuzzy hash’ idea where the hash calculation handles some constructs differently in an effort to detect near miss clones (like ignoring small sub trees for example) seems like it would generate false positives. For my tool false positives aren’t acceptable as I want to automate the clone repair. Of course, a hash based implementation would be a lot faster. I am dealing with the kind of poor algorithmic complexity you mention in your paper, like O(n^3) and worse sometimes.

I would like to read more about how you automated the repair of the clones. Do you talk about that in one of the other papers? I’ve only read the one.

Thanks for the comment,
Josh