You should benchmark all implementations (using cpu time, not realtime) and choose the fastest and while benchmarking check whether the algorithm actually works.
Yeah, while implementing the cuda hasher I thought about this. There should be an interface to the hashing handler (or even a full miner per implementation) and we should have a simple way of giving it a known block, ask it to hash 1000 nonces and compare the result, while benchmarking at the same time. Shouldn’t be too hard to implement and would help when developing new algorithms.
The interface schema would also help if we were to plug in an fpga based engine or something of the kind, having specific entry points into the code without having to tweak on the default mining schema.