Would a small-mass hammer that accelerated rapidly exert more horizontal force on a nail than a large-mass hammer that didn’t accelerate very much?
Yes. Since the only horizontal force acting on the hammer is that exerted on it by the nail, the hammer’s acceleration is entirely determined by that force. The force on the hammer is equal to the hammer’s mass times the hammer’s acceleration (Newton’s second law). If both hammers experienced the same acceleration, then the large-mass hammer would have to be experiencing the larger force from the nail and would therefore be exerting the larger force on the nail. But because the small-mass hammer is experience a larger acceleration, the force that the nail is exerting on it may be quite large. If the small-mass hammer’s acceleration is large enough, the force on it may exceed the force on the large-mass hammer.