Does following Techniques reduce model quality by huge margin ?

[Greatz08]

i was curious to know about the actual performance of model after using multiple techniques to speed run.

[BradKML]

Often times there are two answers to this:

1. Just train it for longer, and the quality issue will generally fade away (generic pedestrian answer)
2. Quality degradation exists (expert-oriented response, maybe @KellerJordan know more about this)

I would like to know which possibility is more likely to be true tho