By rejecting the use of a prior distribution over parameters, orthodox statistics is forced to focus on estimators, functions which guess parameter values, and to invent heuristics for choosing among estimators. Two popular heuristics are unbiasedness and maximum likelihood. Since these heuristics are not consistent with Bayes' rule, they are also not consistent with the axioms of common sense from which Bayes' rule is derived. Hence we expect there to be situations in which they violate common sense and indeed it is not hard to find such situations. This paper reviews a few simple, realistic scenarios where pathologies occur with either the unbiasedness heuristic or the maximum likelihood heuristic.