I don't see why we should refrain from calling a work "infamous" or any other pejorative (as long as we can back it up). It's not as though Tim called the authors themselves "infamous". It's not "ad hominem" if you're criticizing a book.
Or, as I just found on a random website: