There is lots of prior art for what I am doing, so I can ignore the bogus patents. Right?

Not really. If a court, or a patent office re-evaluation, upholds that what you think is relevant prior art really is, that patent may be dismissed. More often, some of its claims are dismissed, and the rest remain in force. Until that time you stand the possibility of defending what you believe in court. Is that something you want to do, considering the cost and the history of unreliable outcomes in these disputes?