Patriots or Broncos D/ST?

So I have the Patriots right now, Broncos are available. Dont really want to play Patriots against Vikings or at Pitt. Broncos have Cincy and Cleveland in those 2 weeks. Should I just go Broncos to have 1 roster spot for defense?