Dark matter that was once in thermal equilibrium with the Standard Model is generally prohibited from obtaining all of its mass from the electroweak phase transition. This implies a new scale of physics and mediator particles to facilitate dark matter annihilation. In this work, we focus on dark matter that annihilates through a generic Higgs portal. We show how partial wave unitarity places an upper bound on the mass of the mediator (or dark) Higgs when its mass is increased to be the largest scale in the effective theory. For models where the dark matter annihilates via fermion exchange, an upper bound is generated when unitarity breaks down around 8.5 TeV. Models where the dark matter annihilates via fermion and higgs boson exchange push the bound to 45.5 TeV. We also show that if dark matter obtains all of its mass from a new symmetry breaking scale that scale is also constrained. We improve these constraints by requiring perturbativity in the Higgs sector up to each unitarity bound. In this limit, the bounds on the dark symmetry breaking vev and the dark Higgs mass are now 2.4 and 3 TeV, respectively, when the dark matter annihilates via fermion exchange. When dark matter annihilates via fermion and higgs boson exchange, the bounds are now 12 and 14.2 TeV, respectively. The available parameter space for Higgs portal dark matter annihilation is outlined. We also show how the bounds are improved if Higgs portal dark matter is only a fraction of the observed relic abundance. Finally, we discuss how to apply these arguments to other dark matter scenarios and discuss prospects for direct detection and future collider searches. If the Higgs portal is responsible for dark matter annihilation, planned direct detection experiments will cover almost all the parameter space. The ILC and/or VLHC, however, is needed to establish the Higgs portal mechanism. Show Partial Abstract