Cosmology and modified gravitational wave propagation from binary black hole population models

A joint hierarchical Bayesian analysis of the binary black hole (BBH) mass function, merger rate evolution, and cosmological parameters can be used to extract information on both the cosmological and population parameters. We extend this technique to include the effect of modified gravitational wave (GW) propagation. We discuss the constraints on the parameter Ξ0 that describe this phenomenon [with Ξ0=1 in general relativity (GR)] using the data from the third gravitational wave transient catalog. We find the constraints Ξ0=1.2-0.7+0.7 with a flat prior on Ξ0, and Ξ0=1.0-0.8+0.4 with a prior uniform in logΞ0 (68% C.L., maximum posterior and highest-density interval), which only rely on the presence of a feature in the BBH mass distribution around ∼30-45 M⊙ and are robust to whether or not the event GW190521 is considered an outlier of the population. We then study in more detail the effects of modified GW propagation on population and cosmological analyses for LIGO/Virgo at design sensitivity. We find that, for a given data-taking period, the relative error ΔΞ0/Ξ0 has a significant dependence on the fiducial value of Ξ0, since the latter has a strong influence on the detection rate. For five years of data, the accuracy ranges from ∼10% on Ξ0 when Ξ0=1 to ΔΞ0/Ξ0∼20% for Ξ0=1.8, which represents a large deviation from GR, still consistent with current limits and predicted by viable cosmological models. For the Hubble parameter, we forecast an accuracy of ΔH0/H0∼20% and an accuracy on H(z) of ∼7% at a pivot redshift z∗∼0.8; this updates the results found in previous studies, by making use of a population model compatible with current observations, and is about a factor 2 worse, in twice the observing time. We finally show that, if nature is described by a modified gravity theory with a large deviation from the GR value Ξ0=1, such as Ξ0=1.8, analyzing the data assuming GR produces a significant bias in the inferred values of the mass scales, Hubble constant, and particularly the BBH merger rate.