During Biblical times, consummated marriages took place at far younger ages - it was common for a young woman to be married off by age 14 (at least, after the Babylonian Exile). Mary, the mother of Jesus, was purported to be age 15 when married to Joseph (I don't know whether that is reliable, however, so don't quote me). Nota bene: I'm not defending pedophilia in the least. I am, rather, arguing that using the Biblical Era (indeed, the Bible) as a precedent is going to be very problematic especially in discussion of this issue.
This is just one of the many issues, which show why RIGID ADHERENCE or strictly-literal interpretations of the Bible, just ends up being inadequate.
Let's face it: Kids having sex with "adults" (as we tend to define kids in America), can be (and is) problematic.
Various cultures can sustain certain practices (and not fall to pieces), while others cannot. What's too young/old for MANY given things, varies from culture to culture, age to age.
Here in the U.S.A., there are cultural reasons to allow and/or forbid certain behaviors. These things are not so "absolute" as they my seem, though any one of us may know them to be for our entire lifetime. Laws are reviewed and they evolve over time; even those interpreted or supported by the Bible.
Despite all of that, as far as "pedophilia" and "polygamy" are concerned, the overall concern of most who view it (negatively), are those who become
victims as a result of those practices.
It is not unusual to hear about some consummate JERK (like the that nutcase, "Jeffs", that they just caught), who had more victims in his wake than most would want to count. Polygamy is something that doesn't have to be evil, but for some reason many men have done nothing but victimize people and abuse families through it. The record is clear enough (in this nation and others).
Pedophilia is not sanctioned, because the many effects of the crime are deemed unacceptable (at many levels in this society); most importantly, pedophilia is viewed by most (normal) people as a victimization of a child. Typically, lack of
legal consent by one or more parties, helps to define laws prohibiting certain sexual actions.
Still, there are "cultural" qualifiers which modify or mitigate the possible views which people may possess about either pedophilia or polygamy. I don't think the Bible specifies anything near what America's laws tend to overall.
-Mel-