My understanding is that the Constitution was intended to limit the power of the government, not to provide an explicit list of "rights" that the people are entitled to. The 9th amendment to the Constitution states: "The enumeration in the Constitution, of certain rights, shall not be construed to deny or disparage others retained by the people."
I think Kavanaugh is wrong, indeed shockingly ignorant, to argue that the Constitution is "neutral" on the issue of abortion (it does not mention abortion anywhere), and therefore the right does not exist. If the Constitution does not explicitly mention abortion, that means the government has no business banning it. To argue otherwise is to argue that the 9th amendment is meaningless. This was actually the position taken by Robert Bork, and it was one of several reasons why he was so strongly opposed when he was nominated to the Supreme Court.