@Fedifensor I feel like that is what we are meant to think and that is definitely how I watched it in the moment. But in retrospect I don't think there was anything actually provided to support it.
If you don't bring in outside assumptions, the show only presents them as so far the only notable obstacle Kingpin encountered. Aloof and vain for sure, but that's it.