In a large mixing bowl, beat the softened cream cheese on medium-high speed until smooth. This usually takes no more than a minute. Then add in the egg yolks individually, ensuring each one is well incorporated. Next, mix in the sweetened condensed milk, key lime juice, lime zest, and vanilla extract until smooth. Make sure to scrape down any bits from the sides of the bowl to eliminate lumps of cream cheese.