Use a chilled bowl to mix heavy cream, vanilla extract, and sugar together until whipped cream forms. Set aside in the refrigerator.
Combine cream cheese and sweetened condensed milk with a hand mixer.
In a large bowl, mix together milk and pudding. Stir in cream cheese mixture and continue mixing until smooth.
Fold in 2 cups of fresh whipped cream; stir until thoroughly combined.
Layer vanilla wafers and freshly sliced bananas in a trifle dish or 9x13 pan. Pour half of the pudding mixture on top. Repeat with another layer or wafers, bananas, and pudding.
Top with remaining whipped cream. Garnish with wafer crumbs and banana slices.