Prepare brownies according to package directions, portioning the batter into miniature muffin cups before baking. Cool baked brownies completely.
When the brownies have cooled, in a mixing bowl with a hand mixer or a stand mixer bowl fitted with a paddle attachment, beat together the peanut butter, cream cheese, confectioner’s sugar and butter until fluffy, about 2 minutes.
Add the vanilla and milk, and continue beating until smooth, about 1 minute. (If the mixture is still thick, add more milk, 1 tablespoon at a time, until smooth and fluffy).
Transfer the frosting to a zipper lock bag and push the mixture down into one corner. Cut the corner of the bag off and pipe the frosting on top of each miniature brownie. Serve.