Yup. No outide food allowed in a Kosher spot. Even bottled beverages are verbotten. Same for Halal joints.
So, did you try Second Avenue Deli? It is my fav by far. I think katz's is a one trick pony. It's just pastrami, the rest of the food there is just ok. 2nd ave deli is much more consistent.
I already sent you to Yonah Shimmel's for knishes and Zabar's for deli take home.
BTW: if you're in the TriBeCa area, check out any of the all night Halal joints along Broadway. Great vegetarian food and the lovely ambiance of taxi drivers and camel jockeys. They are like the NYC version of LA's India Sweets and Spices. Buffet line steam table joints with spicy vegetarian dishes and flat breads.