The Outer Banks are an awesome place for a family vacation. It's remote enough that your kids won't be screaming to go play putt-putt and ride the go-carts. There isn't much to do there but hang out on the beach and eat good seafood. The fishing is good if y'all are into surf-fishing. I think your kids are too young to go offshore on a charter.

Where exactly are you going and I'll get some more specific information together.

If you want a place with more activities without all the crowds I suggest Garden City Beach in South Carolina. It's just south of Myrtle Beach.