The Judge seems absolutely reasonable to me. Why should people get indignant when it is advised that you should be proficient in the language spoken by those who surround you? Isn't this like being outraged when told not to stick your hand into a fire?
Being non-fluent in English certainly is no help to the mother, who will find her job prospects highly limited, and - depending on her level of competency - may find herself constrained to living and functioning in one of America's urban foreign enclaves, which tend to be poor and dysfunctional. The children in non-English speaking homes tend to be at tremendous disadvantage in school, since they do not readily understand the language of instruction or are substantially slower at doing so. This is a crazy situation all around!
Also: Isn't it supposed to be a condition of immigration that one is to be proficient in English?