[BUG] Zero3: Post backward hook is not triggered for submodules whose inputs have .required_grad=False about deepspeed HOT 1 OPEN

A brutal force solution is to enforce the .requires_grad to be True for the model input tensors:

        class PostBackwardFunctionModule(torch.autograd.Function):

            @staticmethod
            def forward(ctx, output):
                ctx.module = module

                if not output.requires_grad:
                    output.requires_grad_(requires_grad=True)
                    output.mark_as_no_grad = True

The .requires_grad value can be then restored to its original in the PostBackwardFunctionModule::backward.
This method works, but seems to be hacky and may introduce some unexpected changes in the torch autograd mechanism.

from deepspeed.