pytorch RuntimeError: Expected to have finished reduction in the prior iteration before starting a n

    Traceback (most recent call last):
      File "train.py", line 166, in <module>
    main(args_)
      File "train.py", line 118, in main
    f_clean_masked, f_occ_masked, fc, fc_occ = backbone(img1, img2)
      File "/home/user1/miniconda3/envs/py377/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
      File "/home/user1/miniconda3/envs/py377/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 692, in forward
    Traceback (most recent call last):
      File "train.py", line 166, in <module>