Running on CPU only?

[viper968]

Would it be possible to move the vast majority of the training to the CPU and have smaller parts run on the GPU, i have a 3050ti with only 4GB of VRAM and i use the same technique when trying to train a LoRa for stable diffusion since the UNet used 3.7GB of VRAM on its own before even adding the training data and other information. If this is possible i would love to know before i go down the rabbit hole of looking through all the scripts and attempting to change everything "cuda" to "cpu" for pytorch.