113 Commits (39e75862b248a20e8233ccee743ba5b2e977cdcf)

Author SHA1 Message Date
comfyanonymous 0cf4e86939 Add some command line arguments to store text encoder weights in fp8. 1 year ago
comfyanonymous 7339479b10 Disable xformers when it can't load properly. 1 year ago
comfyanonymous dd4ba68b6e Allow different models to estimate memory usage differently. 1 year ago
comfyanonymous 8594c8be4d Empty the cache when torch cache is more than 25% free mem. 1 year ago
comfyanonymous c8013f73e5 Add some Quadro cards to the list of cards with broken fp16. 1 year ago
comfyanonymous fd4c5f07e7 Add a --bf16-unet to test running the unet in bf16. 1 year ago
comfyanonymous 9a55dadb4c Refactor code so model can be a dtype other than fp32 or fp16. 1 year ago
comfyanonymous 88733c997f pytorch_attention_enabled can now return True when xformers is enabled. 1 year ago
comfyanonymous 20d3852aa1 Pull some small changes from the other repo. 1 year ago
Simon Lui eec449ca8e Allow Intel GPUs to LoRA cast on GPU since it supports BF16 natively. 1 year ago
comfyanonymous 1cdfb3dba4 Only do the cast on the device if the device supports it. 1 year ago
comfyanonymous 321c5fa295 Enable pytorch attention by default on xpu. 1 year ago
comfyanonymous 0966d3ce82 Don't run text encoders on xpu because there are issues. 1 year ago
comfyanonymous 1938f5c5fe Add a force argument to soft_empty_cache to force a cache empty. 1 year ago
Simon Lui 4a0c4ce4ef Some fixes to generalize CUDA specific functionality to Intel or other GPUs. 1 year ago
comfyanonymous b8c7c770d3 Enable bf16-vae by default on ampere and up. 1 year ago
comfyanonymous a57b0c797b Fix lowvram model merging. 1 year ago
comfyanonymous f72780a7e3 The new smart memory management makes this unnecessary. 1 year ago
comfyanonymous 30eb92c3cb Code cleanups. 1 year ago
comfyanonymous 51dde87e97 Try to free enough vram for control lora inference. 1 year ago
comfyanonymous cc44ade79e Always shift text encoder to GPU when the device supports fp16. 1 year ago
comfyanonymous a6ef08a46a Even with forced fp16 the cpu device should never use it. 1 year ago
comfyanonymous f081017c1a Save memory by storing text encoder weights in fp16 in most situations. 1 year ago
comfyanonymous 0d7b0a4dc7 Small cleanups. 1 year ago
Simon Lui 9225465975 Further tuning and fix mem_free_total. 1 year ago
Simon Lui 2c096e4260 Add ipex optimize and other enhancements for Intel GPUs based on recent memory changes. 1 year ago
comfyanonymous e9469e732d --disable-smart-memory now disables loading model directly to vram. 1 year ago
comfyanonymous 3aee33b54e Add --disable-smart-memory for those that want the old behaviour. 1 year ago
comfyanonymous 2be2742711 Fix issue with regular torch version. 1 year ago
comfyanonymous 89a0767abf Smarter memory management. 1 year ago
comfyanonymous 1ce0d8ad68 Add CMP 30HX card to the nvidia_16_series list. 1 year ago
comfyanonymous 4a77fcd6ab Only shift text encoder to vram when CPU cores are under 8. 1 year ago
comfyanonymous 3cd31d0e24 Lower CPU thread check for running the text encoder on the CPU vs GPU. 1 year ago
comfyanonymous 22f29d66ca Try to fix memory issue with lora. 1 year ago
comfyanonymous 18885f803a Add MX450 and MX550 to list of cards with broken fp16. 1 year ago
comfyanonymous ff6b047a74 Fix device print on old torch version. 1 year ago
comfyanonymous 1679abd86d Add a command line argument to enable backend:cudaMallocAsync 1 year ago
comfyanonymous 5f57362613 Lower lora ram usage when in normal vram mode. 1 year ago
comfyanonymous 490771b7f4 Speed up lora loading a bit. 1 year ago
KarryCharon 3e2309f149 fix mps miss import 1 year ago
comfyanonymous 0ae81c03bb Empty cache after model unloading for normal vram and lower. 1 year ago
comfyanonymous e7bee85df8 Add arguments to run the VAE in fp16 or bf16 for testing. 1 year ago
comfyanonymous ddc6f12ad5 Disable autocast in unet for increased speed. 1 year ago
comfyanonymous 8d694cc450 Fix issue with OSX. 1 year ago
comfyanonymous dc9d1f31c8 Improvements for OSX. 1 year ago
comfyanonymous 2c4e0b49b7 Switch to fp16 on some cards when the model is too big. 1 year ago
comfyanonymous 6f3d9f52db Add a --force-fp16 argument to force fp16 for testing. 1 year ago
comfyanonymous 1c1b0e7299 --gpu-only now keeps the VAE on the device. 1 year ago
comfyanonymous 3b6fe51c1d Leave text_encoder on the CPU when it can handle it. 1 year ago
comfyanonymous b6a60fa696 Try to keep text encoders loaded and patched to increase speed. 1 year ago