KenSchae
/
wyrde-comfyui-workflows
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Browse Source

lora workflow

wyrde-docs
wyrde 2 years ago
parent
50ba536532
commit
23d3f99c5e
3 changed files with 737 additions and 1 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 38
      basics/building-up/README.md
  2. 700
      basics/building-up/basic-workflow-vae-lora.json
  3. BIN
      basics/building-up/basic-workflow-vae-lora.png

38
basics/building-up/README.md
Unescape View File

@ -57,14 +57,50 @@ For a VAE Loader
* Go ahead and place a vae
Download a VAE from [stabilityai](https://huggingface.co/stabilityai/sd-vae-ft-mse-original/tree/main) and drop it in `ComfyUI\models\vae`. Get the pruned.safetensors file.
* Don't click the filename (it results in a page that is confusing until eyes finally spot the "download" link), click to the right. <img src="downloadvae.png" width="20%" align="middle">.
* Don't click the filename (it results in a page that is confusing until eyes finally spot the "download" link), click to the right. <img src="downloadvae.png" width="75%" align="middle">.
* Drop the file in `ComfyUI\models\vae\`
* Once it is downloaded, hit F5 and refresh the window so Comfy knows the file is there.
* With one vae, it is easy to pick in the VAE Loader by clicking on arrows or the list.
In this particular case, there isn't a huge difference in making a new version with the vae linked. For an idea of what differences vae make, check out [this comparison](https://github.com/wyrde/wyrde-comfyui-workflows/tree/main/compare/model-vae). The ones with bad vae are faded and washed out. Good vae are vibrant.
Note
* Automatic1111's method of matching vae file names to models and placing them in the model folder doesn't matter to ComfyUI. Either the VAE is loaded directly or the one built into the model is used.
* [Here's some good info on VAE](https://rentry.org/sdvae)
## Adding a Lora
* <a href="basic-workflow-vae-lora.json">workflow + vae + lora.json file</a>
<img src="basic-workflow-vae-lora..png" align="middle">
Lora (and the varients) are cool mini-models that are used to alter a bigger model. Think of them like the trojan horse, but everyone is happy with the result. Usually.
[here's a lora](https://civitai.com/models/44960?modelVersionId=49584) to test with. Drop it in the `ComfyUI\models\lora\` folder.
* as with other files, hit F5 to refresh the file lists
* ignore the text in the lora page for now, what is first seen is for version 3.0 and version 1 is what we're using.
Adding a lora is a bit more tricky than a vae loader, because they go between the model and the Ksampler.
* Unlike automatic1111, the lora aren't put in the prompt.
Make a little room for the _lora Loader_ node.
* Between the _CLIP Text_ nods and _Load Checkpoint_ hold down the control key and drag with the left mouse button. It will draw a box. When Left Mouse is released, everything inside the box is selected.
* Now hold down shift and Left Click one of the selected nodes. All the nodes will be dragged as a group when the mouse is dragged.
* It is better to make room by dragging nodes right than left
* Refreshing (F5) the window zooms to the barely visiable blue box where the beginning nodes are.
* it gets annoying to pan left all the time
Now drop a _Lora Loader_ in the empty spot.
* Connect the _Load Checkpoint_ model and clip outputs to the Load Lora.
* The _Load LoRA_'s model output goes to the model reroute (to the KSampler model input)
* this will autmatically detach it from the Load Checkpoint.
* The _Load LoRA_'s CLIP output is noodled to both the CLIP nodes' inputs.
* this will autmatically detach them from the Load Checkpoint.
* Select _mPixel_v10pixelArt.safetensors_ for the lora_name field.
* change strength_model and strength_clip to 0.8
## HiRez Fixing
## Expanding on Fixing

700
basics/building-up/basic-workflow-vae-lora.json
Unescape View File

@ -0,0 +1,700 @@
{
"last_node_id": 34,
"last_link_id": 69,
"nodes": [
{
"id": 5,
"type": "EmptyLatentImage",
"pos": [
-70,
330
],
"size": {
"0": 315,
"1": 106
},
"flags": {},
"order": 0,
"mode": 0,
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"links": [
50
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "EmptyLatentImage"
},
"widgets_values": [
512,
512,
1
]
},
{
"id": 32,
"type": "Reroute",
"pos": [
300,
460
],
"size": [
90.4,
26
],
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"name": "",
"type": "*",
"link": 50
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"links": [
51
],
"slot_index": 0
}
],
"properties": {
"showOutputText": true,
"horizontal": false
},
"color": "#233",
"bgcolor": "#355"
},
{
"id": 9,
"type": "SaveImage",
"pos": [
2310,
140
],
"size": {
"0": 400,
"1": 480
},
"flags": {},
"order": 13,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 9
}
],
"properties": {},
"widgets_values": [
"ComfyUI"
]
},
{
"id": 8,
"type": "VAEDecode",
"pos": [
2040,
140
],
"size": {
"0": 210,
"1": 46
},
"flags": {},
"order": 12,
"mode": 0,
"inputs": [
{
"name": "samples",
"type": "LATENT",
"link": 67
},
{
"name": "vae",
"type": "VAE",
"link": 12
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
9
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "VAEDecode"
}
},
{
"id": 7,
"type": "CLIPTextEncode",
"pos": [
960,
270
],
"size": {
"0": 425.27801513671875,
"1": 180.6060791015625
},
"flags": {},
"order": 9,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 17
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"links": [
68
],
"slot_index": 0
}
],
"title": "CLIP Text Encode (Prompt) Negative",
"properties": {
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"text, watermark"
],
"color": "#222",
"bgcolor": "#000"
},
{
"id": 6,
"type": "CLIPTextEncode",
"pos": [
960,
60
],
"size": {
"0": 422.84503173828125,
"1": 164.31304931640625
},
"flags": {},
"order": 8,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 16
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"links": [
69
],
"slot_index": 0
}
],
"title": "CLIP Text Encode (Prompt) Positive",
"properties": {
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"beautiful scenery nature glass bottle landscape, purple galaxy bottle,"
],
"color": "#432",
"bgcolor": "#653"
},
{
"id": 13,
"type": "Reroute",
"pos": [
300,
510
],
"size": [
75,
26
],
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"name": "",
"type": "*",
"link": 18
}
],
"outputs": [
{
"name": "VAE",
"type": "VAE",
"links": [
19
],
"slot_index": 0
}
],
"properties": {
"showOutputText": true,
"horizontal": false
},
"color": "#323",
"bgcolor": "#535"
},
{
"id": 11,
"type": "Reroute",
"pos": [
1880,
510
],
"size": [
75,
26
],
"flags": {},
"order": 10,
"mode": 0,
"inputs": [
{
"name": "",
"type": "*",
"link": 19
}
],
"outputs": [
{
"name": "VAE",
"type": "VAE",
"links": [
12
],
"slot_index": 0
}
],
"properties": {
"showOutputText": true,
"horizontal": false
},
"color": "#323",
"bgcolor": "#535"
},
{
"id": 33,
"type": "Reroute",
"pos": [
1410,
460
],
"size": [
90.4,
26
],
"flags": {},
"order": 6,
"mode": 0,
"inputs": [
{
"name": "",
"type": "*",
"link": 51
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"links": [
52
],
"slot_index": 0
}
],
"properties": {
"showOutputText": true,
"horizontal": false
},
"color": "#233",
"bgcolor": "#355"
},
{
"id": 4,
"type": "CheckpointLoaderSimple",
"pos": [
-70,
190
],
"size": {
"0": 315,
"1": 98
},
"flags": {},
"order": 1,
"mode": 0,
"outputs": [
{
"name": "MODEL",
"type": "MODEL",
"links": [
13
],
"slot_index": 0
},
{
"name": "CLIP",
"type": "CLIP",
"links": [
15
],
"slot_index": 1
},
{
"name": "VAE",
"type": "VAE",
"links": [],
"slot_index": 2
}
],
"properties": {
"Node name for S&R": "CheckpointLoaderSimple"
},
"widgets_values": [
"v1-5-pruned-emaonly.safetensors"
]
},
{
"id": 10,
"type": "VAELoader",
"pos": [
-70,
480
],
"size": {
"0": 315,
"1": 58
},
"flags": {},
"order": 2,
"mode": 0,
"outputs": [
{
"name": "VAE",
"type": "VAE",
"links": [
18
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "VAELoader"
},
"widgets_values": [
"vae-ft-mse-840000-ema-pruned.safetensors"
]
},
{
"id": 3,
"type": "KSampler",
"pos": [
1610,
140
],
"size": {
"0": 315,
"1": 262
},
"flags": {},
"order": 11,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 54
},
{
"name": "positive",
"type": "CONDITIONING",
"link": 69
},
{
"name": "negative",
"type": "CONDITIONING",
"link": 68
},
{
"name": "latent_image",
"type": "LATENT",
"link": 52
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"links": [
67
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "KSampler"
},
"widgets_values": [
1117698844150624,
"randomize",
20,
8,
"uni_pc_bh2",
"normal",
1
]
},
{
"id": 12,
"type": "LoraLoader",
"pos": [
340,
-20
],
"size": {
"0": 500,
"1": 140
},
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 13
},
{
"name": "clip",
"type": "CLIP",
"link": 15
}
],
"outputs": [
{
"name": "MODEL",
"type": "MODEL",
"links": [
53
],
"slot_index": 0
},
{
"name": "CLIP",
"type": "CLIP",
"links": [
16,
17
],
"slot_index": 1
}
],
"properties": {
"Node name for S&R": "LoraLoader"
},
"widgets_values": [
"mPixel_v10PixelArt.safetensors",
0.8,
0.8
]
},
{
"id": 34,
"type": "Reroute",
"pos": [
1410,
-20
],
"size": [
82,
26
],
"flags": {},
"order": 7,
"mode": 0,
"inputs": [
{
"name": "",
"type": "*",
"link": 53
}
],
"outputs": [
{
"name": "MODEL",
"type": "MODEL",
"links": [
54
],
"slot_index": 0
}
],
"properties": {
"showOutputText": true,
"horizontal": false
},
"color": "#2a363b",
"bgcolor": "#3f5159"
}
],
"links": [
[
9,
8,
0,
9,
0,
"IMAGE"
],
[
12,
11,
0,
8,
1,
"VAE"
],
[
13,
4,
0,
12,
0,
"MODEL"
],
[
15,
4,
1,
12,
1,
"CLIP"
],
[
16,
12,
1,
6,
0,
"CLIP"
],
[
17,
12,
1,
7,
0,
"CLIP"
],
[
18,
10,
0,
13,
0,
"*"
],
[
19,
13,
0,
11,
0,
"*"
],
[
50,
5,
0,
32,
0,
"*"
],
[
51,
32,
0,
33,
0,
"*"
],
[
52,
33,
0,
3,
3,
"LATENT"
],
[
53,
12,
0,
34,
0,
"*"
],
[
54,
34,
0,
3,
0,
"MODEL"
],
[
67,
3,
0,
8,
0,
"LATENT"
],
[
68,
7,
0,
3,
2,
"CONDITIONING"
],
[
69,
6,
0,
3,
1,
"CONDITIONING"
]
],
"groups": [],
"config": {},
"extra": {},
"version": 0.4
}

BIN
basics/building-up/basic-workflow-vae-lora.png
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 101 KiB

Write Preview
Loading…
Cancel
Save