Jun 28, 2023·edited Jun 28, 2023Liked by Charlie Guo
I've actually been wondering just how difficult it was to "plug into" the underlying LLM model from a coding perspective.
In my simple brain it was about "just" making some boxes for input that send it to the model and "just" some UI to make the output formatted to whatever the interface/app is supposed to do.
So I kind of assumed switching which model you communicate with (Claude, GPT, or other) was a relatively simple tweak. Sounds like it's a lot more complex and the degree of lock-in isn't trivial.
By the way, I'm quite sure Midjourney isn't built on top of Stable Diffusion. There was a brief period in late last year when Midjourney V3 had a "--beta" parameter that generated images using a separate Stable Diffusion-bases model. Then Emad (Stability AI) miscommunicated / oversold what was happening, which is where the confusion came from. They also dropped SD altogether for V4 and onwards. (More good discussions about this in the comments here: https://www.reddit.com/r/StableDiffusion/comments/10liqip/if_midjourney_runs_stable_diffusion_why_is_its/)
Then again, since the MJ model is a black box, it's hard to have a definitive read on it.
I've actually been wondering just how difficult it was to "plug into" the underlying LLM model from a coding perspective.
In my simple brain it was about "just" making some boxes for input that send it to the model and "just" some UI to make the output formatted to whatever the interface/app is supposed to do.
So I kind of assumed switching which model you communicate with (Claude, GPT, or other) was a relatively simple tweak. Sounds like it's a lot more complex and the degree of lock-in isn't trivial.
By the way, I'm quite sure Midjourney isn't built on top of Stable Diffusion. There was a brief period in late last year when Midjourney V3 had a "--beta" parameter that generated images using a separate Stable Diffusion-bases model. Then Emad (Stability AI) miscommunicated / oversold what was happening, which is where the confusion came from. They also dropped SD altogether for V4 and onwards. (More good discussions about this in the comments here: https://www.reddit.com/r/StableDiffusion/comments/10liqip/if_midjourney_runs_stable_diffusion_why_is_its/)
Then again, since the MJ model is a black box, it's hard to have a definitive read on it.