add stable diffusion slackbot blog - #3
Conversation
|
|
||
| ## Introduction | ||
|
|
||
| Reinforcement Learning from Human Feedback (RLHF) is a powerful tool for improving the performance and alignment of generative models through the direct incorporation of human preferences into model reward functions. While its benefits for both [language modeling](https://arxiv.org/abs/2009.01325) and [image generation](https://arxiv.org/pdf/2302.12192.pdf) applications are well-documented, the pathways to the actual collection of this human evaluation data are often difficult and unclear. |
There was a problem hiding this comment.
For the second link can we do the abstract page again, just because of loading time when I click on it
|
|
||
| Reinforcement Learning from Human Feedback (RLHF) is a powerful tool for improving the performance and alignment of generative models through the direct incorporation of human preferences into model reward functions. While its benefits for both [language modeling](https://arxiv.org/abs/2009.01325) and [image generation](https://arxiv.org/pdf/2302.12192.pdf) applications are well-documented, the pathways to the actual collection of this human evaluation data are often difficult and unclear. | ||
|
|
||
| At Oxen, we provide a suite of dataset version control tooling that makes this data collection process fast, secure, and infinitely integrable with platforms your team and users already love. |
There was a problem hiding this comment.
Honestly...I'm cool to take this out, let's lean into the fact that we built this slackbot so that we could build out a large repo of generated Oxen instead of describing oxen in this one.
|
|
||
| **Image generation model** | ||
|
|
||
| - We’ll use the [`diffusers`](https://github.com/huggingface/diffusers) library and a stable diffusion model hosted on Hugging Face to generate our images. We’re using a model we’ve previously fine-tuned to generate cute cartoonish Oxen (see this tutorial), but is based off of `CompVis/stable-diffusion-v1-4`, which is a great starting point. |
There was a problem hiding this comment.
we should mention modal here as serverless compute
|
|
||
| Then remotely commit the image: | ||
|
|
||
| ```python |
There was a problem hiding this comment.
This indentation looks funny, and don't we only want to commit if the image is successfully added?
| repo.add_df_row(DF_PATH, row) | ||
| except Exception as e: | ||
| print('Error adding df row to Oxen', e) | ||
| repo.commit(f"Remote commit - {row['rater']} voting on image {row['path'].split('/')[-1]}") |
There was a problem hiding this comment.
Again, I think we only want to commit if add was successful
|
|
||
| The bot will render ox images in response to prompts from our team with the `/ox` slash command: | ||
|
|
||
|  |
There was a problem hiding this comment.
I'm wondering if we want to hook this up to stable diffusion XL and scratch the link to the other blog, and just say we are catching errors so that we can fine tune later...? Might get some more compelling images out of it.
There was a problem hiding this comment.
were you envisioning pure stable diffusion XL, or running dreambooth / lora something on it? / did you end up getting that to work? definitely down either way, these images make me a little sad
| python server.py | ||
| ``` | ||
|
|
||
| Using [ngrok](https://ngrok.com/), we can expose this port on a public IP to allow us to receive messages from the Slackbot while developing locally. |
There was a problem hiding this comment.
I actually think we should also use modal here to deploy the serverless flask app, what do you think?
There was a problem hiding this comment.
I thiiiink we talked about this a while ago but can't totally remember - definitely fine to do so to tighten up the post but I think it's probably not super cost-effective since we'll have to re-download the oxen lib after every slack reaction? and we also don't need GPUs at this stage..but agree that it's awk to tell people to use ngrok and then be like deployment's on you, go check out digital ocean
|
|
||
|  | ||
|
|
||
| Over the next few months, we’ll be using this bot to generate lots of additional training examples from the comfort of our Slack workspace. With this additional training data generated directly from human feedback, we can work on fine-tuning out some of the glitchier oxen (including the broccoli-ox hybrid shown above) and move towards a more consistent, adaptable ox character. |
There was a problem hiding this comment.
Let's actually do this with stable diffusion XL, a modal flask server, and a slack cron job that posts into the channel every morning. I think we can do a similar thing as our github stars repo and generate "random" prompts and then we can select the best generations.
2 participants
No description provided.