ChatGPT上使用Stable Diffusion实现AI绘画教程

ChatGPT Plugin是由OpenAI推出的基于ChatGPT的开发者工具。ChatGPT插件扩展了ChatGPT的能力，允许它与开发者创建的特定API进行互动。在这些插件的帮助下，ChatGPT可以执行各种任务，如获取最新的信息，如体育比分、股票价格或最新消息，协助用户完成任务，如预订机票或订购食品，提供有用的指导和支持。

那么今天主要教大家如何制作一款ChatGPT插件，通过输入提示词来实现Stable Diffusion在GPT里的AI绘画功能。

前提条件

下载与你的操作系统兼容的Visual Studio Code，或使用任何其他的代码编辑器，如：IntelliJ IDEA,PyCharm等。要使用ChatGPT插件API，你需要加入插件等待名单。要使用Stable Diffusion，我们需要API密钥，请到Dream Studio，创建一个账户并获取你的API密钥。

开始使用

第一步 – 创建一个新的项目

让我们从为我们的项目创建新的文件夹开始。打开Visual Studio Code，在终端运行以下命令，创建名为chatgpt-plugin-stable-diffusion 的新文件夹：

mkdir chatgpt-plugin-stable-diffusion
cd chatgpt-plugin-stable-diffusion

快速提示：为了使插件正常工作，我们需要通过以下步骤：

• 建立一个实现OpenAPI规范的API（Flask、FastAPI等）。
• 创建JSON清单文件，为插件定义相关元数据
• 以OpenAPI yaml或JSON格式记录API。

第2步 – API的实现

作为第一步，我们应该实现API。在本教程中，我将使用Flask，一个轻量级的WSGI Web应用框架。它被设计成能够快速、简单地入门，并能够扩展到复杂的应用程序。然而，你可以自由地使用任何其他框架，如：FastAPI、Django、Starlette等。

让我们创建一个名为app.py 的新文件。打开终端，运行以下命令：

touch app.py

现在是安装所需依赖项的时候了。在终端运行下面的命令：

pip install flask 
pip install stability-sdk

现在，我们可以开始实现API了。首先，我们需要导入所需的依赖项：

from flask import Flask, request, jsonify, send_file, Response
from stability_sdk import client
import stability_sdk.interfaces.gooseai.generation.generation_pb2 as generation
import base64

接下来，我们需要设置Flask应用程序和Stable Diffusion API客户端：

# Set up the Flask app
app = Flask(__name__)

# Set up the Stable Diffusion API client
stability_api = client.StabilityInference(
    key='',  # Replace with your Stability API key
    verbose=True,  # Set to True to enable verbose logging
    engine="stable-diffusion-xl-beta-v2-2-2"  # Replace with the engine you want to use
)

最后，我们需要定义用于生成图像的API端点和在本地启动开发服务器的方法：

# Define the API endpoint for generating images
@app.route('/generate-image', methods=['POST'])
def generate_image():

    # Get the prompt and other parameters from the request
    data = request.json  # Get the request payload
    prompt = data.get('prompt')  # Prompt to generate the image from
    seed = data.get('seed', None)  # Set to None to use a random seed
    steps = data.get('steps', 30)  # Number of steps to run the diffusion for
    cfg_scale = data.get('cfg_scale', 8.0)  # Scale of the diffusion model
    width = data.get('width', 512)  # Width of the generated image
    height = data.get('height', 512)  # Height of the generated image
    samples = data.get('samples', 1)  # Number of samples to generate

    # Generate the image using Stable Diffusion
    answers = stability_api.generate(  # Call the generate() method
        prompt=prompt,
        seed=seed,
        steps=steps,
        cfg_scale=cfg_scale,
        width=width,
        height=height,
        samples=samples,
        sampler=generation.SAMPLER_K_DPMPP_2M
    )

    # Retrieve the generated image(s) from the response
    # Extract and encode the generated image(s) so that they can be easily transmitted in the API response as a JSON object
    generated_images = []
    for resp in answers:
        for artifact in resp.artifacts:
            if artifact.type == generation.ARTIFACT_IMAGE:
                encoded_image = base64.b64encode(artifact.binary).decode('utf-8')  # encodes the binary data of the image using Base64 encoding, then decoded to a UTF-8 string using 
                generated_images.append(encoded_image)

    # Return the generated image(s) as the API response
    return jsonify(images=generated_images)


# Run the Flask app locally
if __name__ == '__main__':
    # Set debug=True to enable auto-reloading when you make changes to the code
    app.run(debug=True, host='127.0.0.1', port=5000)

快速提示：debug=True 参数可以在对代码进行修改时自动重新加载服务器，这在开发过程中很有用。host='127.0.0.1' 参数设置服务器的IP地址为localhost，port=5000 参数设置服务器的监听端口号。

完美！现在我们可以运行Flask应用程序了！现在我们可以运行Flask应用程序并测试API端点：

创建新的test.py 文件并复制/粘贴以下代码：

import requests
import json
import base64

# Define the API endpoint URL
url = 'http://127.0.0.1:5000/generate-image'

# Set up the request payload
payload = {
    'prompt': 'a donut',
    'seed': 992446758,
    'steps': 30,
    'cfg_scale': 8.0,
    'width': 512,
    'height': 512,
    'samples': 1
}

# Send the POST request to generate the image
response = requests.post(url, json=payload)

# Check the response status code
if response.status_code == 200:
    # Get the generated images from the response
    data = response.json()
    generated_images = data['images']

    # Process the generated images
    for i, encoded_image in enumerate(generated_images):
        # Decode the base64-encoded image
        decoded_image = base64.b64decode(encoded_image)

        # Save the image to a file
        image_filename = f'generated_image_{i}.png'
        with open(image_filename, 'wb') as image_file:
            image_file.write(decoded_image)

        print(f'Saved generated image {i+1} as {image_filename}')
else:
    print(f'Request failed with status code {response.status_code}')

不要忘记在终端运行pip install requests ，安装requests 库。

现在，运行Flask应用程序：python app.py，确保Flask应用正常运行。然后，打开新的终端窗口，运行测试脚本：python test.py, 几秒钟后，你应该在你的文件夹中看到生成的图像。

祝贺你！我们已经成功地建立了API，以满足你的需求！我们已经成功地建立了使用Stable Diffusion技术生成图像的API。请自由地进一步探索和尝试不同的提示和参数，以释放使用Stable Diffusion的图像生成的全部潜力。

第3步 – 插件清单

每个插件都需要一个ai-plugin.json 文件，该文件稍后将被托管在API的域中。在这里阅读更多信息。

创建名为ai-plugin.json 的新文件并粘贴以下代码：

{
    "schema_version": "v1",
    "name_for_human": "Image Generation Plugin",
    "name_for_model": "image-generation",
    "description_for_human": "Plugin for generating high-quality images using Stable Diffusion.",
    "description_for_model": "Plugin for generating high-quality images using Stable Diffusion.",
    "auth": {
        "type": "none"  # No authentication required, but you can use "api_key" or "oauth2" instead
    },
    "api": {
        "type": "openapi",
        "url": "http://127.0.0.1:5000/openapi.yaml"  # URL to the OpenAPI specification
    },
    "logo_url": "http://127.0.0.1:5000/logo.png",  # URL to the plugin logo
    "contact_email": "support@example.com",
    "legal_info_url": "https://example.com/legal"
}

第4步 – OpenAPI规范

OpenAPI规范是一个描述REST API的标准。它被用来定义插件将用于与模型通信的API。在这里阅读更多。

创建名为openapi.yaml 的新文件并复制/粘贴以下代码：

openapi: 3.0.1
info:
  title: Image Generation Plugin
  description: A plugin that generates high-quality images using Stable Diffusion.
  version: "v1"
servers:
  - url: http://127.0.0.1:5000  # URL to the Flask app
paths:
  /generate-image:
    post:
      operationId: generateImage
      summary: Generate an image
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/generateImageRequest"
      responses:
        "200":
          description: OK
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/generateImageResponse"

components:
  schemas:
    generateImageRequest:
      type: object
      required:
        - prompt
      properties:
        prompt:
          type: string
          description: The prompt for image generation.
          required: true
        seed:
          type: integer
          description: The seed for deterministic generation.
    generateImageResponse:
      type: object
      properties:
        images:
          type: array
          items:
            type: string
          description: The generated image(s) in base64 format.

现在，回到app.py 文件，为插件的标识、清单和OpenAPI规范添加端点：

# Define the API endpoint for the plugin logo
@app.route('/logo.png', methods=['GET'])
def plugin_logo():
    filename = 'logo.png'
    return send_file(filename, mimetype='image/png')


# Define the API endpoint for the plugin manifest
@app.route('/ai-plugin.json', methods=['GET'])
def plugin_manifest():
    host = request.headers['Host']
    with open('./ai-plugin.json') as f:
        text = f.read()
        return Response(text, mimetype='text/json')


# Define the API endpoint for the OpenAPI specification
@app.route('/openapi.yaml', methods=['GET'])
def openapi_spec():
    host = request.headers['Host']
    with open('./openapi.yaml') as f:
        text = f.read()
        return Response(text, mimetype='text/yaml')

全部完成!现在你已经完成了所有这些，你已经完成了所有的四个步骤，你有了一个已经启动并运行的服务器，你的功能已经准备好与之对话，现在是将其与ChatGPT实际整合的步骤。

以下是app.py 文件的完整代码：

from flask import Flask, request, jsonify, send_file, Response
from stability_sdk import client
import stability_sdk.interfaces.gooseai.generation.generation_pb2 as generation
import base64


# Define the API endpoint for generating images
@app.route('/generate-image', methods=['POST'])
def generate_image():

    # Get the prompt and other parameters from the request
    data = request.json  # Get the request payload
    prompt = data.get('prompt')  # Prompt to generate the image from
    seed = data.get('seed', None)  # Set to None to use a random seed
    steps = data.get('steps', 30)  # Number of steps to run the diffusion for
    cfg_scale = data.get('cfg_scale', 8.0)  # Scale of the diffusion model
    width = data.get('width', 512)  # Width of the generated image
    height = data.get('height', 512)  # Height of the generated image
    samples = data.get('samples', 1)  # Number of samples to generate

    # Generate the image using Stable Diffusion
    answers = stability_api.generate(  # Call the generate() method
        prompt=prompt,
        seed=seed,
        steps=steps,
        cfg_scale=cfg_scale,
        width=width,
        height=height,
        samples=samples,
        sampler=generation.SAMPLER_K_DPMPP_2M
    )

    # Retrieve the generated image(s) from the response
    generated_images = []
    for resp in answers:
        for artifact in resp.artifacts:
            if artifact.type == generation.ARTIFACT_IMAGE:
                encoded_image = base64.b64encode(artifact.binary).decode('utf-8')
                generated_images.append(encoded_image)

    # Return the generated image(s) as the API response
    return jsonify(images=generated_images)


# Define the API endpoint for the plugin logo
@app.route('/logo.png', methods=['GET'])
def plugin_logo():
    filename = 'logo.png'
    return send_file(filename, mimetype='image/png')


# Define the API endpoint for the plugin manifest
@app.route('/ai-plugin.json', methods=['GET'])
def plugin_manifest():
    host = request.headers['Host']
    with open('./ai-plugin.json') as f:
        text = f.read()
        return Response(text, mimetype='text/json')


# Define the API endpoint for the OpenAPI specification
@app.route('/openapi.yaml', methods=['GET'])
def openapi_spec():
    host = request.headers['Host']
    with open('./openapi.yaml') as f:
        text = f.read()
        return Response(text, mimetype='text/yaml')


# Run the Flask app locally
if __name__ == '__main__':
    # Set debug=True to enable auto-reloading when you make changes to the code
    app.run(debug=True, host='127.0.0.1', port=5000)

打开https://chat.openai.com/, go over to the 插件商店>开发你自己的插件and then click我的清单已经准备好了, copy/paste the base link to our app which is https://127.0.0.1:5000`在我们的例子中。然后点击find manifest file >next >install for me >continue 和install plugin 。

WooW!我们的插件安装好了，可以使用了。现在按你的要求使用它们吧!

结语!

在本教程中，我们探讨了使用Stable Diffusion构建ChatGPT插件生成图像的过程。通过利用Stable Diffusion的力量，我们可以增强ChatGPT的能力，根据单一的文字提示生成逼真而多样的图像。

顺便说一下，插件在扩展ChatGPT的功能方面起着至关重要的作用，使其能够与外部应用程序和API进行交互。