Creating a Zulip bot with ChatGPT
Posted in Research on March 26, 2023 by parallelo3301 ‐ 13 min read
My prompts and notes are usually shown as a quote, the rest is generated by the GPT.
GPT-3.5
I’ve started with GPT-3.5 as the number of prompts to GPT-4 is being limited. Let’s start with our simple idea:
Prompt: I need to write Zulip interactive bot. Give me an example which will react to “/gpt” command.
Sure, I can provide an example of how to write a Zulip interactive bot that reacts to the “/gpt” command. Here’s an example using the Zulip Python API and the OpenAI API to generate text using the GPT-3 model:
import openai
import zulip
from typing import Any, Dict
# Initialize the OpenAI API key
openai.api_key = "YOUR_API_KEY"
# Initialize the Zulip API client
client = zulip.Client(config_file="~/zuliprc")
# Define the function that will be called when the bot receives a message
def process_message(event: Dict[str, Any], bot_handler: Any) -> None:
# Check if the message contains the /gpt command
if "/gpt" in event["content"]:
# Get the text to generate from the message content
text = event["content"].replace("/gpt", "").strip()
# Generate text using the OpenAI GPT-3 model
response = openai.Completion.create(
engine="text-davinci-002",
prompt=text,
max_tokens=1024,
n=1,
stop=None,
temperature=0.5,
)
# Send the generated text as a reply to the message
bot_handler.send_reply(event, response.choices[0].text)
# Register the function to be called when the bot receives a message
client.call_on_each_message(process_message)
# Start the bot
client.run_forever()
In this example, the bot listens for messages in a Zulip stream and checks if the message contains the “/gpt” command. If the message contains the command, the bot uses the OpenAI API to generate text based on the remaining content of the message, and sends the generated text as a reply to the message.
Note that you will need to replace “YOUR_API_KEY” with your actual OpenAI API key, and configure the Zulip API client with your Zulip credentials. You may also want to customize the OpenAI API parameters, such as the engine and temperature, to suit your specific use case.
Seems like a promising start, let’s continue.
Prompt: add some console logs to see what’s happening
Sure, here’s the same example with some console logs added to show what’s happening:
import openai
import zulip
from typing import Any, Dict
# Initialize the OpenAI API key
openai.api_key = "YOUR_API_KEY"
# Initialize the Zulip API client
client = zulip.Client(config_file="~/zuliprc")
# Define the function that will be called when the bot receives a message
def process_message(event: Dict[str, Any], bot_handler: Any) -> None:
print("Received message:", event["content"])
# Check if the message contains the /gpt command
if "/gpt" in event["content"]:
# Get the text to generate from the message content
text = event["content"].replace("/gpt", "").strip()
print("Generating text with OpenAI GPT-3 for:", text)
# Generate text using the OpenAI GPT-3 model
response = openai.Completion.create(
engine="text-davinci-002",
prompt=text,
max_tokens=1024,
n=1,
stop=None,
temperature=0.5,
)
generated_text = response.choices[0].text.strip()
print("Generated text:", generated_text)
# Send the generated text as a reply to the message
bot_handler.send_reply(event, generated_text)
print("Sent reply:", generated_text)
# Register the function to be called when the bot receives a message
client.call_on_each_message(process_message)
# Start the bot
client.run_forever()
In this example, console logs have been added to show when a message is received, what text is being generated, what the generated text is, and when the reply is sent. This can be useful for debugging and understanding the flow of the bot.
Resolving bugs
Seems good for now. I’ve added the
zuliprcfile and replaced theYOUR_API_KEY, and tried to run it, but got the error.
Prompt: getting this error: process_message() missing 1 required positional argument: ‘bot_handler’
If you are getting the error message “process_message() missing 1 required positional argument: ‘bot_handler’”, it means that the process_message() function is expecting a second argument called bot_handler, but it is not being passed in when the function is called.
This error can occur if you are using an older version of the Zulip Python API that requires the bot_handler argument, but you are not passing it in correctly.
To fix this error, you can modify the line where the process_message() function is registered with the Zulip API client to include the bot_handler argument, like this:
client.call_on_each_message(process_message, include_submessages=True)
In this example, the include_submessages=True argument is optional, but can be useful if your bot needs to process submessages within a message.
Alternatively, you can modify the process_message() function to not require the bot_handler argument, like this:
def process_message(event: Dict[str, Any]) -> None: ...
In this example, the bot_handler argument has been removed from the function signature, so it is no longer required. However, if you need to send a reply message from within the function, you will need to use the client variable to do so, like this:
client.send_reply(event, "your reply message")
By making one of these modifications, you should be able to fix the “process_message() missing 1 required positional argument: ‘bot_handler’” error.
Another error, let’s ask about it.
Prompt: another error: AttributeError: ‘Client’ object has no attribute ‘send_reply’
If you are getting the error message “AttributeError: ‘Client’ object has no attribute ‘send_reply’”, it means that the Zulip Python API version you are using does not have the send_reply method for the Client object.
This error can occur if you are using an older version of the Zulip Python API that does not have the send_reply method, or if you are using a different API method that is not compatible with the version of the API you are using.
To fix this error, you can modify the line where the reply message is sent to use the send_message method instead, like this:
client.send_message({
"type": "private",
"to": event["sender_email"],
"content": generated_text
})
In this example, the send_message method is used to send a private message containing the generated text to the sender of the original message.
Alternatively, you can try upgrading the Zulip Python API to a newer version that has the send_reply method. You can do this by running the following command:
pip install zulip --upgrade
This will upgrade the Zulip Python API to the latest version, which should have the send_reply method. Note that you may need to modify your code to use the new version of the API.
By making one of these modifications, you should be able to fix the “AttributeError: ‘Client’ object has no attribute ‘send_reply’” error.
Somehow fixed it. Now, this change meant that the bot will be able to respons to private messages only. The responses are long, so I’ll try to make it shorter with a prompt and the response description.
More tl;dr
Prompt: how to respond to a origin and not private message only?
Response: properly printed the code
Now it works, but everything lives in one big handler which answers to /gpt only and doesn´t remember previous context.
Prompt: add context remembering with previous messages
Response: added code to fetch 1 previous message
Prompt: get previous messages with maximum possible tokens for used model
Response: added token limit, but still fetched only 1 previous message
Prompt: modify it to obtain as many previous messages as possible to fill the token limit
Response: Yes. It added a loop to fetch one-by-one message until token limit is met. BUT! It has a bug. Two ways to fix it: modify the anchor parameter, or fetch more messages with one request.
Prompt: this will fetch the same message multiple times, as you are not modifying the num_before
Response: It did both solutions - properly modified anchor with the last fetched message, but introduced new bug with num_before.
Prompt: you properly modified the anchor, so num_before modification is not needed, right?
Response: Fixed.
Prompt: great. now make it possible to omit the /gpt command if it’s a private message
Response: It removed the if which checked for /gpt
Prompt: Yes, but how this will work in public streams? Wouldn’t it generate a request with every message without even mentioning the bot?
Response: Added if to check for bot mention or /gpt command
Prompt: great. Now add check for a private message where mention nor /gpt is required
Response: Ok. But it turned to a spaghetti code, as there are two big if branches. But let’s focus on different things.
Prompt: rewrite with not using the Zulip Botserver - bot_handler doesn’t exist
Response: rewrote it to be a little bit better.. but I have decided to try GPT-4 at this moment with better defined prompt based on this exploratory part.
GPT-4
Prompt: Write a zulip bot (not using the Zulip Botserver, so bot_handler is not present). The code should log what’s happening. Bot have will use GPT-3 API to respond to user message. Bot can be triggered by a private message, where you directly ask the question. Another way is a public stream, where you need to activate it by typing /gpt message, or by using the mention. Bot should be able to read previous messages up to the given model token limit (e.g. 8K tokens), so it uses as much context as possible.
To create a Zulip bot that uses the GPT-3 API without the Zulip Botserver, you can follow these steps:
- Install the required libraries:
pip install zulip pip install openai- Create a
gpt3_zulip_bot.pyfile and paste the following code:
import os
import logging
import re
import openai
import zulip
# Set up logging
logging.basicConfig(level=logging.INFO)
# Set up GPT-3 API key
openai.api_key = os.environ['OPENAI_API_KEY']
# Set up Zulip client
client = zulip.Client(config_file="zuliprc")
# GPT-3 model name
MODEL_NAME = "text-davinci-002"
def send_reply(reply, message):
if message['type'] == 'private':
response = {
'type': 'private',
'to': message['sender_email'],
'content': reply,
}
else:
response = {
'type': 'stream',
'to': message['display_recipient'],
'subject': message['subject'],
'content': reply,
}
client.send_message(response)
def get_gpt3_response(prompt):
response = openai.Completion.create(
engine=MODEL_NAME,
prompt=prompt,
max_tokens=150,
n=1,
stop=None,
temperature=0.5,
)
return response.choices[0].text.strip()
def handle_message(event):
msg = event['message']
content = msg['content']
if msg['type'] == 'private' or re.search("/gpt|@**gpt3-bot**", content):
if msg['type'] == 'stream':
content = re.sub("/gpt|@**gpt3-bot**", "", content)
prompt = f"{content}"
response = get_gpt3_response(prompt)
send_reply(response, msg)
def main():
logging.info("Starting the GPT-3 Zulip bot...")
client.call_on_each_event(handle_message, event_types=['message'])
if __name__ == "__main__":
main()
- Create a
zuliprcfile in the same directory with the following contents:
[api]
email=BOT_EMAIL
key=BOT_API_KEY
site=ZULIP_SITE
Replace BOT_EMAIL, BOT_API_KEY, and ZULIP_SITE with your bot’s credentials and your Zulip server URL.
- Set the environment variable
OPENAI_API_KEYto your GPT-3 API key:
export OPENAI_API_KEY=your_api_key
- Run the bot:
python gpt3_zulip_bot.py
The bot should now be active and able to respond to private messages and public stream messages that mention it with @gpt3-bot or start with /gpt. Note that this example assumes a relatively simple conversation structure and may not fully utilize the available token limit. To read previous messages and use more context, you can modify the handle_message function to fetch more messages from the conversation history using the Zulip API.
Prompt: write that functionality to fetch those previous messages to get up to 2K tokens
Response: ok
Prompt: how to load .env file in python
Response: properly writes the code + offers the example with currently existing code like
# Set up Zulip client
client = zulip.Client(email=os.environ['BOT_EMAIL'],
api_key=os.environ['BOT_API_KEY'],
site=os.environ['ZULIP_SITE'])
Prompt: what will be the best way to add functionality to start new conversation without the history?
You can add a simple command to your bot to start a new conversation without including the message history. Let’s say you want to use the command /gpt new to start a new conversation without history. Modify the handle_message function in the gpt3_zulip_bot.py file as follows:
def handle_message(event):
msg = event['message']
content = msg['content']
if msg['type'] == 'private' or re.search("/gpt|@**gpt3-bot**", content):
if msg['type'] == 'stream':
content = re.sub("/gpt|@**gpt3-bot**", "", content)
if content.strip().lower() == 'new':
prompt = f"{content}"
else:
previous_messages = fetch_previous_messages(client, msg)
prompt = f"{previous_messages}\n{content}"
response = get_gpt3_response(prompt)
send_reply(response, msg)
Now, when you send the message /gpt new or @gpt3-bot new to the bot, it will start a new conversation without using the message history. For other messages, the bot will include the previous messages in the conversation as context, as implemented earlier.
GitHub Copilot
From now on, I just try to run and debug it. First issue was that the bot will answer even it’s own “prompts”.
So I’ve started to write:
if ms
# what Copilot suggests
if msg['sender_email'] == client.email:
return
I didn’t even know how specifically I’ll do the check, but Copilot answered it for me. :-)
Another issue: The script always failed like 30 seconds after the start. Fortunately, I had a debugger at the handler beginning, so I found out that there is a heartbeat type of message.
if eve
# what Copilot suggests
if event['type'] != 'message':
return
We are almost done here.
There were some other issues, like having the messages in reverse order if there was a chat history. The GPT code appended the messages, so the prompt history was in a reverse order. Easily fixable.
The bot should remove the bot mentions, and/or remove the new keyword when starting new conversation.
And the last one issue (at least for now): The Zulip messages come as HTML, so there is a lot of <p> tags, etc. So simple tag stripping has to be added. And we have a final code (at least for now).
Next steps to do
This bot now uses the InstructGPT Davinci model. It will be more suitable to use the ChatGPT model for this purpose, plus it’s much more cheaper option too.
- Use the
gpt-3.5-turbomodel - Set adequate token limits
- Use
messages arrayinstead of one big prompt with new lines
Final code
File structure
|- .env
|- gptbot.py
|- requirements.txt
|- zuliprc
requirements.txt
zulip==0.8.2
openai==0.27.2
python-dotenv==1.0.0
zuliprc
[api]
email=gpt-bot@zulipinstanceurl
key=privkey
site=https://zulipinstanceurl
.env
OPENAI_API_KEY=xy
MODEL_TOKEN_SENT_LIMIT=3000
MODEL_TOKEN_RECEIVED_LIMIT=1000
MODEL_NAME="text-davinci-002"
gptbot.py
import os
import logging
import re
import openai
from collections import deque
import zulip
from dotenv import load_dotenv
from io import StringIO
from html.parser import HTMLParser
# Load the .env file
load_dotenv()
# Set up logging
logging.basicConfig(level=logging.INFO)
# Set up GPT-3 API key
openai.api_key = os.environ['OPENAI_API_KEY']
# Set up Zulip client
client = zulip.Client(config_file="zuliprc")
MODEL_TOKEN_SENT_LIMIT = int(os.environ['MODEL_TOKEN_SENT_LIMIT'])
MODEL_TOKEN_RECEIVED_LIMIT = int(os.environ['MODEL_TOKEN_RECEIVED_LIMIT'])
MODEL_NAME = os.environ['MODEL_NAME']
class MLStripper(HTMLParser):
def __init__(self):
super().__init__()
self.reset()
self.strict = False
self.convert_charrefs= True
self.text = StringIO()
def handle_data(self, d):
self.text.write(d)
def get_data(self):
return self.text.getvalue()
def strip_tags(html):
s = MLStripper()
s.feed(html)
return s.get_data()
def send_reply(reply, message):
if message['type'] == 'private':
response = {
'type': 'private',
'to': message['sender_email'],
'content': reply,
}
else:
response = {
'type': 'stream',
'to': message['display_recipient'],
'subject': message['subject'],
'content': reply,
}
client.send_message(response)
def get_gpt3_response(prompt):
response = openai.Completion.create(
engine=MODEL_NAME,
prompt=prompt,
max_tokens=MODEL_TOKEN_RECEIVED_LIMIT,
n=1,
stop=None,
temperature=0.5,
)
return response.choices[0].text.strip()
def fetch_previous_messages(client, message, token_limit=MODEL_TOKEN_SENT_LIMIT):
if message['type'] == 'private':
query = {
'anchor': message['id'],
'num_before': 50, # adjust this value as needed
'num_after': 0,
'narrow': [{'operand': message['sender_email'], 'operator': 'pm-with'}],
}
else:
query = {
'anchor': message['id'],
'num_before': 50, # adjust this value as needed
'num_after': 0,
'narrow': [{'operand': message['display_recipient'], 'operator': 'stream'}],
}
previous_messages = client.get_messages(query)['messages']
previous_messages.reverse()
prompt = ""
token_count = 0
stop = False
for msg in previous_messages:
content = strip_tags(msg['content'])
if message['type'] == 'stream':
content = re.sub("/gpt|@GPT", "", content)
content_chunks = content.strip().lower().split()
if content_chunks and content_chunks[0] == 'new':
content = content.replace('new ', '', 1)
stop = True
tokens = len(content.split())
token_count += tokens
if token_count > token_limit:
break
prompt = f"{content}\n{prompt}"
if stop:
break
return prompt.strip()
def handle_message(event):
if event['type'] != 'message':
return
msg = event['message']
content = strip_tags(msg['content'].strip())
if msg['sender_email'] == client.email:
return
if msg['type'] == 'private' or re.search("/gpt|@\*\*GPT\*\*", content):
if msg['type'] == 'stream':
content = re.sub("/gpt|@\*\*GPT\*\*", "", content)
content_chunks = content.strip().lower().split()
if content_chunks and content_chunks[0] == 'new':
prompt = f"{content.replace('new ', '', 1)}"
else:
content_tokens = len(content.split())
previous_messages = fetch_previous_messages(client, msg, MODEL_TOKEN_SENT_LIMIT - content_tokens)
prompt = f"{previous_messages}"
response = get_gpt3_response(prompt)
send_reply(response, msg)
def main():
logging.info("Starting the GPT-3 Zulip bot...")
client.call_on_each_event(handle_message, event_types=['message'])
if __name__ == "__main__":
main()
