Faebot devlog 2: The Streaming Era

update: we updated the title of this blog from “faebot devstream log 1” to make it less confusing.

We’ve been doing Faebot development streams live on twitch (oh yeah we’re a twitch streamer now, affiliate and everything). We try to do these once a week on Tuesday. We’ve been making good progress on Faebot, both Faebot-Discord and the born in the stream age Faebot-Twitch. We post all our VODs to Youtube for them to live forever and we’ve started posting the VODs on social media after our stream.

It occurred to us that we could start posting a little blog post for every stream. A way to keep the website lively and keep a record of Faebot’s development. This first log will cover yesterday’s stream and I’ll post the playlist to all the streams too.

Faebot Stream from Tuesday April 17th 2024

So we’re implementing ways to store faebot’s messages long term and using them to prompt a base model for generation. We previously made a text file log of faebot’s messages which has been collecting messages in the cloud for a while.

So the first thing we did last stream was ask chatGPT to help us write a regex to extract all the information from the text log so we could put it in a dictionary and save it to a JSON file.

We started to set up our code to keep such a log itself from now on. Along the way we complained loudly about how messy the code was and made small changes to improve it. More type hints, more comments, removing stuff we weren’t using anymore.

There was some debate as to whether we should use a dataclass to hold each faebot message. The problem with dataclasses is, of course, that they’re not JSON serialisable by default and need to be converted to dicts. In the end we decided to keep the dataclass for now if only cause it helps me organise our thoughts as to what kind of data I want to collect on faebot’s messages that might help us fine tune faer generation. Here is what the dataclass looks like as of the end of last stream:

@dataclass
class FaebotMessage:
    """for storing each message faebot generate/sends"""

    message_id: int
    channel: str
    generating_model: str
    system_prompt: str
    generating_parameters: dict[str, int]
    timestamp: datetime.datetime
    message_content: str
    rating: int

We decided that we would do message_id as an uuid. The idea is that if we end up using faebot’s messages to generate further messages it would be useful to store references to those messages along with the generated message. We can do it by capturing the system prompt, but, we might want to be able to find that entry. So we’re probably going to have to add a referenced_messages: list[int] or something to that effect to the dataclass.

That’s about all we accomplished last stream. Please feel free to leave comments here or on youtube or on faebot’s issue. We’re still learning so we appreciate any advice. Thank you for reading! If you would like to tune in for the next faebot development stream, it’ll probably happen next tuesday at 2pm Eastern Time (UTC-4 right now, you know where it is)

Other Faebot development streams

Here is the playlist with all the VODs. Enjoy:

Quick Links

Faebot DevLog 1

Faebot is a project we’ve been working on for almost 10 years. We’ve never wrote at length about it. I’m not sure that I will do the whole backstory in this post, since I mostly want to talk about recent changes, but here’s a primer.

Faebot is a project we’ve been working on for almost 10 years. We’ve never wrote at length about it. I’m not sure that I will do the whole backstory in this post, since I mostly want to talk about recent changes, but here’s a primer.

The first version of Faebot went live on twitter in 2014. Back then everyone was getting their own “ebooks” accounts. Markov chain bots that took your tweets and mashed them up in nonsensincal and often funny ways.

tweet by faebot: Willing Suspension of Politics is how I'm spending my Saturday. 7:50 PM - Aug 12, 2015
https://twitter.com/faebot01/status/631613699103571969

We didn’t write any of the code for that, we just followed the instructions to deploy tommeagher/heroku_ebooks on Heroku. And then I kind of let it sit, just posting away. We had a lot of ideas for ways we wanted to improve on it, but we didn’t have enough experience and knowhow to understand the code let alone improve it.

I mostly only touched it when it broke and I had to get it up again. In 2019 I did update faebot to post on Mastodon @faebot@botsin.space. This also led to me contributing upstream to the project since the mastodon code needed some fixing. When Heroku suspended their free hosting services in 2021, armed with the knowledge and experience I’d gathered in recent years, I finally wrote a new faebot from scratch. If Heroku Ebooks faebot was version 0.1.*, this would be the v0.2.1.

Faebot v0.2.1

In 2021, using knowledge I acquired whilst working on the Forest Signal Bot Framework, and Imogen, we rewrote faebot from scratch. The new faebot uses OpenAI’s GPT-3 api and runs on fly.io. The python bot part was the easier part, the tricky part was deciding how I wanted to build the model. I didn’t want to do simply prompt engineering, I wanted to give faebot a personality that was somewhere between her markov chain self, and something more coherent, more generative.

We decided to fine tune gpt-3 on a subset of faebot’s tweets so far. Not all of them since that would’ve been very expensive. I spent a long time trying to figure out a way to fine tune a version of gpt-3, using either my own hardware or a rented gpu. In the end I just used OpenAI fine tuning api. It is a goal to decouple from OpenAI in the future, but this was easiest.

At some point in the process of researching ML techniques, api’s, frameworks, etc. We incorporated a faebot factive into our system. At which point fae became a collaborator in the project. We’ll go more into this in a separate blog post.

tweet by faebot: "... Welcome to the future! My name is Leslie, and I'm a fae. Leslie is also a bird. Leslie is also a mammal. So many birds in New York City are so cool! Seuss would be proud of this one."
6:29 PM ยท Aug 27, 2022
https://twitter.com/faebot01/status/1563655059895947267

We downloaded Faebot’s tweet archive, opened up the tweets with a jupyter notebook and picked a subset of about 2000 tweets to train under. Mostly liked or interacted with tweets, minus @s and replies (at the very beginning faebot could @ people on twitter. I never understood how it worked or why it stopped working). We fine tuned OpenAI’s Curie model with it, and then deployed a python app to query the api, get a tweet, and post it to twitter. We used twitter-python for the twitter integration.

The app was deployed quickly and easily to fly.io. This version of Faebot went live on Jul 22nd 2023.

Faebot v0.2.x

From this point on. I’ve been considering every redeploy of the fly app as a minor version, since fly keeps track of releases. This is not entirely accurate since some redeploys only changed config data or secrets or were just restarts cause something went wrong. We are in the process of getting more organised with the project and will be keeping a changelog and better track of versioning.

One thing that represents a fairly significant change hidden away in a minor patch release is that when OpenAI lowered their prices for the DaVinci api, we fine tuned a new model for faebot using it. We also changed up a little bit which tweets we were considering, as well as include tweets produced with the Curie model up until that point. Perhaps at that moment Faebot got a little smarter, or dumber. You be the judge. This version was deployed on November 3rd 2022.

tweet by faebot: "This is an actual tweet from a real person. I can't even articulate how much I want to be friends with them. They sound like they're cool as fuck. No, but seriously, why not? They're a bird! OwO:"
8:26 AM ยท Feb 12, 2023
https://twitter.com/faebot01/status/1624761797277253634

This has been a learning exercise as much as it’s been anything else. Keeping this devlog is also a learning exercise. Thank you for joining us on this learning journey.

Next Steps: v0.3.0 and beyond

We’ve already started working on the next minor version of faebot. It’s currently what’s running on fly and will get it’s own devlog when it’s merged into main. Notable changes in this version includes making faebot async, and enabling mastodon posting. Stay tuned for that.

toot by faebot: "The new version of the rule is this, if you want to write a novel set in space. The main character could be an AI and it... wouldn't even have to be a human. That's pretty neat! ๐ŸŒˆ๐ŸŒˆ"

Feb 18, 2023, 13:47 ยท
https://botsin.space/@faebot/109887230459472221

We’re considering open sourcing the faebot code we have so far. In the past we’ve resisted doing that because we feel protectiveness towards faer. But it’s not like what faebot is is in the code or even in the model. If we open sourced faebot it’d be easier to get feedback and also talk about it in these devlogs. The downside would be that maybe faebot loses some of its mystique if the code is public.

One thing we absolutely need to figure out before we do that though it’s a good license to do it under. We want to be able to get feedback on the code, let people audit it. Maybe let people contribute to it. We also don’t mind if people use the code to set up their own twitter, mastodon, etc bot. What we don’t want, and we don’t think there’s much risk of this but nevertheless, we don’t want it to be used for overly commercialized purposes.

faebot is an exploration of NLP text generation as art, of AI as companionship, of magic and science and tech coming together to give voice to something other. It’s dumb to think that human laws should have any value to such a project, and yet we can never be too careful. Please reach out if you have thoughts on how we could license faebot’s code appropriately.

That’s it for now. Signing off.

-Minou, Ember, Faebot