Stay Curious and Play — My attempt to make use of generative AI at work
I’m by far not the hype-driven type (anymore). In general, I don’t use new tools or “things” just because they are hyped. However, I have to admit that I have fallen in love with generative AI since OpenAI announced its first(?) public product: ChatGPT. It felt like magic to me that a computer — let’s face it, it’s nothing more than a software program running on a computer — could generate answers (regardless of the quality) when you type in a question. I still remember the first week after I signed up for it. I spent hours asking ChatGPT how generative AI worked. Always keeping in mind that it might not be 100% correct. But still, it was good enough for someone who didn’t know anything about it. Of course, more follow-up research was needed. But at least it gave me a first glimpse of how things worked.
In short, I like generative AI.
So does a colleague of mine.
In the beginning, we only discussed this “new thing” from time to time. That changed at our annual company conference, BeeCon. He proposed the topic of having a Community of Practice (CoP) for AI. Not knowing what we would do there and how it would end. But that was the start for me to explore AI a little bit more in detail. I even went to an AI meetup and gave a talk (slides — just the basics, don’t expect anything fancy here). Yes, as an Android Developer.
I came up with an idea for the CoP after I suddenly got an email from an internal (support) ticket (GitHub issue) that mentioned me and already had +30 comments at that time. I scrolled through the ticket to get the gist. I replied with a more general answer — not knowing exactly what these people had discussed before and why I was mentioned in the first place.
Afterwards, I felt a little bit bad because I didn’t answer “in a good way”. But at the same time, I was also not in the mood to read all of those comments.
So suddenly my brain started working. It was juggling with a lot comments, GitHub issues, GitHub API, generative AI, ChatGPT, OpenAI API, …
What if… what if you could type a comment on a GitHub issue like
“AI summarize please”, and after some time “AI commented” back on that issue with a summary of the discussion?
Would that be possible? Hell yes, it is!
So I proposed the idea in our PoC. Of course, privacy concerns were raised. We shouldn’t expose all our internal/private comments to a public AI provider. We’re probably exposing things that shouldn’t be exposed. And you really don’t know what these providers will do with that input. Yes, we’re Germans, we don’t trust ToC anyway and the claim in there “we won’t use your data to train LLMs” doesn’t help either. So OpenAI (and siblings like Antrophic, Gemini, Mistral, you name it) are out. Luckily, we already had a running server with Ollama installed.
Note: Ollama is a software to run some AI models locally (or on a server) on your own hardware. So things will never get out of your hands.
Since people come in all shapes and different sizes and therefore use software in different ways, we decided to split up our software into two parts. The “core” summarizer, which handles both “fetching GitHub issues” and “communicating with Ollama”, and a GitHub Action that would handle the communication with GitHub. Like listening to the AI trigger and commenting back on the issue — using the core summarizer to do that.
Once we had our first core prototype, which also consists of a CLI, we thought we could adapt it to support GitLab too (we use GitHub and GitLab at work, don’t ask why). I realized that the code was already quite adaptable to support more “sources”. So I quickly added GitLab as a possible source for comments. While we are on the subject, we also use Slack. Can we add Slack (threads) as a source for comments? Sure we can. Even Reddit threads can be supported. So we did.
Since we support multiple input sources (GitHub, GitLab, Slack, etc.), wouldn’t it be great to support also multiple AI providers as well? Besides Ollama, we also wanted to support OpenAI, Anthropic (Claude), Google (Gemini), and so on. Yes, we are not allowed to use the latter at work, but as an open source enthusiast, I wanted to publish it on GitHub anyway. Maybe someone else is allowed to use it. Or maybe I want to use it for non-work related stuff.
With that summaraizer was born.
Its sibling project, summaraizer-action, is just a thin wrapper around the summaraizer. It can be used in your GitHub Action workflow to handle the communication between GitHub and the summaraizer. Only a few lines of code are required to use it:
- name: Summarize
uses: ioki-mobility/summaraizer-action@main
with:
provider: 'ollama'
provider-args: '--url https://ollama.example.com --model llama3.2'A full example can be found on the repos README.
Today, almost all of our GitHub repositories (android, ios, bugs, backlog, pit-backlog) at work can make use of it. The actions are set up and ready to go. All it takes is a single comment — AI summarize please — is needed to create a non-deterministic summary of the issue.
I’m currently actively working on these projects. I’m looking forward to your feedback. Contributions are of course also welcome.
Oh, and before you go, two more things:
- There is also a summaraizer-slack project. You might want to check it out as well!
- Did you know that you can run Ollama in GitHub Actions? If you don’t want to pay for an AI provider (hint: Gemini has a free tier), you might want to use this information somehow!
