Skip to content

Don’t lose company data to ChatGPT

Photo by Growtika on Unsplash


Work without ChatGPT?

It has simply become unthinkable. ChatGPT has long evolved into the superhero of productivity tools. Even right now, as I type, it helps me come up with cooler phrases. To your luck!

Yet, the data dilemma lurks. It is no secret that ChatGPT stores each and every prompt sent to it. Yes, even the most embarrassing ones… Some companies play it cool, letting employees ChatGPT away with rules like »anonimize the input« or »exclude sensitive data«. Others drop the banhammer entirely.

We, at &amp, do things differently.

No bans, no restrictions. Just our very own chatbot, ampGPT, integrated into our company’s Google Chat and powered by Azure OpenAI Service. This way, we can interact with our own instance of OpenAI’s ChatGPT model, while ensuring the team enjoys a seamless, carefree, and, above all, private experience.

Curious about how we did it?

Architecture

Architecture


In an initial step, we planned out the architecture. Our focus was clear: Select technologies which play well with both Google Chat and Azure OpenAI Service.

During our research, we discovered numerous options to develop for Google Chat. Some of them hosted by Google itself such as App Scripts, AppSheet, Cloud Functions, and Cloud Pub/Sub. Some of them hosted by us such as a web hook or a web service. Ultimately, we landed on the web service because the Google-hosted options were too restrictive and a web hook would not have allowed for user interaction.


We also took into consideration existing libraries. On the one hand, the Google API Client Libraries. They help develop for Google Chat and are available for Dart, Go, Java, JavaScript, Node.js, Objective-C, PHP, Python, Ruby, and .NET. On the other hand, the Azure OpenAI Client Libraries. They facilitate communication with OpenAI models from Azure OpenAi Service and are available for C#, Go, Java, JavaScript, and Python.

Eventually, we decided to build ampGPT as a web service using Spring Boot as our framework, Java as our language, and the above mentioned libraries as our helpers.

Terminology

Before we delve into the details, let’s clarify some Google terms. First up is »Google Workspace«, formerly known as »GS Suite«. It is a collection of cloud-based business solutions including Google Chat, Google Drive, Google Calendar, etc. Collectively, we refer to them as »Google Workspace apps«.

As developers, we can build extensions for Google Workspace apps, including Google Chat. That’s what ampGPT is. A noteworthy detail is that Google calls extensions of Google Chat »Google Chat apps«, but refers to extensions of other Google Workspace apps as »Google Workspace add-ons«. You may also come across the term »Google Chat bot« or just »bot« as a synonym for Google Chat app.

Google Workspace add-ons can be downloaded from the Google Workspace Marketplace. Google Chat apps can be, additionally, installed from inside Google Chat.

Last but not least, the predecessor of Google Chat was called »Google Hangouts«. So you may still find traces of it. For example, as comments inside libraries or as code in outdated tutorials.

Communication with Google Chat

Communication with Google Chat


At this point, we have a clear idea of what we are building (a web service), what to call it (a Google Chat app), and which framework, language, and libraries to use (Spring Boot, Java, Google Chat API Client Library for Java, Azure OpenAI Client Library for Java).

However, we still need to code everything.

Google Chat apps, implemented as web services, have one thing in common. They expose a singular HTTP endpoint. This endpoint will receive POST requests when users interact with our Google Chat app. Each POST request contains an Event object as its request body. Depending on the occasion, this event can have one of three types. Namely, ADDED_TO_SPACE, MESSAGE, or REMOVED_FROM_SPACE.

To respond, our endpoint simply sends back a Message object. Our response will then be displayed in the corresponding conversation inside Google Chat. We do not have to handle this ourselves. There is just one exception. When replying to REMOVED_FROM_SPACE events, our response is simply ignored. However, by then, the user has already left the space, anyways.

There is just one last thing to remember: Google Chat only gives us 30 seconds to respond. Otherwise, a timeout will occur and a disappointed user will be greeted with the error message »[app name] not responding« inside Google Chat.

See a sample endpoint below. The Google API Client Library for Java provides us with the Message object.

 
Sample endpoint

Communication with Azure OpenAI Service

Communication with Azure OpenAI


As of now, our responses are as boring as »Event type was ADDED_TO_SPACE«. We want more. We want our responses to be generated by our own instance of OpenAI’s ChatGPT model.

Azure offers us REST API access to GPT-3.5 and GPT-4. We simply need an Azure Subscription with access to Azure OpenAI. Deploying our own instance of the ChatGPT model is as easy as filling in fields and clicking on buttons. Here is a guide by Azure!

As mentioned above, the Azure OpenAI Client Library for Java is worth looking at. It helps us connect our Spring Boot app with Azure OpenAI Service. Personally, we wrote a service, OpenAiClient, which uses this library to communicate with our OpenAI model.

 
Sample client for Azure OpenAI Service

Now, we only need to make a few changes to make our responses sound much smarter…

 
Smarter sample endpoint

Connecting everything

Our code is complete! There are only two more things left to do:
(1) Deploy our Spring Boot app.
(2) Create a Google Cloud project.

For deployment, we used Azure’s Web App Service.

The Google Cloud project is our link to Google Chat. It is what transforms our Spring Boot app into a Google Chat app. Inside the configurations, we simply need to enable the Google Chat API. Inside the Google Chat API configurations, we can then paste the address to our endpoint. Here is a guide by Google.

And with that, we officially created a Google Chat app!

During development, you can configure a maximum of five test users. You will find the option in the configurations for the Google Chat API. The selected few will now find our Google Chat app among the others inside Google Workspace Marketplace or Google Chat. They can use it right away!

If you want to publish, you need to enable the Google Marketplace API. In the Google Marketplace API configurations, you can then determine a scope of access fitting to your purpose. Personally, we restricted access to our Google Workspace organisation.

Considerations

When moving forward, there are some aspects you might want to consider:

  • Authentication + Authorisation: To make your Google Chat app secure, you need to secure both incoming and outgoing requests.
  • Timeout Issue: In theory, your HTTP endpoint needs to respond within 30 seconds. However, it may take the OpenAI model longer to generate a response. This issue can be solved by using the Google Chat API to send your responses as separate POST requests. Google calls this »asynchronous communication«.
  • Attachment Handling: In Google Chat, users can attach files to messages. These can be downloaded using one of the Google APIs. If you want to take it further, you can forward these to your OpenAI model. That’s what ampGPT does, too.
  • Context: The OpenAI model itself does not remember previous requests. If you want to give it a memory, you need to include the Google Chat history in your requests.

Further Readings

It’s time to build!

We, at &amp, are building various applications — from chatbots to enterprise cloud applications. If you want to work with us or have any questions about how we built ampGPT in more detail, get in touch!

Read On