Skip to content
DE | EN
A Case Study on a private, LLM-powered chatbot

An LLM-Powered Chatbot With A Focus On Privacy

ampGPT is a privacy-first LLM-Chatbot directly integrated into the familiar working tools - Google Chat. It is based on Azure OpenAI GPT, providing powerful functionality while ensuring data privacy. The chatbot has the ability to retrieve information from various files, including internal company resources.

Smartphone with ampGPT chat

Customer

SME

Type

Chatbot

Industry

Productivity Tools

Duration

2 months

Country

Austria

Establishing Clear Goals For ampGPT

1

Ensure Data Protection

Our main goal was to create a tool, that meets our standards for data privacy and keeps our client’s sensitive data secure.

2

Process Company Specific Data

We focused on implementing the features that would make our project a success, including the ability to access data in Google Drive or Confluence. We also identified internal resources that were important to our project and integrated them as well.

3

Increase Productivity

By automating the process of retrieving information from various files and documents, highly skilled workers can focus on their core tasks.

Creating ampGPT

ampGPT was developed through a structured and systematic process that went through three phases - Sketch, Craft, and Ship.

To ensure a streamlined project management process and meet our timeline targets, we conducted detailed planning to ensure completion within a two-month timeframe, without compromising on quality.

We leveraged Jira, our favored project management solution, to craft a comprehensive roadmap that facilitated progress monitoring, task allocation, and seamless collaboration. This approach led to a smooth and efficient development journey.

Detailed timeline of the implementation process of ampGPT
Phase 1

Sketch

Kickoff Workshop

Our craft phase was a critical component of the software development process, aimed at delivering a high-quality and effective solution. During the kickoff workshop, we collaborated with stakeholders to identify their vision, requirements, and desired outcomes. We engaged in in-depth discussions to gain a deep understanding of their needs, preferences, and constraints, to inform our decision-making process.

Animation of a Roadmap

Roadmap

Animation of a Pen

User-Story Mapping

Animation of an Telescope aiming

Business Goals

Animation of a written Document

Jobs-to-be-done

Animation of two People talking

Feature Brainstorming

Animation of a person nodding

User Personas

Architecture Diagram showing how ampGPT works. From the person using it to the actual REST call.

Architecture Diagram

To communicate the architecture of our software to stakeholders and developers more effectively, we aimed to create a clear and concise diagram that encapsulates different levels of abstraction. We chose the well-established C4 model as our approach for designing and documenting our software architecture. Using this model allowed us to create an architecture diagram that accurately captures the various aspects of our software and makes it easier to communicate to others.

Phase 2

Craft

Tech Stack

Our tech stack consists of various components, each playing a crucial role in ensuring the secure and reliable functionality of our solution. We employed Spring Boot's embedded servers and authentication clients for their capabilities, and hosted our server in Azure, leveraging its scalable and dependable infrastructure to meet user demands. Furthermore, through the use of LangChain, we prevent vendor lock-in, giving us the flexibility to keep pace with future advancements in AI and remain agile in a constantly evolving technological landscape.

A animated window of a chat with ampGPT, asking "what are the benefits from being agile?"

Implementation

To ensure that our software solution was reliable and free from defects, we employed a test-driven development (TDD) approach. This involved writing automated tests before writing the code, ensuring that the code met the requirements and passed the tests before being integrated into the system. By using TDD, we were able to catch bugs early in the development process, reducing the time and effort required for testing and debugging.

Agility In Action

For our implementation, we utilized Jira Kanban board, which allowed us to work on multiple aspects of the project simultaneously. The boards enabled our team to visualize and manage the flow of work across different stages, enabling us to address various tasks in parallel. By using Kanban boards in Jira, we were able to track the progress of each task, visualize bottlenecks, and identify areas where we could improve our development process.

We provided the stakeholders with regular updates and feedback to ensure that we were meeting their requirements and expectations. This helped to ensure that we delivered a product that met their needs and provided them with a significant competitive advantage.

Unique App, Unique Challenges

Animation of a rotating Lupe

Navigating The Complexities Of AI Models

With numerous models available, selecting the right one can be a daunting task, made even more challenging by the need to fine-tune various parameters for optimal performance. Achieving this can be a time-intensive process that requires a deep understanding of the underlying algorithms and their intricacies.

Animation of a tightening wrench

Seamless Integration of internal data

Integrating AI models into existing software systems can be a complex and challenging process, requiring a thorough understanding of the underlying architecture to avoid potential compatibility issues. Furthermore, integrating with internal data sources such as files in our Google Drive or Confluence can further increase the complexity of the integration process, requiring careful consideration and expertise to ensure a seamless and successful integration.

Animation of a writing document

Staying Ahead Of The Curve

The AI ecosystem is a rapidly evolving and dynamic space, with new SDKs, libraries, and techniques emerging regularly. Keeping up with the latest developments and determining what to include and what to leave out can prove to be a daunting task. As such, it is crucial to stay up-to-date with the latest advancements and continuously evaluate and adapt our strategies to ensure that we are leveraging the most effective and efficient tools and techniques available.

Animation of a fingerprint scan

Ensuring Client Confidentiality

To address the data privacy concerns of our stakeholders, we utilized Azure's hosted GPT model, which provided an excellent foundation for our chatbot project. Azure's privacy policy ensures that all inputs, outputs, embeddings, and training data are protected from access by other customers, OpenAI, Microsoft, or third-party products or services.

Read more about Azure's privacy policy in their documentation
Phase 3

Ship

Bringing AI To Life

To ensure that users could effectively utilize the chatbot to its fullest potential, we provided comprehensive educational materials and training resources. These materials were designed to help users quickly understand how to use the chatbot and take full advantage of its capabilities. Since the chatbot's release, we have received overwhelmingly positive feedback from employees who have been frequently using it to assist with their daily tasks. This high level of usage has provided us with valuable insights and feedback, allowing us to continuously improve the chatbot's performance and tailor it to better suit the evolving needs of our users. By leveraging this feedback, we are able to provide a more seamless and effective user experience, further increasing the value of the chatbot for our clients.

A screenshot of a chat with ampGPT showing how the file upload works

Impact Unleashed

ampGPT-1

Summary

In just two months, we created ampGPT, which enables all employees to make well-informed decisions more easily, without compromising the safety of the company’s data. This was made possible thanks to the integration of multiple modern technologies.

Animation of a spinning Globe

Impact

ampGPT has improved information management and access by providing a secure solution for retrieving data and answering routine queries. Using natural language processing and machine learning, ampGPT enables employees to quickly and easily retrieve information on demand, without the need for specialised training or technical expertise. The result is a more agile, responsive and efficient workforce that can adapt to changing business needs and emerging opportunities.

About &amp

&amp uses the power of technology to empower people in what they are doing. Depending on your ideas and needs, we develop, shape, and improve software products to increase your digital resilience.

Portrait of Andreas Wittmann
Andreas Wittmann
LinkedIn Icon
Project Lead and Managing Partner at &amp

Ready to transform your business with innovative software solutions?

At &amp, we're passionate about turning your ideas into reality and delivering exceptional software solutions that meet your unique needs.