Scalable SaaS-Based Customer Service Chat

This is a large-scale software-as-a-service (SaaS) application for customer service chats. The application is designed to be efficient and cost-effective and support tens of thousands of users concurrently.

I was responsible for the entire development process of this application, from gathering requirements to writing the code. I understood the needs of the users, designed the architecture of the application, and implemented the code. I also tested the application to ensure that it worked correctly. I'm proud of this app because it showcases my coding skills and ability to build applications.

This is a chat system that allows website visitors to ask questions to human chat agents. Although artificial intelligence is improving, many organizations and visitors alike prefer human answers. Humans are better at understanding complex questions and providing nuanced answers. They are also better at building rapport and trust with customers.

This chat app was made with businesses and organizations in mind that want to provide top-notch customer service. It recognizes the importance of human interaction in resolving customer issues. It can be used in a variety of industries, including e-commerce, corporate, marketing and sales, education, nonprofits, and NGOs. The chat app is also highly optimized for outsourced customer service organizations. These organizations typically have a large customer base and manage chat operations on their behalf. The administrative side of the app is designed to make it easy to add new customers and their websites and domains.

Overall, this app is a comprehensive, user-friendly live chat solution that helps businesses provide superior customer engagement and service. It seamlessly blends technology with the irreplaceable human touch, ensuring that customer issues are resolved quickly and efficiently.

The app is fast, but speed was not the primary focus of the design. The main goal was to keep resource usage low so that pricing for users could be kept as low as possible. This is important because the app is used by a large number of people, and the cost of running the app can be significant. This low resource usage is achieved by using efficient technologies and hosting providers and by giving agents the tools to handle as many chats in the shortest time possible. In the context of this type of chat app, speed is not as important as it may seem. This is because website visitors are often slow typists, and agents are often handling dozens of chats at the same time. This means that there will always be some waiting time, regardless of how fast the app is.

Features

• An Admin app for administration and agents chatting with visitors
• A chat component that is embedded in a webpage
• A chat server that handles the traffic between chat agents and website visitors

Chat agents are grouped by their areas of expertise or subject matter to answer a variety of questions from a diverse customer base. Messages are first placed in a group queue before being assigned to multiple agents who handle the same group. An agent can handle multiple chats at once, depending on their speed, the complexity of the questions and the visitors' response time. A complex scheduling module distributes messages from the group queue to the agent depending on the performance of previous messages.

The chat system is designed to optimize both speed and accuracy, which are essential components of any customer service interaction. It has a smart search feature that scans through a catalog of frequently asked questions (FAQs) based on the visitor's inquiry, and instantly suggests the most relevant responses to the agents, not the visitors. This feature is being further enhanced with artificial intelligence, which will allow the system to comprehend the visitor's question and craft multiple tailored responses, ensuring that customers receive quick and accurate information every time.

The agents also have the ability to expand the list of FAQs, which will continually improve the system's knowledge base. In addition, agents have access to a repository of pre-set phrases, or "snippets," which are linked to keyboard shortcuts. This allows them to communicate more quickly and efficiently. The chat system also promotes collaborative problem-solving. If an agent faces a more complex inquiry, they can quickly consult with seasoned colleagues or supervisors. If necessary, they can transfer the chat to another user, ensuring that each customer's needs are best served by the appropriate expert.

All of these features are designed with efficiency in mind. Faster response times mean that more queries can be handled per hour, and more resolved issues mean happier customers. This reduces the number of customer service agents required, which decreases overhead costs. The system's smart features make customer service not only more streamlined but also more cost-effective for businesses.

The admin app lets you manage the chat app. You can add and assign agents, add and modify frequently asked questions, create chat subject groups, allow internet domains to access the chat app, and set shortcut snippets for agents. Managers can also set and change greetings or status messages that visitors see when they open the chat by clicking on the chat icon. These welcome messages can be automatically changed based on the time of day, if desired. The admin app also gives you the ability to set parameters for domains and websites, both for security and management purposes.

Visitors can share screenshots or other files with agents during a chat. After the chat is over, a rating page will appear where visitors can rate their experience and provide feedback on the chat. All chats will be stored in a database where they can be reviewed by customers and chat managers to improve the quality of service and rate agents on their performance. Additionally, data analysis can be conducted to identify the most frequently asked questions and other issues that visitors have difficulty with. The findings can be used to improve service, web design, and documentation.

Future Features will have audio/video calling, co-browsing (This allows the agent to browse the website with the user to guide them real-time, improving the user experience and resolving issues faster) and a better AI-assisted solution for chat agents. Additionally, CRM integrations or even a light CRM functionality will be available.

Architecture & Technologies

The Chat Server is a service that handles the traffic between chat agents and website visitors. It ensures that messages are delivered to the correct recipients and that the chat experience is smooth and reliable. It is also the service that will use the most resources as it is the heart of the application. It sits between the visitor chat app and the Admin app where the agents work. The chat server, constructed as a Node.js service via WebSockets, operates within a Docker container. This setup not only bolsters its robustness and security, but critically, empowers its scalability. Whether scaling out to accommodate high demand or scaling back when less capacity is needed, the accommodation of fluctuating needs is simplified.

Further accentuating this adaptability is the use of Fly.io a globally represented cloud hosting platform specializing in fully managed Docker containers. Fly.io's strength lies in its global span, featuring servers across all continents. This ensures that the chat server is proximate to the chat agents, yielding optimal performance. Moreover, Fly.io simplifies chat server management with features like automatic load balancing and health checking, allowing for nimble recalibration. Easing management effort while maintaining steady, high-quality performance, Fly.io guarantees reliability and continuous availability of the chat service. The combination of Docker container utilization and partnership with Fly.io inherently promotes scalability.

Provisions of both horizontal (quantity) and vertical (quality) resources adjust automatically to the server's load, preventing over- or under-provisioning. This intelligent use of resources guarantees swift message handling, all the while optimizing cost. Simply put, the chat server exemplifies smart scaling and strategic deployment at its best. Complementing the scalability of the chat server is an efficient communication strategy facilitated through an agile, managed Redis server. By harnessing Redis' publish-subscribe (pub-sub) mechanism, rapid, real-time messaging between Docker instances is achieved. Redis renowned for its speed and efficiency, forms the perfect backbone for managing inter-container communication. The pub-sub messaging paradigm it utilizes enables decoupled communication, making it easy to distribute updates across multiple Docker instances in a secure, organized manner. This collaborative infrastructure greatly enhances the chat server's performance, ensuring that no message delay or loss occurs even when the server scales up or down according to demand. Overall, the blend of Docker, Fly.io and Redis presents a well-rounded solution, delicately balancing scalability, performance, and efficient resource utilization.

The Admin App This SaaS application is designed to support tens of thousands of users, with strict data separation between different customers and privacy policies. The admin app is written in Sveltekit and uses server-side rendering for ultra-fast response times. It is hosted on Cloudflare's edge network, which has over 250 locations worldwide. This ensures that users will always be close to a Cloudflare access point, improving performance. Cloudflare also has a large set of data protection mechanisms and security features. Supabase was chosen for data storage because it is a robust, scalable, and PostgreSQL-based database that supports complex data structures. RLS (Row-Level Security) was a key factor in choosing Supabase over other SQL databases. RLS allows you to control which users have access to which rows in a database, which is useful for SaaS applications that need to restrict access to data based on user roles and organizations. RLS also allows you to make database calls from the client, which can reduce server resource usage and results in significantly lower operating costs. Supabase has a decent authentication module, but it does not have good support for SaaS applications or Role Based Access. Therefore, I created my own authorization system using RLS.

The Chat Component is a piece of code that customers can embed in their websites. It can be added by adding two lines to the header of their site: one for the JavaScript code and one for the CSS with the styling information. A separate CSS file makes it possible to completely adapt the style of the chat component to the customer's website. The JavaScript is built using Svelte, which compiles to very compact but fast JavaScript code. The javascript code is only 200 Kbytes in size. Additionally, this component is housed on the Cloudflare Edge network too.