Rewrite of Music Search Web Application

musicaltheatersongs.com is a web application for musicians and users in the musical and theater sector to search for musical songs based on musical-technical terms. The app is mainly used by musical students and teachers. It contains more than 11,000 songs and is used by tens of thousands of users globally. The original app was written more than 6 years ago and used a MVC framework called Derby which became obsolete. It was difficult to maintain the code or to add new functionality. I was asked to rewrite the application using a more modern toolset.

The original application had two functional parts (actually three if you count the Mongodb database as a part of the application) but was built as a single app (except for the database which is a separate service). The two functional parts are the website front-end, the part end users see and use for their searches and store their songs lists. The other part is an admin section where new songs, shows, composers and lyricists are entered and modified. It also manages user’s subscriptions, both individual and institutional (universities, etc).

I split the app into three separate parts that work independently. I created a separate admin front end written in Vuejs 2 combined with the Bootstrap CSS framework and some 3rd party components like a versatile table component. The admin front-end is a Single Page Application that runs on Amazon’s S3 web server combined with AWS CloudFront for world wide cached content delivery and HTTPS access. I wrote Cypress tests to test the front-end after and during modifications. The front-end is fully stateless and does not rely on previous requests. Some of the information that is not too large, is held in memory for as long as the page remains open. Login info and the JSON web token are stored in LocalStorage and will be reused if a user closes or refreshes the page.

Admin Back-end

To access and modify data from the admin front-end, I created an API service that serves data securely and stateless to both the admin front-end and a new website front-end. This API service is built using nodejs with Express.js. The routes are secured by using JSON Web Tokens for every request accessing non-public data. There is also a “development” version of the API to test new features under development. Both the production and development services are packed inside Docker containers.
Inside the containers I use the PM2 process manager to monitor the node js app and immediately restart it if it would ever crash. Both the front-end and API service are completely stateless. There is no reliance on previous requests so a quick restart of the service is not noticed by the users.

Back-end servers need a lot of sensitive information like database passwords, secret keys to external APIs like the Stripe payment API or the external email service, etc. It’s always a bit problematic where to store this sensitive info, it’s very important these secrets stay secret. For this I created a small module that, when the service starts, reads this info from an AWS Parameter Store. In addition to secrets I also added some other bits of key information that is likely to change during the lifetime of the application like subscription prices. Without making changes to the code and a need for redeployment, you can simply and securely change this info though the AWS Admin console and the updated info will be used by the application.

Jobs Service

The app shows some statistical information to its users about their logins and searches. This information needs to be calculated from login and search logs. Because it consumes considerable resources, I decided to implement this collecting of statistics as a separate service so it would not interfere with normal operations. This is also a node js app that runs periodically through the use of a cron plug-in. This service is built in a way that new job types can be easily added. There is a main process that imports jobs as separate javascript files and adds those to a cron queue.

Public Website

Lastly, I created a new front-end for the public website (the one customers use to search for songs). This web app is made with Vue.js 3 and the Quasar Framework. It uses Server Side Rendering (SSR) for SEO reasons. SSR needs a web server (the code for this is generated automatically by the Quasar Framework) so for this, a Docker container is used as well. The new website is built with a Mobile First approach resulting in a much better mobile usability compared to the old website. (Actually, the old website is more or less unusable on a mobile device)

Lambda

Some services like payment processing when subscribing through the website or sending emails to expiring subscribers are implemented with AWS Lambda functions. The reason for using lambdas is to offload processing time from the API server, easier maintainability of those services and to gain experience with full serverless and Lambda functions. Lambda functions communicate their status to the back-end through the AWS Eventbus. Deployment of the Lambda and eventbus functions is done with the new AWS Cloud Development Kit (CDK). This is a toolset for Infrastructure as Code whereby everything is set up and configured through Typescript/Javascript. This makes it possible to store modifications of infrastructure in a code repository and keep different versions of the infrastructure in use.

Deployment

All separate parts of the application have production and development/test versions. Modifications can be made and tested first before adding them to the production app. This brings the total of services to 6 ( 2 api backends, 2 jobs services and 2 websites) all created as Docker containers. I added two more Docker container apps to the deployment. A nginx proxy web server together with a special nginx companion app that automatically creates and updates Let's Encrypt SSL certificates without any manual work. To ease the management and deployment of 8 containers I’m using Docker Compose. All instructions for running the containers, their names, ports, domain names, etc are stored in a .yaml file. When a service needs updating, a new Docker container is built and automatically deployed without interrupting the unmodified, already running containers. To automate deployment, I wrote linux shell scripts that will copy files from my development machine (linux or Mac) to the EC2 instance and remotely execute a docker build command and apply changes with docker-compose. All of this is done through a secure shell (ssh). The application has many users but does not have many transactions. We can run all containers out of a single EC2 instance. If the transaction volume would increase it would be very easy to add another EC2 instance and use a load balancer in front of the multiple EC2 instances. We could also switch to AWS Elastic Container Service or AWS Fargate and run the containers from there. Since everything is already containerized no code changes would be necessary. Instead of using Docker Compose we would switch to Kubernetes.