SamAI

Introduction

I wanted to create a project that would both get me familiar with the latest AI technology coming out and also put the AWS Solutions Architect Professional certificate I recently received to use. I settled on creating an AI powered financial advisor that would make it simple for the user to input their financial information and ask questions with the bot being aware of that input so it can tailor specific advice. Although this is only a side project, I do believe that given only a limited number of people have access to a financial advisor democratizing some of that access to high quality financial advice can go a long way in making solid financial information available to everyone. I am certain in the future companies like Mint or your whichever bank you use will have these bots built into their services but I wanted to see what I could create on my own and as a software engineer test out some of the technology that I'm sure will be finding it's way into my work sooner or later.

Technologies Used

• AI (Open AI's ChatGPT API)
• Angular 2+
• Express.js (with Typescript)
• AWS Serverless (with Terraform for IaC)

My choice of frontend/backend tech here was fairly simple. I wanted to balance what I was comfortable with with what made the most technical sense. I've used Angular 2+ professionally for a number of years so that was a simple choice. Most of my backend development work has been with .NET/C# but I decided to try something new with Express for the purpose of keeping the entire codebase (besides the Terraform) in Typescript. This allowed for quick development, ability to reuse code, and simplified the hosting process.

Going with AWS was another simple choice because of the certificate I mentione receiving above as well as being the cloud provider I'm most comfortable with. The decision to go with serverless technologies was due to both cost and getting the project off the ground as quickly as possible. The most difficult decision I faced was whether to go with Lambda for the backend or host servers/containers on EC2/ECS. I settled on Lambda due to both ease and the fact that I am not anticipating there to be a large number of users on this project. I am aware that Lambda API's can get expensive with a large number of users but I believe for the small number of users I am expecting it will be cheaper to use Lambda than host a traditional server. I also considered the problem of having to deal with Lambda "cold starts" but decided I can live with the potential for queries to take longer than I'd like in some scenarios but I still believe the pros outweigh the cons. Choosing DynamoDB was primarily because I haven't used it before and wanted to get a feel for it while also not wanting the hassle of maintaining an RDS instance. However, after using DynamoDB for this project I still much prefer Postgres to it and would probably go through the trouble of setting up RDS for future projects.

Architecture Overview

AI Integration

The integration of AI is done through Open AI's Chat Completion API. For model I've currently selected gpt-4-0613. I'm going to try and run it on this for now but if it gets too costly I will switch to gpt-3.5-turbo. The model is sent a "system prompt" that tells it how to act. I instruct it with something along the lines of "you are a financial advisor and the user is your client, the client has provided {x} information use that to calibrate your advice", where {x} is the financial information the user has sent formatted into a string in plain english. Everytime a new message is sent the model must be fed the entire history of the chat so it can have "memory" of the previous messages. When a user sends a new message the backend will check DynamoDB for previous messages and if they exist it will add the new message to the top of the list and send the entire thing to the model.

Front-end Development with Angular

I wanted to stick with one component library for a consistent look so I went with Nebular. Normally I use material or ngbootstrap but they both lacked a chat component so I chose nebular and found it very easy to use. I've previously implemented my own chat components (for the Blirb project linked on my main page) and know it can be difficult so decided to go with something prebuilt. After implementing my own auth with that project as well I decided to go with Auth0 for this project just to get it off the ground quickly. It's worked well thus far and I am glad to offload the managing of passwords but I will be keeping an eye on the costs. I do use an auth0 event to hit my "signup" endpoint on the backend once a user has signed up so I can add the user to my own database as well as having them in auth0.

Back-end Development with Express.js

Not a ton to say here, it was very easy to configure Express to use Typescript and the basic pattern of the framework makes it extremely easy to get something up and running quickly. I did have to do some interesting work for converting the financial information form into plain english thats suitable for the model to read. I employed a technique of checking for each input and if it exists forming that into a normal sentence and adding it on to a continuously growing list of assets/info. This all gets incorporated into the system prompt at the beginning instead of being appended to each message to the bot.

CI/CD

The code for this project was all hosted in Github (monorepo) and gets deployed via CodeBuild. For the frontend CodeBuild simply builds the project then updates the S3 bucket it lives in with the new files. For the backend CodeBuild will build the latest code and update the lambda function that code lives in.

Management of infrastructure is all done via Terraform. I used a pattern of having a central AWS accounts and set up SSO for development/production accounts alongside Terraform Workspaces for each. This enabled me to easily scale up different environments for testing and keep separation between the environments.

Future Development

For the time being I'm going to step away from this project and focus on other things, but depending on if people get interested or not I may increase complexity or consider something such as hosting my own model I can more finely tune. The fact that it was fairly simple for someone like me with no formal AI experience to pick up this technology and create a product in a relatively short amount of time convinced me of it's power and that we will be seeing much more of it all over the place in short order. I do believe if a project like mine was to be integrated with something like Mint where all your financial information already lives it could be extremely powerful.