The NASTy Stack
The Origin Story
In my mind, I am a frugal person. Some would say "cheapskate", and maybe that's true sometimes. I have no problem putting money where true value can be found, but it causes me great stress to consider that I could be wasting money on a product or service that has a more cost-effective or valuable alternative.
This bleeds over into the technology world for me. Many years ago when I was a middle-schooler/high-schooler with no job or money, I wrote PHP code and deployed it to $3/mo "unlimited!!" shared hosts that my parents so kindly paid for, and that was that. I didn't worry about scalability. Deployments didn't need to be automated, because I just had to drop files in an FTP (in retrospect, I had so much more patience as a kid...). When I started to take root in the ASP.NET world, I ran into some big problems: shared Windows hosts were more expensive, more painful to use, and deployments were annoying (what do you mean I have to cycle an app pool? What the hell is IIS - shouldn't my app just "go"?)
I loved writing .NET code, but I hated trying to get it out the door. At my full-time jobs, I didn't really have to worry about this too much. Usually our company or clients had servers already up and running and I just delivered the DLLs, with the occasional IIS configuration. At home, it was a whole different story. I didn't want to pay the Windows host premium (I still don't). Instead, I learned PHP Yii to get some semblance of MVC architecture. I didn't like it much (I later found Laravel and was actually pretty fond of it). Then I learned Python Django thanks to Two Scoops of Django, and I didn't really love that either. Then I tried Ruby on Rails and hated it (sorry). For all of these, I was publishing to Heroku for free, knowing that with any real amount of use I'd be paying ay least $16 a month for a single app and a database - I used to only pay $3 a month, this was just unacceptable. All I wanted was to have fun writing code in a style I liked, deploy it in the easiest way possible, and be pretty hands-off maintenance-wise. Also it should scale amazingly well. Also I want to pay almost nothing for it. Is this too much to ask!?
Yes it is. But it is completely within my grasp now.
Introducing, the NASTy Stack:
- NestJS for API development: A progressive Node.js framework for building efficient, reliable and scalable server-side applications.
- Angular: An application design framework and development platform for creating efficient and sophisticated single-page apps.
- Serverless: A cloud computing execution model in which the cloud provider runs the server, and dynamically manages the allocation of machine resources.
This architecture hinges on the idea of using a JamStack approach, sans server-side rendering. For me, this means an Angular app hitting a REST API for all its data. This works so great for applications because it separates your concerns very cleanly and is very manageable. However, if SEO is a huge concern (or you're building a straight-up website and not an application), you may have issues with this approach because the data is fetched on-the-fly and not pre-rendered.
This architecture works great for startups looking to be financially efficient as well as enterprise companies looking to scale efficiently. There are cases where NASTy stack doesn't do the job - see the final notes of this post for more details.
- Quick - the CLI scaffolding for new projects and new components (and test beds) lets me move really quickly without tedious boilerplate.
- Standardized - no more guessing games as to "what folder does this go in?" or "what should I call this type of class?" because NestJS is opinionated and makes these largely unimportant decisions for you. Your code will look largely the same as your teammate's code because you have enforced guidelines at the framework level.
- Easy to test - The dependency injection paradigm coupled with the scaffolding for test beds makes it super easy to get started writing tests. It is a bit of a shift in thinking as it somewhat merges the idea of mocking and occasional monkey-patching, but once you get your grips on it, it's a great testing setup.
- Easy to run and deploy - I'll discuss this more later in the Serverless section, but Nest is easy to run in the serverless cloud, on a PaaS like Heroku, or on your own servers. It is no more complex than serving an Express app, but you get all of the enterprise development features of a language and framework like C# and .NET.
Learn more about my thoughts on NestJS and Typescript in this video, "Why I'm Focused on TypeScript and NestJS".
Angular is my favorite front-end framework as of now and it fits beautifully into this stack next to NestJS - which makes sense, because NestJS design paradigms were shaped from Angular's lead.
While many argue against Angular these days, favoring instead for React and Vue, I find that Angular is a very, very strong choice when building applications - especially when the application is mid-sized to large and especially when you're working with teams. See more about my opinion on Angular in this video, "Why I Choose Angular over React and Vue".
There is also something very important to be said for ubiquity between your front-end and back-end. It is something I have chased after for years and I was so excited to discover this combination back in 2017.
Part 1: APIs - AWS API Gateway and Lambda
My first real introduction to serverless API deployments with AWS (API Gateway and Lambda) when I joined Lumeris. I had dabbled with it a bit before, but I didn't get to dive too deep. Working with an amazing team I learned the ins-and-outs of serverless API design and management.
AWS API Gateway is essentially a proxy that enables a web request to perform functionality in the cloud. This is the entry point from the HTTP request to calling our code hosted via Lambda.
AWS Lambda is a serverless compute engine that allows developers to deploy code without having to manage server concerns; in effect, Lambda spins up a temporary server very quickly to handle all code executions within a time frame, and then spins the server back down. This paradigm enables you to only pay for the resources and amount of time your code requires to run, often saving tons of money. There are cases where this isn't appropriate, and you'll find details on that at the bottom of this page.
Start with CloudFormation. CloudFormation is critical for a couple reasons. First, it is a source of truth for every piece of infrastructure your API needs - this is helpful not just for yourself, but for those maintaining your app after you. Secondly, it makes deploying as easy as a couple shell commands. And lastly, you now have a completely ready-to-deploy infrastructure at your fingertips if you ever need to tear things down or move accounts. This is a bit of an uphill battle to learn; I won't pretend otherwise. Fortunately, AWS documentation is pretty solid, and you'll find some good templates on GitHub/Gists that should guide you.
Use the serverless libraries for your framework. For Node & Express (Nest too!), this is aws-serverless-express. For .NET, this is Amazon.Lambda.AspNetCoreServer. I've seen projects where people re-invent the wheel here to route the requests to code - you don't need to do this. Write your API "as usual" and use this proxy layer to do the legwork for you.
Favor Node or Python over .NET or Java. Yes .NET and Java will work. However, the cold-start times are pretty brutal and the memory usage is still notably higher than Node or Python.
API Gateway and Lambda cost as much as they are used. You pay per invocation, per memory, per ellapsed time. If you're doing basic API data management work, this is a home-run solution and costs very close to $0, even with a large number of requests. If you're doing more complicated things like video encoding or large data processing, this is probably not your best bet.
Part 2: UIs - AWS S3 and CloudFront
This part is a breeze. Drop files in an S3 bucket. Attach it to a CloudFront CDN. Put a Route 53 domain alias in front of it for a clean URL. Boom - super cheap, super simple. Honestly, I don't have a lot to say here because it is so simple. I would say the only thing to watch out for is bucket permissions - you don't want anyone able to drop files in your bucket!
AWS S3 is just what the name implies - Simple Storage Service. It hosts files (very affordably!) and enables you to use them as a private dropzone, as asset hosting, or like in our case, static website hosting.
AWS CloudFront is a CDN service that handles caching and distribution, and we leverage this for our S3 bucket to make it more available and simplify security. While you can run your site purely through S3, I would recommend reading this article to undernastyd why using CloudFront may be a better option.
Similar API Gateway and Lambda, you pay by usage and by amount of storage your app takes. This ends up, again, being very, very cheap. It can get up in price on the CloudFront side, but it only becomes as pricey as your traffic increases - which hopefully means you can afford (still ridiculously low) cost.
Part 3: Databases - Mongo Atlas or Heroku Postgres
Yeah, I didn't say AWS DocumentDB or AWS RDS. For a couple reasons: pricing and (in the case of Mongo) lacking features.
Mongo Atlas: Mongo has their own cloud services offering, and Atlas is their Mongo server service. I go with Mongo Atlas over AWS DocumentDB because Mongo Atlas has a life-long free tier with reasonable limits and simple pricing scalability. DocumentDB doesn't particularly have that, and more damning is that DocumentDB is fairly out of date from the Mongo query API which becomes very frustrating, especially when doing more complicated aggregations.
Heroku Postgres: If RDBMS is more appropriate for your architecture, Heroku offers a $0 database with the restriction that you can only have 10,000 rows. Again, this only will cost you when you get past 10,000 rows - hopefully by then you've been able to monetize enough to afford the next tier at $9 for 100,000 rows. This is significantly cheaper than an RDS innastyce (though you do get a generous free tier for a year), and when your app is ready, you can always migrate over.
Both Mongo Atlas and Heroku Postgres start at $0. Next tier is ~$9, and it goes up from there. These are the most reasonable prices I've found for any strong data stores, and the reliability from both have been great. Databases are always going to be pricey, but both options provide a $0 starting point and will give you enough resources to get your product built, released, and monetized. Not only is the cost incredibly low, but the reliability is through the roof - no more worrying about server up-times or maintenance.
A parting note on cloud providers
I talk about AWS a lot here because that is my cloud platform of choice. Azure/GCP offers competitive products, and while I can't speak to their cost modeling or pleasantness, I've heard great things. If Azure or GCP are your preferred vendor, you can pretty much map all of the services I've described to their platform and see similarly great results.
I love TypeScript; I started using it early on in v1 with AngularJS, and I loved the language. Admittedly, getting a build system and working types was less enjoyable back then. Now, we have a wonderful ecosystem in the
@types NPM organization, and when you are working in Angular and Nest you are very well supported by third-party modules that natively support TypeScript.
I talk about TypeScript more in this video, "Why I'm Focused on TypeScript and NestJS".
When the NASTy Stack architecture fails
This stack relies heavily on the serverless metholodogy, and for good reason: when used properly, it is incredibly cost-effective. However, as Spider-Man fans know, "with great power comes great responsibility". There are pitfalls where serverless Lambda functions are NOT your friend. Particularly when:
- You are doing process-intensive work, such as media encoding.
- You are serving a large number of requests consistently and connastytly. This can still be effective on Lambda, but if these requests are longer than a few hundred milliseconds or you're doing many thousands of requests, you may want to reconsider.
We now have an enterprise-ready UI and API that costs pennies and scales (very politely) with app usage. The front-end and back-end code bases are both very similar in design, but not forcibly intertwined. Your team works in one (beautiful!) language. Your developers can be full-stack without a mental shift that requires going from a functional mindset like in React to an object-oriented programming mindset like in Java/Spring or C#/.NET. There is tremendous value in this programming environment, and coupling it with the ease of serverless cloud hosting, delivering scalable and maintainable applications has never been easier.