Malte Ubl delves into the advancements in managing infrastructure through Framework-defined Infrastructure (FdI), a progressive shift from the traditional Infrastructure as Code (IaC).
Malte Ubl serves as the CTO at Vercel. Before his current role, he held the position of Principal Engineer at Google, focusing on Search Rendering, and emerged as the Engineering Director for Search across various devices including laptops, tablets, and desktops. His contributions include pioneering the frontend frameworks for numerous Google Web Applications and broadly influencing the internet ecosystem. Additionally, he founded and leads JSConf EU.
Software continues to revolutionize our society. QCon San Francisco is dedicated to enhancing software development by promoting knowledge exchange and innovation within the developer community. This practitioner-driven event caters to technical team leads, architects, engineering directors, and project managers who play a crucial role in fostering innovation in their teams.
Ubl stated, “My name is Malte, the CTO at Vercel, and I wish to explore the concept of framework-defined infrastructure comprehensively. At Vercel, which is a cloud solution for frontends, we provide teams with the necessary tools and capabilities to develop, preview, and deploy instantly upon inspiration. Picture this scenario: you create a web app, toss it over to us, and we handle the hosting. It’s convenient as it reduces the need for constant monitoring on your part and we excel at it because it’s our responsibility.”
If you are engaged in a contemporary software development environment, such as operating through GitHub and making pull requests, we participate by taking your contributions, deploying them on our platform, and providing a unique preview URL for you to review your modifications. This process occurs every time you perform a Git push. Furthermore, when you’re ready for production deployment, we execute that as well. During these deployments, we not only distribute your application but also perform an analysis of it.
Take a Next.js application as an instance. We delve into its contents before deployment, scrutinizing components like middleware, which are crucial for the application’s performance – this point is essential for this discussion. We also manage a development framework named Next.js, the leading React framework extensively utilized worldwide, celebrated for its widespread adoption and significant download rates among notable web brands.
Discussing framework-defined infrastructure introduces an intriguing perspective from Werner Vogels, Vercel’s CTO. Werner discussed this at possibly the re:Invent conference, emphasizing the transition from primitives to frameworks. This shift marks a crucial evolution in infrastructure deployment, guiding us to prefer frameworks over primitives. Although this approach is generally beneficial, there are instances where it might not be ideal. Reflecting on the historical deployment methods of the 1990s, where methods ranged from SSH access to machines to FTP – a challenging era indeed – illuminates the progress made. From these early practices, we transitioned to semi-automated deployments, utilizing basic bash scripts or advancing to more complex solutions. This evolution naturally led to infrastructure as code, a method where infrastructure is formally defined and deployed automatically. Framing our current topic, framework-defined infrastructure represents an evolution but links intriguingly to a concept known as infrastructure from code. This presentation aims to explore both the connection and distinctions between these concepts.
Why do we care about framework-defined infrastructure? It provides us portability between different target infrastructure providers, so no lock-in. I’ll talk about that. It eliminates the need to manually configure infrastructure to run application production, which is nice. Which also means I have now more time to write product code, instead of doing system management. Really important is, I can use my favorite framework’s local development tools unchanged. I don’t have to figure out how to run my sophisticated server on my local machine.
Because there’s this indirection to how infrastructure is defined, we can actually only use the stuff that the security team ok. That’s great. I used that word infrastructure a lot. Infrastructure is network equipment and setups, web application servers, databases, message queues. This obviously is not a complete list. That’s not what it meant to be. It’s the stuff that you have in a data center, and we have to deploy it. I think what’s slightly more complicated of a question is the question of, what is a framework? The way I think about it, at the core of the answer to that question lies the Hollywood Principle.
The Hollywood Principle is, don’t call us, we’ll call you. That really summarizes how a framework is different from a library. In a library, you call the library. It has a function. It’s a nice function, you call it. In a framework, you go and write code to a pattern that the framework then invokes while it is managing a lifecycle, which is very powerful. That can be imperative. We pass a callback, or, which is really popular today, and certainly in Next.js, what we make, how it works is where you have files that are in certain places. These files export functions, and they get called. There are many ways to do it. The common pattern is that the framework calls you.
The framework controls the lifecycle of the system. That led me to this interesting observation that frameworks are really solving the halting problem. Not in general, obviously not. What they do is they allow us to understand what a program is going to do. I can’t, in general, look at a program and understand what it’s going to do. If I know it complies with the framework, and I understand the framework’s flow and lifecycle, that I can actually understand whether this program will halt. That forms the basis for what then will enable us to do framework-defined infrastructure, FdI. The way it works is that you write code in a framework dialect. What I mean with a dialect is that it might be Go, or it might be C++, it might be TypeScript. You still use TypeScript.
DSL, which stands for Domain-Specific Language, isn’t exactly what I’m referring to when I speak about complying with a framework’s vocabulary. It’s more about the structured communication facilitated by the framework. In our system, we utilize a framework-specific compiler to transform your program into an infrastructure as code definition compatible with Vercel’s platform. We currently support 35 different frameworks, each with its own unique compiler that maps your code to the deployable infrastructure primitives.
Discussing the essentials of modern frontend frameworks is crucial for those who may not be experts in the area. These frameworks, 35 of which we support, share several common features significant to understanding their operation. A key feature common to nearly all modern frameworks is isomorphic rendering. This allows code to be executed both on the server and the client without major alterations depending on the environment.
Another fundamental element is routing. Most frameworks handle requests by mapping them to specific functions or operations, a process integral to how frameworks facilitate the flow of information. This encapsulates the essence of routing within these systems.
Furthermore, modern frameworks typically include features like pages and API routes. Pages are the visual components seen by users, while API routes handle behind-the-scenes interactions with other systems. Middleware often plays a role here, existing within routing infrastructures to modify how routing operations are conducted. Additionally, capabilities such as data fetching, caching, and ACID optimization are commonplace, catering specifically to the needs of frontend development environments where content like images is frequently managed.
Exploring a typical Next.js application structure, we begin with the file pages/index.tsx, which contains a function named getServerSideProps. In Next.js terminology, this means each invocation of the page initiates server functions. Due to this, a serverless function is provisioned to run these computations. Moving to pages/blog.tsx, it utilizes getStaticProps, implying that the page is rendered at build time, resulting in a static file being deployed in production for high efficiency.
Another key file, middleware.ts, operates at the routing phase globally, requiring deployment of a Global edge worker to handle the process efficiently. Additionally, for high-quality image delivery, particularly for larger images, an image optimization service is employed instead of just using a static file service to ensure optimal quality is maintained for images presented on the site.
Understanding these file-service mappings and deployments is crucial for setting up the application’s routing table. This aids the gateway service in managing ingress traffic, ensuring it is routed to the appropriate infrastructure tailored for each specific need.
Diving deeper, we revisit the blog post scenario. The primary element here is not the usual rendering block but the presence of getServerSideProps. This method’s existence necessitates the deployment of a corresponding serverless function tailored to handle these specific needs effectively.
We push it, Vercel deploys a serverless function. Next up, I’m going to go to my text editor and make a change. getServerSideProps, let’s change it to getStaticProps. Again, this is a change of the source code. We saw that in the previous diagram that if we have getStaticProps, we can actually render it at build time and deploy it to production as a static file. Just the code changed, you redeploy. The infrastructure that is needed for this page changes. Vercel will just do that for you. A final change. It’s a little bit more subtle. I’m adding this one line here, revalidate: 60, again we’re going back and we’re going forward, revalidate: 60.
What this means is that every 60 seconds with traffic, this page will be rerendered. Outside of those renderings, we serve a static file. We need to deploy both the static file plus the ability to rerender the page, so we need another serverless function, plus the ability to, in the background, perform the invocation against that serverless function, and then write it to our static file serving system. Again, one line of code and a whole bunch of orchestration all has to happen. That all comes from the FdI compiler understanding what you do. Just to complete the example, finally, we got the middleware.
Again, there’s just this file, this is my project, but it leads to the global provisioning of the edge worker that can actually perform this work. Eventually, you deploy to Vercel, and we give you this deployment summary. Again, you get this every time you push any change to your Git provider. We will then determine the infrastructure that you need and tell you what we deployed. Because you didn’t tell us, you only gave us your program, and then we compiled your program to this infrastructure.
I want to pivot a bit. This is the basic of FdI. This is how it works. I want to now look at various aspects of it. The first one that I think is really key for FdI really differentiates itself from other way of doing things, is that it’s different in terms of vendor lock-in. I want to be very open about this. Vercel, we are, very broadly speaking, a platform as a service company, and they have a reputation of having vendor lock-in. Let’s look at where it actually comes from. I’m going to walk through a few files. This is Terraform. I’m not a fan of Terraform. Clearly my editor doesn’t even have syntax highlighting. I didn’t use it all that much. That’s ok. We can all read Terraform.
Indeed, the focus is on deploying a lambda function using Terraform. On platforms like Vercel, simply uploading your code can deploy serverless functions without specifying whether you’re using lambda or Google Cloud Run. The Terraform script solidly binds you to AWS because it specifically configures for AWS lambda, unavailable on Google Cloud. This relationship with AWS exemplifies vendor lock-in, similar to another vendor whose CLI, named wrangler, must be utilized to manage their deployments. Additionally, this vendor created a custom syntax in a toml file for defining headers.
The specificity in code to match vendor specifics leads to vendor lock-in, contrasting with platforms like Vercel and FdI (Framework Development Interface), which do not require specific vendor coding. An example can be drawn from a large customer with a vast codebase. It showed minimal Vercel-specific configurations, primarily a Git ignore entry for a build artifact. Despite this, their highly lucrative applications, generating revenues over a billion dollars, are deployed seamlessly without any bespoke configurations for Vercel. The process is automatic, adhering only to the chosen framework’s conventions.
Local development environments are notably critical. Deploying environments with complex cloud configurations often necessitates arrangements like Docker Compose. However, managing local setups for services like lambda can be quite challenging. Conversely, utilizing a framework usually provides straightforward instructions for setting up a local development environment. Frameworks are typically designed to enhance the local development experience. Unlike Vercel, which lacks a dedicated local development environment, relying instead on the existing support of whatever framework is in use. Simply follow the startup commands listed on the Getting Started guide of your framework to enjoy a first-rate local development experience.
The concept of immutable deployment is critical to understand. In Vercel, each push triggers a unique deployment tied to a specific git commit. Imagine making a git commit labeled ‘abc 123’ and deploying it; Vercel provides a dedicated virtual infrastructure for that specific commit. Consider a scenario where I update from getServerSideProps to getStaticProps—such a revision alters the required infrastructure. This change, when committed and deployed, prompts a new setup without altering the existing one.
Instead of modifying extant infrastructure with each code commit, Vercel assigns a distinct virtual setup for each commit. Using content addressing, the platform minimizes redundancy while ensuring each deployment is unique. This approach supports a standout feature: instant rollback. In a production environment, the platform retains the framework of previous deployments, allowing for quick reversion to any prior state if a bug is detected, regardless of how far back it is.
Without an explicit policy for deduping deployments, Vercel employs what is known as instant rollback. This system doesn’t involve complex processes; it simply involves switching a pointer. When www.mydomain.com is set to a different deployment, a global signal is sent to the serving stack to switch to the new deployment, which is a relatively simple action compared to traditional methods.
A notable additional advantage of this system is what’s termed skew protection. This feature allows clients to ensure the server operates on the same software version across deployments, guarding against discrepancies in server-client communication due to updates. This capability of handling effectively infinite simultaneous deployments is enabled by the distinct and immutable nature of each deployment.
This is an interesting topic. I think what it does bring up is whether FdI depends on serverless, because I called it virtual infrastructure, that really only works if you have some notion of serverless infrastructure. Because if you actually have a computer, and you need it for every deployment, and every git commit makes a deployment, then you really quickly run out of computers. You need some notion of something that’s serverless, and in particular, something that scales to zero, so that if you have in theory that old deployment, but it gets no traffic, it doesn’t take more resources than whatever the storage cost that actually has the data, which shouldn’t be all that much. Yes, we need serverless, but if you’ve really determined you could probably make it work without.
Next question, does it work for stateful workloads? What I mean with that would be like a database. It’s definitely much harder, but it’s not impossible. What it really means is that you need some way of provisioning infra lazily when usage is discovered. Say, to connect to a database, ok, then we make one. You need some way of isolating changes to their branch. In Rails, there were these database migration scripts, which are best practices nowadays.
If they operate on your core database, then you don’t want to run them on a branch or for every commit. Obviously, most databases don’t have this. You have an instance here, an instance there, but I think what we are seeing is that there’s a trend of even stateful workloads coming towards Fi. What, for example, we are seeing is a provider like *Neon, which is a Postgres provider, and they support copy-on-write branching of your database. They can actually give you a copy at very little cost, every time you deploy, and really bring you closer to that FdI vision, even for a stateful workload.
Having said that, there’s definitely tradeoffs that I want to acknowledge. Especially if you’re a DevOps person, it can feel a little bit off that you don’t really know what’s going on in production, because it comes from the code so you have to deploy to find out. Again, we tell you when we deploy, exactly what we did, but there’s not this Terraform file we can just look, this is what’s going to happen. There’s definitely this indirection. As I was mentioning, especially with stateful workloads, this can be a bit of a leaky abstraction.
If you possess only a single solution, it may not cover the wide range of deployments necessary. Within your observability strategy, it’s crucial to incorporate the transient nature of resources. We can establish an observability hierarchy top-down as needed information emerges from the bottom up. This strategy is surely manageable, but it’s important to recognize these dynamics. Reflecting on the evolution from traditional infrastructure-as-code, which is generally a declarative approach defining the desired infrastructure, to a more evolved method known as infrastructure from code. Here, you use code that not only handles infrastructure but also incorporates business logic through specific SDKs tightly linked to the infrastructure. This approach is closely associated with FdI, focusing on business logic without the abstraction layer typically seen in infrastructure thinking, relying instead directly on the frameworks wherein compilers translate framework logic into functional components. This clearly delineates a progression and relationship between these methodologies.
Exploring beyond specific examples like Vercel reveals fascinating innovations such as Google Service Weaver, which pioneers in deploying serverless services within a microservices architecture. Another intriguing case is Deno KV, offering a key-value store which operates consistently both locally and in production environments, thus smoothing out the deployment process. Additional noteworthy frameworks include Encore.dev and Nitric framework, which bridge the gap in the infrastructure from code continuum. These examples emphasize the importance of exploring various frameworks.
In conclusion, the discussion advocates for a deeper engagement with frameworks rather than solely focusing on programming foundational elements directly. The aim is to enhance portability across various infrastructure platforms, reduce the reliance on manual infrastructure configurations for production environments, and allot more time to developing the product rather than managing systems. The objective is to harness the full potential of the framework used in local development.
Finally, as a security team, we want to know what’s going on in production. We love it, that there’s a system that just only allows folks to do what actually the system does, and doesn’t have an easy way of bypassing it, and folks deploying whatever they want to production.
See more presentations with transcripts
Aug 30, 2024
Welcome to DediRock, your trusted partner in high-performance hosting solutions. At DediRock, we specialize in providing dedicated servers, VPS hosting, and cloud services tailored to meet the unique needs of businesses and individuals alike. Our mission is to deliver reliable, scalable, and secure hosting solutions that empower our clients to achieve their digital goals. With a commitment to exceptional customer support, cutting-edge technology, and robust infrastructure, DediRock stands out as a leader in the hosting industry. Join us and experience the difference that dedicated service and unwavering reliability can make for your online presence. Launch our website.