Episode #112: Abstracting Stateful Serverless with Jonas Bonér

September 27, 2021 • 57 minutes

On this episode, Jeremy and Rebecca chat with Jonas Bonér about stateful serverless with Akka Serverless, the use cases that stateless serverless opens up, why reactive principles are important for distributed applications, and what future abstractions will mean for infrastructure.

Jonas Bonér is founder and CEO of Lightbend, creator of the Akka project, initiator and co-author of the Reactive Manifesto and the Reactive Principles, and a Java Champion.


Website: http://jonasboner.com

Twitter: https://twitter.com/jboner

LinkedIn: https://www.linkedin.com/in/jonasboner/

Akka Serverless: https://www.lightbend.com/akka-serverless

Akka: https://akka.io/

Reactive Manifesto: https://www.reactivemanifesto.org/

Reactive Principles: https://principles.reactive.foundation/


Jeremy: Hi everyone. I'm Jeremy Daly.

Rebecca: And I'm Rebecca Marshburn.

Jeremy: And this is Serverless Chats. Hey Rebecca, how are you doing?

Rebecca: Hey, I am really good. I'm just going to jump in without you asking me, my weekend was great, thank you for asking.

Jeremy: You already know what I'm going to ask you.

Rebecca: A final wedding and then I went camping in Yosemite with my dad during the fall equinox. And what a stunning, stunning place. We did some backcountry hiking, slept in the woods for a number of days, I got really smelly. And all in all, that just means it was a really great trip.

Jeremy: It sounds like it. Well, speaking about-

Rebecca: How about you?

Jeremy: Well, I was going to say speaking about smelly, I just started up my bowling league again.

Rebecca: That'll do it.

Jeremy: Yeah. And now, for people who are wondering, it's like a typical bowling league, although it's candlepin bowling, so it's very small balls and you get three, people know the rules of actual bowling, this is a very small subset that's isolated to the Northeast and parts of Canada, I don't know why, it's weird. But anyways, so I've been doing that, obviously continuing to do my soccer and all that other stuff. And then, of course, always in my perpetual quest to try to find the perfect serverless abstraction.

Rebecca: Well, speaking of serverless abstractions, we have the perfect guest for that, and I'm wondering if you would want to introduce him.

Jeremy: I would love to. So our guest today is the CEO and founder at Lightbend. He's the creator of the Akka project, initiator and co-author of the Reactive Manifesto and reactive principles, and the Java champion Jonas Bonér is with us. Hey, Jonas, thanks for joining us.

Jonas: Thanks a lot. I'm happy to be here.

Jeremy: So let's start with some questions, first of all, about what is Lightbend, what does it do? What is the Akka project? And then let's let's get a quick overview of Akka Serverless, and then we can jump into some of the details.

Jonas: Sure, that sounds good. So starting with Lightbend then I suppose. Yeah, I founded the company about 10 years back around the opensource project, Akka. And back then, it actually started as a collaboration between me and Martin Odersky, the creator of the Scala programming language. And Akka was a product, we can talk more about that later, but it was built in Scala, and it had Scala and Java APIs, it's sort of distributed middleware platform on the JVM.

Jonas: And Scala, at the time, was pushing a lot hard for concurrent programming, and it still, of course is great, but they were doing a lot of work there, so it was natural to join forces in trying to sort of create the next platform on the JVM for tackling what we now call cloud essentially. And it was early days, cloud native wasn't really a thing, but we were really trying to build the best platform for multicore and multinode, so to speak, distributed systems. It's been 10 years, we've been going through a lot of chapters along the way, so to speak, and adding capabilities to it and the latest one is Akka Serverless, so that we can talk more about later as well, so.

Jeremy: Yeah, so with Akka Serverless, so one of the things, again, there's been some good buzz about it, very interesting. I'm super interested in server abstractions, I really am like trying to find a way that we get more developers to build with serverless. So Akka Serverless talks a lot about stateless... I'm sorry, stateful serverless with this idea of getting rid of the database completely abstracting that away. So can you explain a little bit more what stateful serverless means?

Jonas: Sure. We've been going through a lot of incredible innovation, I think in the serverless ecosystem in general the last two, three years, and it's adding more capabilities in almost every aspect of things, all the way from infrastructure things like Knative and KEDA doing really cool stuff, and developer tools, including operational tools like observability and all the things. And we start to see more and more products being able to be consumed in a serverless way in a way. So I think it's clear that serverless is really revolutionary. I actually see serverless personally as the future of the cloud, it is really the way we should build systems.

Jonas: As developers, the infrastructure's so good, we have all this cloud vendors now doing amazing stuff, we shouldn't really have to care about the nitty gritty details of operating all these servers and et cetera. So I really think that it's really the future. That said, a couple of years ago, like two, three years ago, when I started getting super excited about serverless, I spent a lot of time talking to our clients, our existing Akka platform customers. And it was clear that a lot of things held them back for being able to move all of their bigger enterprise systems to the serverless experience. It was clear that stateless serverless function as a service was, and still is. I mean, it's a really, really good tool for a specific set of use cases, like a subset of everything you want to do.

Jonas: And you can argue about how big, for some apps, it might be everything you need to do, for some it's a smaller piece. But it is clear that there was something missing in order to build essentially any app, a general purpose type of applications, enterprise applications, and et cetera. And when we started thinking more about it, it was clear that most of the things, not all of it, but a big piece, missing piece, that was how to tackle state, distributed state, state at scales, available state, et cetera. And the reason why we decided to try to tackle that challenge is that that's what we've been doing the last 10 years in Akka, and we have a lot of experience working with our Fortune 500 customers on trying to solve this problem.

Jonas: So we were just thinking, is it possible to take this distributed system patterns [inaudible 00:06:39], so to speak, how to manage state, and sort of data in general, data in motion, data at rest, both these aspects of state and sort of wrap that up or dress that up sort of in the serverless dress up. And that sort of what led to like stateful serverless. It's not really a great name because it sounds like it's replacing stateless serverless, but that's not at all the idea.

Jonas: The idea is just to add more tools to the toolbox. Sometimes you need stateless, which Akka serverless can absolutely do, and many other tools can do it. But our contribution is adding more tools to that toolbox around data, and data availability, state availability, and how to scale that, and making sure that it's always consistent, it's always correct, and you don't have to worry much about it. So I guess that's what we're sort of trying to bring to the picture, and many, others as well state, stateful serverless is starting to become more of a thing now than two, three years ago when we started thinking about these things.

Rebecca: Yeah. And then... Oh, go ahead. I didn't mean to cut you off.

Jonas: No. I was just going to say we can talk more about these things in depth if you like, but this is sort of the overview.

Rebecca: Well, that's about what we're going to do, we're going to talk more about it. Earlier in an article, I think just a couple months ago actually, you called it a quantum leap of value, when you get to add stateful serverless to preexisting tools that do a lot of stateless serverless. So I'm wondering, you talk about these challenges, you say the next quantum leap of value generated by the cloud will come when serverless is able to address the full gamut of applications required to run a business.

Rebecca: And so, I'm wondering with those companies that you've worked with, you said you've worked with a lot of Fortune 500 companies and really people all over the map, where and how do you see them experiencing some of those challenges today? And then what did those challenges that they had, how did it inform how you built Akka Serverless?

Jonas: Yes. That's a great question. I don't know if we should first talk about what Akka is and why people gravitate towards that, doesn't-

Rebecca: Probably. We should probably add that context.

Jonas: And what Akka is really trying to do, is to through building a platform on top of what I think is one of the best abstractions, actually the best abstraction for distributed systems, the actor model. The actor model is a model since the 70s. It's been used very successfully in many different more specific situations. Like Erlang, for example, is like this obscure language that Ericsson have been using to build extremely reliable telecom switches, for example.

Jonas: And it's starting to creep up, now you have [inaudible 00:09:30] started to become more, and their web frameworks on top of that. So it started to become a little bit more mainstream, a little bit here and there, but when I started Akka, the actor model wasn't really that much known, it was like not 2009, so it was about 12 years back.

Jonas: But that the best foundation I could have for building a platform for distributed systems as well as concurrent systems. Because the way I view it, they are essentially two sides of the same coin. They have the same challenge if you want to scale out to multiple cores, as well as multiple nodes, as well as multiple data centers. So all the way from, in size, distributed systems in a way, if I may get a little bit geeky, within the actual server. Because if you have multiple cores, you essentially have a distributed system within a server, all the way up to the larger scale, the distributed, those are like multiple data centers. And having a single abstraction, the actors, and message passing, and sort of that encapsulate state and behavior, and I'll get back to why I think that matters in serverless later.

Jonas: But having that sort of abstraction and working with message passing as a first class thing including having a really good way of managing failures, that is very honest instead of trying to hide things like many distributor systems like EJBs and CORBA, many try to sort of hide things. But Akka and the actor model really tries to make it explicit and making it very honest, in a way, to the challenges that we have when we embark on building distributed systems. So the only complexity you have in a way is like the actual essential complexity and no accidental complexity, if that makes sense.

Jonas: So we built this platform around this premises folded in streaming, quite soon afterwards to tackle like fast data type of problems. And it was clear that a lot of the customers that went sort of gravitated to us were like, "This is really large Fortune 500 companies that have systems that can never, ever stop, that had immense levels of scalability." And which was not honestly, that common back in 2011, 2012 when we started, but with the emerging of the cloud, it has really become extremely common.

Jonas: Now, when you deploy apps to the cloud, you actually don't know if it's going to be quite rapidly hit with hundreds of thousands of requests if you're successful and you have the ability of scaling out. But 10 years back, having this farm of servers on premise was quite expensive and quite... it's like more for the bigger systems or bigger type, bigger customers I would say.

Jonas: So what we really try to do is, since it's like the needs have become more and more common, to actually stay always available, never, ever crash, to gracefully scale out and up and down for efficiency reasons, et cetera. It's been clear that the platform we built for Akka was actually not just for the bigger guys like Starbucks, and HP, and some of the other systems, ING, Capital One, some of our larger customers. But it's actually started to become something that everyone values and it's important for everyone when they deploy to the cloud.

Jonas: And it's been also clear that that even our Akka platform is now actually deployed mainly to the cloud, we have 75% of our customers actually building things for the cloud, but they're still struggling honestly, because they're struggling with the infrastructure of the cloud, and they're struggling with pulling all the pieces together, and operating it, and all of these things. Because stitching it together, even though you have great tools, and I really believe that Akka platform is a great platform to doing that, it's still challenging. And that's the reason why we sort of try to move all of this great stuff, all the scalability, and availability, and performance, extremely low latency to the cloud, in the serverless dress up, to make it available for everyone, more or less, in a sort of pay as you go way.

Jeremy: I actually love how you said sort of this honesty of distributed... or that distributed systems need to be honest. So you get all these companies that start moving to distributed systems, and they have to accept this change from reliability to resiliency, where they now have to just embrace failure as one of those things that's going to happen. And I love that term just keeping it honest, don't hide things. If a system's down, react to it, find a way to have a resilient way to deal with it, but don't just try to keep everything up 100% because that is never going to happen in a distributed system.

Jonas: Exactly, and where I think that pays off most, is around how you deal with failure. Because if you've optimized for your happy path, so to speak, where everything is working well, it's like beautiful abstraction, everything just works. But as soon as things start to fall apart, if your abstraction doesn't expose failure, it doesn't have a way to working with failure. I mean, it's almost game over. But instead of if you're sort of... I'm sometimes called about it, and talked about it as like embracing the network, and all the sort of intricacies, and all the shortcomings of the network. If you bring that into the program and model, sure, it becomes a little bit more complex, it does, but its essential complexity, because having that under your fingertips means that when things go south, you have control over it and you can actually manage it, and sort of manage failure as part of the programming model. So that has served us really well, and for us that has served our customers extremely well, so.

Jeremy: So you talked a lot about scalability or being able to scale out gracefully. And one of the ways that you do that with Akka Serverless is by deploying on top of Kubernetes, right? So could you explain a little bit more about what is the scaling characteristics of this and how is that sort of managed for the user?

Jonas: Yeah. We have sort of an unusual way of dealing with scalability. Since we actually trying to manage distributed state and bringing in state, and we can talk about why that matters later. I don't know if we can talk about why, actually, why it matters first-

Jeremy: We can talk about it right now.

Jonas: Before we taught about-

Rebecca: I think the why is pretty meaty?

Jeremy: Yeah. The why is important.

Rebecca: Yeah. Let's go with the why.

Jonas: [crosstalk 00:16:44] I was just thinking. I think that the reason why I think it matters is that everyone knows that there's a ton of value in data today, most of the value is actually in data. And a lot around is actually getting intelligence out of data quickly. And when applications are deployed onto the cloud and you have thousands of extremely impatient users, users usually expects to get intelligent answers back quickly, that means that the systems need to be able to process data, almost like 'flies by'.

Jonas: I mean, sort of just grab it on the fly, do intelligent things with it, sometimes merging, transforming, and enhancing data streams, it can be... But regardless of how you do it, to mine value out of data in real time. And that's really, really hard to do if you always need to put your data somewhere else. If you're always have to need to like... if you sort of change it, you always need to store it down in the database somewhere else, often with the network in-between, sometimes there might be even in another data center, but there's a lot of latency over just putting data somewhere else. And when you then need it, you need to go and fetch it or pull it back into context where it's needed.

Jonas: So here, data locality is extremely important. If your data is always right where it needs to be, whenever you need it, wherever you happen to be, then things are good because you can just use it, and there's actually zero latency then. That's not always the case, there might be some loading that needs to be done and so on, of course, and they might be sort of migration of components, and then there might have to be rehydrated and things like that. But in most of the cases, if you have an efficient and good platform that understands state, it can actually making sure that your data is always where it needs to be, whenever you need it, even before you need it so to speak. It's always right in process, it's always right there next to the processing.

Jonas: And this is really what we've tried to do, and I think this is one of the key things about stateful serverless, bringing state into the application, into entities. We talk about entities, instead of just having sort of snippets of codes, so to speak. It's a really nice thing, I like the way people talk about it in serverless because it points toward simplicity, but sometimes where there's too much simplicity because snippets of code doesn't really have semantics around state. It doesn't have any semantics around, what does your data model look like. And here, I think it's often met, sometimes at least, I shouldn't be too arrogant, but sometimes, it's useful to think about it in terms of entities. I mean, entities mean having data and data models, some sort of scheme or an understandability around what data means, what semantics... together paired with the processing.

Jonas: And if one of these entities need to be migrated, it does it as a whole, state and processing moves around with it. And this can give you a really good foundation for extreme low latency, especially if you start thinking about moving things out to the edge. And I'm not talking about actual edge here, the actual devices, that's the ultimate edge, but just sort of start leveraging many, many points of presences, having sort of edge clusters in a way, and pushing things out to where the users are, then is extremely useful to have the state come along with the processing and being able to do everything that's needed almost out there.

Jonas: Sometimes there might be some fancy query or such fancy computation that you don't have all your data for, and then you fall back to the more central cloud, or you might bring in state. But let's say, if you can do 80% of all the processing with state you have right there, then you can serve a lot of very new, cool use cases, like consumer and industrial IOT for example, and real time financial services, factory automation, telemedicine, and like internet based gaming, a lot of these cool things that have really low latency guarantees can then be sort of used in the serverless dress up.

Jonas: So this is how I'm thinking about the problem and what we're trying to bring to the picture. Now you sort of asked how we do scaling and things like that. And here is where things become a little bit different or a little bit more complex, because if all I have is stateless services, then it's very easy to scale. In two seconds, it'll bring up 1000 instances, having a load balance just redirects everything to them, and then into all these new instances, have them run for a while, and it can sort of just kill them all. And it's just fine because state is somewhere else. So that's a benefit of stateless scaling, scaling is almost free in a way.

Jonas: When you have state, it's not that easy because first, it's more complex in how you manage it, but it's also a lot more expensive to scale out stateful components because then you need to of replicate the state, and state goes along with the processing, as I said. So we can't, for example, use vanilla Knative autoscaler because it's simply too aggressive for stateful needs, it's not really meant to be used with stateful components like this.

Jonas: And we actually tried for a while, and we worked with the Knative team. They're extremely open, but everyone's busy on what they're... so it wasn't the top of their priority back in the day, so we actually backed out and implemented our own autoscaler that understands state is less aggressive and tried to be more smart. But the heavy lifting is actually not done by that autoscaler that just spins up instances but it's actually done by Akka and the module in Akka called Akka Cluster, where we actually... Are you familiar with DynamoDB for example?

Jeremy: Absolutely.

Jonas: And Riak and Cassandra, they all use this model of a node ring. So they put out all their servers in a sort of node ring, and they have intelligent replication in how things can be replicated across the cluster. We use exactly those algorithms in Akka Cluster. So we sort of lay out all your servers in an Akka Cluster node ring so to speak, and we use in intelligent replication like shards across. So we actually shard your whole dataset, and of course we make sure that shards normally spans multiple nodes, so you have fall back. It can span three, five, seven nodes, but sort of sharded out across the whole cluster. And that means that, when you create an entity, then it's actually placed on the shard where the data lives including, will be around there in terms of processing.

Jonas: And let's say that you resize the cluster because there's a lot more load here, then the autoscaler will make sure that this whole node ring is sort of scaled up, and then there will be more nodes and available power to be used, and then we're sort starting to reallocate these shards across this node ring until you have some sort of even distribution. And all this is happening under the hood.

Jonas: Same thing, of course, it's equally important if you want to scale down, if you don't have that much load, then the cluster will detect that and automatically resize itself so you can consider all resources, money and energy and all of these things. So it is a sort of dual approach, we have a cluster doing all this cluster sharding and intelligent placing of functions, physical placing so you get collocation and then we have a little bit more intelligent or I won't say more intelligent, but aware of state, load balance, or that works in concert with that, so [crosstalk 00:25:36].

Jeremy: Yeah, I'm curious about sort of, you said the intelligent routing of data, also sort of this intelligent placement of functions, co-location of functions, so is that something that the system does where it says, "Okay, these five functions or whatever, these are often communicating with one another, somehow, they use similar state." Maybe the entity design pattern really helps because you can say, "These functions, access this type of entity, so I only have to replicate so much data to these nodes or so forth." I mean, is that sort of the thing you're doing, or is that the type of optimization, and if not, is that somewhere you want to go?

Jonas: Yeah, it's both, I'd say. We do some of that, but we have a lot of work, it's a never-ending story in terms of work here, how we can make it more and more intelligent. So yes and no, I think there's definitely a lot of interesting optimizations we can do here. But what we have is actually what our customers use in the Akka platform on-premise and in the cloud and have used the last 10 years and is working quite well, but we're never done, of course.

Jeremy: Of course.

Rebecca: Okay, I think, correct me if I'm wrong, that we know three things about how Akka Serverless scales, and how this data persists, and how it scales. One is Akka Serverless is built on Akka, two is it uses a high-performance distributed database for data persistence, and then three and this is the one that I saved the best for last, that might be a leading hint into the question that I'm about to ask you is, Akka Serverless leverages Kubernetes under the covers to handle container orchestration at scale. And I'm wondering if using Kubernetes to power stateful serverless applications, should we pull the thread around that purist vernacular, and can you have Kubernetes and serverless in the same sentence? I'm wondering if you've had conversations around that, around using Kubernetes for serverless applications and whether or not people have wanted to antagonize that idea?

Jonas: Yeah, I mean, we do it very successfully, Kubernetes is really serving us extremely well. And we've been trying to get into... Actually, two times we've reported over portal, we started bringing in all the good stuff done with Knative that sits on top of Kubernetes, if not everyone knows. And it hasn't really been there and it's not the Knative teams' fault have to say, they've been extremely open and would love us to help out and are super interested in the ideas that we have. It's just that, there's just so many things you can do now. And now, we've been really focused on shifting this platform.

Jonas: But I think you, I'm pretty certain that we will get back to Knative and start leveraging that more and more along the way because there's some really interesting work being done there. So in short, we've been extremely happy running on Kubernetes and that also makes it very easy to port. Now we're running on GCP only because you need to start somewhere, but we're currently, working on porting it over to become multi-cloud. And perhaps even being able to wrap it out in private clouds and running on generic sort of infrastructure, if that makes sense, it makes that easier, of course.

Rebecca: So does that sort of dovetail into that idea of essential complexity versus accidental complexity in terms of when you evaluate something you're like, "Okay, is this essential or accidental?" And you're like Kubernetes is actually an... well, I would say it's less complex, but that's something you're choosing so that you can have the essentialness rather than the accidentalness?

Jonas: Yeah. I mean, of course, if you really want to optimize everything from the ground up, if I had unlimited amount of time, it would be fun to build something without Kubernetes, being in control of everything from the metal all the way up, but I don't have, we're all have a limited time and Kubernetes has done an amazing job abstracting a lot of these complexities, same thing as Akka's really done a great job, in my opinion, if I may say, in abstracting some of the programming model in application-level complexities in distributed systems.

Jonas: So I think building Akka Serverless on top of these two, Kubernetes, and then Akka on top means that Akka Serverless itself is actually not that much code, to be honest, comparatively, it mainly evolves around bringing all the pieces together, providing SDK for any language, we haven't talked about that, but that's a given, that Akka Serverless is not Java Scala only, just because Akka is. We provide that, we have SDKs for almost any language that is popular out there, JavaScript, TouchWiz, Python, C#, Rasp, there's a long list to go, of course, there's a long list of... So most of the work had been done in two areas, it's bringing everything together in this side, core architecture, the proxy architecture sitting next to your app, and SDKs exposing that to the outside users, so.

Jeremy: Yeah, and I think you're right, I mean, the abstraction piece of it is where Kubernetes comes in, where it's like there's all that complexity of the cloud. Kubernetes adds a ton of complexity in and of itself just managing it, but it reduces that complexity than when you implement it, and then you build things like Knative or OpenFaaS, or any of these other things that then run on top of that, that are another level of abstraction.

Jeremy: And again, from a serverless purist standpoint, if you got to manage the Kubernetes infrastructure yourself, it's probably not as serverless as you would like it to be, but when someone else is managing that, then all of a sudden it sort of changes. So totally get the abstraction thing there and agree with you. I'm just curious about abstractions in general and how you think Akka Serverless fits into this sort of paradigm where you talk a lot about the limitations it removes and this is always something we try to do, like, "Okay, well, now you have state in your servers applications, that's great. So now you can build other types of applications."

Jeremy: But I'm curious, looking at it from the other side, I've built a lot of abstractions in my day, I've used a lot of abstractions, and abstractions inherently also add limitations sometimes. So are there limitations that this adds? I mean, whether it's operationally when somebody has to manage this or maybe even just a developer workflow, does your abstraction add any limitations? Not necessarily that they're bad things, sometimes limitations can be called constraints, which are sometimes good but I'm just curious if that's something that you've addressed or you think about?

Jonas: Yeah, absolutely, that's a great question, and one that I've been thinking a lot about that's important to me. And I think that Akka Serverless is the opposite of Akka in a way when it comes to that. Akka is like this perfect toolbox for the distributed system geek, you can do anything you want, anything under the sun, everything is there and it composes nicely. But it also means that this can be overwhelming to a lot of users building these type of systems. That's why we sort of reach for a more constrained or limited, more guardrail approach, where you can actually... You can't do everything that you can that some people might want to do. But we settled on a set of abstractions that we think can solve most of the use cases, but in a way that simplifies things like having one way of doing things instead of five, and having a specific set of ways of looking at state and we haven't talked about what it looks in terms of actually building something with Akka Serverless but we have a notion of state models.

Jonas: And we now have sort of three state models we're probably going to add more in the future and they all have different semantics, they all have different guarantees because I don't believe in one size fits all. If all you have is a SQL database, then you need to pay the price of strong consistency, always, even if you don't need that, it's quite wasteful, but that's the reason why almost [inaudible 00:34:32] systems out there, enterprise systems, usually use a lot of different databases, you have NoSQL database, and you have a SQL database, you have a graph database, et cetera because it's really no site fits all.

Jonas: And I think that since we have state first-class in the programming model, having a way to define your domain state sort of just annotate more or less in declarative, we do that in Protocol, we haven't talked about that. But you sort of define the schema for your domain state. Here you can sort of annotate simply, how that state should be viewed and what kind of semantics it should have. And ultimately, that will affect, how it's replicated, how it's stored, the consistency, the guarantees it has. So it's like a little bit of guarantees or consistency, a la carte in a way. And that's, of course, liberating, but you only have three, we might add more, but you have to work with what you have still.

Jonas: And when it comes to processing, you have the function essentially, that's essentially the way you look at it, and you have events in and events out. And, of course, we have also gRPC, you make a gRPC call-out, but it's still fairly limited but it turned out to be a great disfunction abstraction and it was one of the reasons why I think Serverless has really taken off because it's so simple. You have state in that's why you have event in and event out.

Jonas: And the only thing really, if you look at it from that perspective, that we have added, is that instead of just having event in and you don't need to care about how the event ended up in your function, and then you emit the event when you're done, we view state the same. So you have state coming into to the function, injected whenever you need it and when you're done, you just emit it. So there's a duality between events and state in our model, which simplifies things but is also quite constraining in a way because your state is only handed to you and you can only emit it, you can't do anything else. But I believe that can be liberating when it comes to cloud, because there's so many things that are so complex anyway.

Rebecca: So yeah, at the top of our conversation, you're like, "I do think that serverless is the future." And I'm actually going to reference the article because I've referenced this twice and so, I want to make sure that Sarah Schlothauer, I think is how you say her name gets credit for an article that helped me understand a bit about Akka Serverless with her interview, with you, from JAXenter and so, credit to you and her for that conversation and for inspiring this next question.

Rebecca: But you talk a bit about Akka versus AWS Lambda and it's like, "Is this the elephant in the room?" And basically, you say "We see Akka Serverless as bringing the possibilities of the incredible serverless paradigm to a much broader set of developers, who until now, have not been able to use serverless for building high-performance business-critical applications." And so, I'm wondering if you can walk us through a few examples of high-performance business-critical applications that do require state, where Akka Serverless now allows you to do this, so that maybe it opens the minds of someone who's listening, being like, "Oh yeah, that is exactly the way I would want to use that, and now I see where that's going."

Jonas: Okay, sure, I mean, it's, of course, a complex question that ideally, I would like to have a whiteboard or something to walk you through. But just superficially and I really think that, one, for example, really excellent use case for this I believe is a digital twin pattern in IoT where you sort of want to map some sort of device in the cloud, like a physical representation of a device in the cloud and having that, because devices are naturally bringing together state and processing, so it's a very natural thing to do, to be able to map this sort of entity that we talked about, but both regardless of where they run, always have their state with them as they move around and having that sort of virtual entity mapping to a physical device in the cloud. I think that's a use case that fits that world perfectly and often that is paired with some sort of real-time streaming, you have sort of streams between these two and streams between these devices and things like that.

Jonas: And having the data always in process then, ensures that you're able to do this sort of more limited, I have to say, but often, sufficient analysis on the fly, so to speak, without having to go back to some sort of backend running Spark, or Flink, or like Hadoop, or whatever that do the mining offline, which also has its place. But being able to actually quickly get intelligent answers back is something that matters a lot in these type of IoT devices.

Jonas: You see some of the same needs when it comes to real-time financial services. Here, I'm drawing a lot of experience from where people have used Akka now, in the cloud and our goal is now to actually enable ideally all of our probably not actually all, but most let's say, of our customers to eventually be able to move to Akka Serverless in the cloud with their full, complete application and not losing much, instead of gaining this great programming model and being able to have teams use whatever language they like, et cetera. So it's of course, a lot more to be said, but I hope it made some sense.

Rebecca: Yeah, illustrative. Thank you.

Jeremy: Yeah. So I'm curious too, kind of following up with that question from Rebecca, so a bunch of these new applications, perfect use cases, you mentioned, the things like the twin pattern and some of those sort of things, I think that makes a lot of sense. I do think though, that there are developers out there who have experience with the cloud, whether it's Azure, or GCP or AWS, and they've been able to build a lot of these applications, but I think we know this is the strife we go through, constant striving for trying to build a better abstraction.

Jeremy: I do think that Akka Serverless and other abstractions, even like Nimbella and some of these other ones, they do open it up to a broader set of developers like you said. I'm curious who those developers are? Do you think those are people that are just not into the cloud yet or do you think these are cloud people that just want to make it easier? Or is there some classification of developer, that you think this opens up to? Or is this just a new type of developer, is this just like a new cloud developer and we should expect stateful serverless to just be a thing?

Jonas: Yeah. I mean, first, I'd like to talk about sort of the composed thing of stateful and stateless serverless, same thing as people who've been building stateless apps with Akka, that's fine and we provide a lot of value there. So it's not either or, I just want to stress that, it's like adding more tools to your toolchain. But that said, I think it can actually open up for more of these classic enterprise backend developers, the people that are on the JVM, perhaps using Spring and Oracle and these things, and then feel reluctant to move to the cloud, they don't think it has everything they need. They're super excited about serverless, but they know that they need more than what has been offered up to now.

Jonas: I think it can open the door for more average developers, perhaps not the early adopters, the bleeding edge developers, they can do everything with anything. And as you said, people have built all of these systems probably using Lambda and using stateless serverless and sometimes with some really clever engineering. But in my opinion, you very often end up with suboptimal and not as efficient systems that it can be because if you try to shoehorn something that's not intended for it, into your solution then you miss out on something, and I believe that when it comes to the actual state, it's better to use abstractions and tools that are meant for addressing that.

Jonas: So it's really in the name of efficiency and in the name of using the best tool for the job, but absolutely, I think it can open up for more people moving to this great new serverless experience and it really deserves that, in my opinion. It should really be the way we consume the cloud, as I said in the beginning. Hopefully, in five years, this is the way we build systems and put the hard stuff on companies Likebend or Microsoft and others to do the heavy lifting and making sure things are up and meet your SLAs.

Jeremy: So we're running out of time and I want to get to two more things-

Rebecca: This happened fast.

Jeremy: It did happen fast, this was an awesome conversation and I know we didn't get into a lot of the technical details or a lot of how to build an application on Akka Servers and so forth, but we're going to put all that information in the show notes, all the links so you can go and you can check that out. But while we've got you here, what was it, eight years ago or more, it was 2014, so how many years ago was that, I can't do the math?

Rebecca: Math, Jeremy.

Jeremy: Math is so hard. Yeah, maths is hard. Anyways, you wrote this thing and initiated this thing called the Reactive Manifesto. And you laid out these really cool sorts of... And then you also have this whole reactive principles thing that you do, and we'll put all the stuff in the show notes, but I thought the reactive systems or the idea of what a reactive system was, and what was in this manifesto, was this idea of being responsive, resilient, elastic, and message-driven.

Jeremy: Now, this is 2004 routine, so this is before Lambda, this is before serverless is even really sort of a thing, and reading that document, it seems like a precursor, I don't know, like the spark for building serverless applications, like you were ahead of your time. But I'm really curious, is that something that you felt that the serverless movement and this idea of building message-driven distributed systems and all that kind of stuff, again, Lambda just reacting to an event and, of course, obviously the other FaaS services that do that. I mean, how do you sort of see that? Was that the vision you had way back then when you were writing that document?

Jonas: I wish I could say yes, I saw serverless coming, but then I probably would've invented it like you said while writing that document. It's not that simple, it's always easy to put one and one together when you look back, harder to predict the future, but sure, halfway, yes, as well. Because I knew where I wanted the industry to go when it comes to tackling these new challenges when building things for the upcoming cloud, building things for the great infrastructure, Amazon storage and provide, and others as well.

Jonas: And the reason for writing it was that, when I was out talking to a lot of people, Akka's been around for about five years back then, people didn't really understand why and how they should conceptualize about these new types of systems. So what I tried to do is distill it down to some core principles and make it easier to grasp the gist of it. It's of course, very superficial in a way and that's the reason why I added this document about reactive principles, it goes a little bit more into depth trying to explain what reactive systems is all about.

Jonas: But anyway, really trying to give a common vocabulary, having one way of looking at it so people communicate and talk about the systems in the same way. And also, a little bit like call for arms. This is what we have to do as an industry, this is where we're going, we better get prepared and join forces in trying to invent this future, like Alan Kay would say. So that's really what it was all about out. And I wish I would have come up with the Lambda serverless experience, I didn't, but that sort of followed these ideas, I suppose. While the reactive systems is more like the groundwork and how the system should work and how we should design these type of systems.

Jeremy: Well, I'm going to give you more credit than you give yourself because I do think that it was very forward-looking and I would be surprised if that document wasn't brought up in some of those early planning meetings.

Rebecca: And you're actually going to get a second chance right now because it is certainly hard to predict the future, but we often ask a lot of our guests to say like, "Hey, where do you see serverless in five years?" But we don't usually have guests that are actively trying to shape that future of serverless. And so, if we were going to predict the future and perhaps you're going to write another manifesto about what's coming, two questions, where do you think serverless is going? And where do you want it to go? Are they aligned?

Jonas: Yeah, I have two passions, and I think they are aligned. I think we're going there and I think we have to go there. And the first one is around the user experience. Even though serverless was really groundbreaking when it comes to that, of course, more has to be done around the user experience and the whole toolchain, from developer tools, through the CI tools, staging, all the way up, and also around the programming malls, that's also part of the user experience, and how we actually resonate, reason about these type of systems. A lot of interesting work is being done there, but I think more work need to be done. And that's a place where we are really trying to think hard what it would mean.

Jonas: I think that's will sort of make it or break it for serverless, unless we make it super accessible. It already is, but I think we can do even better. And then everyone will now start using it, which I think it deserves. So there's, of course, more to say, but off to the next one that I think is equally important for me, at least. And it might be a little bit more early days, but that is the move to the edge, and how the new type of infrastructure we're getting, things like 5G, for example, with the vision of having hundreds or millions of points of presences, even now, essentially in your backyard, where you can have local groups communicating a little bit more offline and having all this super optimization that can be done with 5G. Being able to push serverless up in that realm and having hybrid solutions, because it won't just happen overnight. We'll still have old cloud infrastructure, old in quotes, but traditional cloud infrastructure that want to communicate with these edge clusters, and do it efficiently. And how do you program for that?

Jonas: You mentioned that there is a very interesting small working group, most people probably don't know about, that I was happy to be invited by, it was one of my former colleagues, Roland Kuhn that created this working group around what he calls Local-first software, which also goes a little bit hand in hand with this, but it comes from the other view. What would it mean to build systems where local is the default, and then you can sort of, if needed, if things start to fail, or if you're having incomplete data, then call back to the cloud. So it inverses the problem in a way, and where things can go offline and function very, very well off the grid so to speak or offline with no connection back to the backend.

Jonas: And I think, having great developer experience that serverless has, bringing all of these cool things together for the future of how we build systems, I think it's super exciting. I'm thinking a little about that now and Akka Serverless can absolutely go in that direction, but there's just so many hours in the day, so we haven't done much interesting work to talk about, but that's a lot where I spend my time thinking about, at least.

Jeremy: There are only so many hours in the day. And yes, I totally agree with you, accessibility is a huge thing. Edge computing is going to be huge like you said, that offline and that sync pattern, and what is the abstraction? How did developers do that without it being ridiculously confusing that you've got data at three different levels here, and you've got syncing between all those different nodes and so forth? Super exciting stuff and I am so happy that we have people like you in the serverless community, that are pushing for this type of stuff and trying to make these things better. And it was absolutely awesome having you as a guest, Jonas. So if people want to find out more about you, they want to find out more about Akka Serverless and the other things you're on, all that stuff, how do they do that?

Jonas: Yeah. If you want to find more about me, you can go to my website, jonasboner.com. I don't update it that frequently, so you probably shouldn't go there, but anyway, since you asked.

Rebecca: Great plug, Jonas.

Jonas: If you want to know more about Akka Serverless, go to akkaserverless.com. Akka, by the way, spelled A-K-K-A, two Ks, so that's a bit weird, but go to akkaserverlesss.com. And you want to know more about Lightbend and what we do, then go to lightbend.com, so lightbend.com.

Jeremy: Awesome.

Jonas: Akka is at akka.io, and all these things are crosslinked more or less, so I think you can find your way to whatever you find interesting.

Rebecca: And then they can find you on social, where will they find you on social?

Jonas: Yeah, sure. I mean, jboner, J-B-O-N-E-R on Twitter.

Rebecca: Beautiful.

Jonas: Feel free to reach out, I'd love to know what everyone thinks about this. And both, if you agree, even more, interesting if you disagree, because that's usually where good ideas come.

Jeremy: Get the best feedback that way. All right, well, we will put all of the stuff in the show notes, so you'll have all those links in and people can find him there. Thanks again, Jonas.

Rebecca: Thank you so much.

Jonas: Thank you so much for having me on the show, I'm honored to be in it. I really enjoy your show, so I was really happy to be able to ramble about things I find interesting. So I've enjoyed the conversation. Have a great day.

Rebecca: You too.