Episode #10: Testable Serverless Applications with Slobodan Stojanović

August 19, 2019 • 47 minutes

Jeremy chats with Slobodan Stojanović about why testing is important, how it changes with serverless, and how to make our applications more testable using hexagonal architecture.

About Slobodan Stojanović

Slobodan Stojanović is CTO of Cloud Horizon, a software development studio based in Montreal Canada, and the CTO of Vacation Tracker. He is based in Belgrade and is the JS Belgrade meetup co-organizer. Slobodan is an AWS Serverless Hero, Claudia.js core team member, and co-author of "Serverless Applications with Node.js" book, published by Manning Publications.


Jeremy: Hi, everyone. I'm Jeremy Daly and you're listening to Serverless Chats. This week, I'm chatting with Slobodan Stojanović. Hey, Slobodan. Thanks for joining me.

Slobodan: Thanks for having me.

Jeremy: So you're the CTO at Cloud Horizon and Vacation Tracker. So why don't you tell the listeners a bit about yourself and what these two companies do.

Slobodan: Yeah, so for the last seven years, I’m a partner and CTO at Cloud Horizon.  We basically do services for companies, and we build web applications for them. We worked with start ups and some enterprise companies and things like that. For a long time, we thought about, like, building a product. So last year, we finally started doing that. And we built a small tool that will help us to, so whenever our... we have, like, almost 30 people in the company and it's really hard for us now to track who is on a leave and who will be on a leave at some point. So we built a tool to help us out, to track just that. We don't need the full HR system and things like that. So we used serverless to build Vacation Tracker, our product, which is basically a slackbot and now web application that will help you to manage leaves for your company in just a few clicks, and people can request leaves through Slack and things like that. Besides that, I'm  writing a lot about serverless. Not a lot in the last couple of weeks. But before that, I wrote a book about serverless called “Serverless Applications with Node.js” with my friend Aleksandar Simović and I have a couple of Medium posts and a few other articles that are explaining mostly testing architecture of serverless apps. So, that’s it, basically.

Jeremy: Awesome. All right, so I wanted to talk about testing in serverless applications. And so maybe for people who are either new to development or, you know, are maybe used to different,  ways of testing. I mean, why is testing so important? Let’s maybe start with that.

Slobodan: Um, probably the best example I saw so far is the story, one of the stories from the previous book from Gojko Adzic called Humans vs Computers. So there was a guy somewhere in, I think, US that wanted to have custom plates for his car and he tried to fill out the form and he was into sales and boats and things like that. So, he had three choices in that form. First one was like ‘boat’, second one was ‘sailing’ and he didn't want the third choice, so he tried to leave it empty. He wasn't able to do that, so he typed ‘no plates’ or something like that. The first one was occupied, the second one was occupied, so he got ‘no plates’ plates. That was fun so he kept them and after a month, he started receiving a lot of tickets for parking because, you know, in the software for the guys that were filling the tickets for parking, no one predicted that there will be a guy with no custom plates or without plates. And whenever they don't get the plates, they just typed something. And most of time, they typed ‘no plates’ plates. So with testing, they would probably have handled that thing much before it hit the production and everything. So our applications are not perfect. There are so many things that when people start using, that can do in our applications. And when we start testing first we do some analytics of our application and think about the end users and the way that they will test our application. And on the other side, when our application grows really big it's easier for us to, like, be sure that we didn't break something. Unless we wanted to break it, of course.

Jeremy: Right. And when people are building applications, too. I mean, this is something where I mean, I'm a big fan of test-driven development. Where you you actually write your tests first and then you write code to make the tests pass, right? Because then you know what the expected outcomes are, as opposed to, you know, kind of going back after the fact and trying to make some changes there. So, let's talk about testing with serverless, right? Let's get a little bit specific. So is there, or are there, different things that you need to do or, sort of, what's different about testing serverless versus maybe testing a traditional monolithic application?

Slobodan: There are a few different things but, in general, testing is still the same. You want to check if your application works and the way that you want it to work. But some of the things are not your responsibility anymore. For example, infrastructure is, like, the responsibility of your vendor, such as AWS or Microsoft or someone else. So there's no point in really testing that part because that's not really something that, they have their own testing, things like that. But you still need to be sure that your business logic is working in a way that it works. And also all serverless applications are basically microservices, that they're working together. Most of the time, you don't have one monolithic application that is just uploaded to AWS Lambda or something like that. Most of the time, you have, like many different functions. For example, in Vacation Tracker we have, I think more than 80 functions now that they're working together. So it's really important to be sure that all those small services are working together the way we want them to work together, and that our end users have a decent experience and that they can use our application.

Jeremy: Right. And so that the types of tests that you would run I mean, you're still gonna do unit testing, right?

Slobodan: Yeah. So, basically, these types of tests are not that different. We still want to have unit tests because they are still the fastest. But, we also want to have integration tests, that are more important than ever, because we want to check if all these things work in integration, not just our code with the database, but maybe two different Lambda functions that are talking through some SNS topic or something like that. And of course, you wanted to have some kind of end-to-end tests. And maybe UI tests if your application is heavily using UI and things like that. So you still want to keep all these different types of testing that we had in previous non-serverless applications.

Jeremy: So, the other thing I think that's important about UI tests and integration tests is with serverless, they're not quite as expensive as they were before, right?

Slobodan: So yeah, one of the things that I used to kind of show that these testing pyramids. So, testing pyramid was defined by, I think Mike Cohn in his book Succeeding with Agile, a couple of years ago. Probably much more than that, actually, 10 years ago or something like that. So, he tried to explain with the pyramid which tests are the most important for your application, with traditional non-serverless applications. They're probably not traditional, but whatever. Let's call them traditional just to illustrate the point. So you should have a lot of unit tests because they're fast. If you want to test your code, you don't need to speed up the database. You don't need to have even the server or anything. You can just run them on your local machine, and that's it. They're the fastest one, and, of course, the cheapest one because you don't need the infrastructure for them. Then you need the service layer or integration test that will just test your code against the real database and some other things. And these tests are much lower because they require you to have the infrastructure, they have latency and many other things, and they're more expensive because you need to have that infrastructure. In the past, infrastructure was really expensive for some applications, especially for big applications. And, many times so servers that they're working just during the day for test environments and dev environments and they're shut down during the night and things like that. And it's even more expensive and slow when you want to run end-to-end or UI tests because you need to have everything in some environment, and then you need to simulate clicks and many other things. So you don't want to have, those are the things on top of pyramid that are the most expensive and the slowest , so you don't want to have a lot of them.

And in serverless, I like to call that serverless, the testing pyramid. It's basically still a pyramid, but more like Mayan pyramid without, it doesn't look like a triangle anymore because unit tests are still the fastest one and the cheapest one. But then we have integration tests that are cheaper than ever and faster than ever, because we can run them in parallel and speeding up a new Lambda function or DynamoDB table don't take like minutes, it takes a few seconds or even less. And, you pay just for execution, so it's really cheap for you. Most of the time, it will be free. So you can have more of them, and you need to check like integration between your services, that's why you want to have more of them. And finally, for end-to-end, again, tt's cheaper and faster than ever, because it's easy for you to speed up a new environment that looks exactly like production, where you want to test everything. But you can also do some crazy things, like putting your browser inside your Lambda function or something like that. And then you can, instead of running just one track off your UI test, you can split them into, like, 10 different tracks and running 10 different Lambda functions and the cost will be the same because you pay per requests, so it doesn't really, so 10 requests that will be sequential will cost you the same as ten parallel requests. So it's easier and faster.

Jeremy: Yeah, It happens, it happens much faster. So you had mentioned too, this idea of multiple environments, right? So that's actually another really cool thing that you can do in serverless where each developer can have their own environment. And if you wanted to just spin something up and do some manual tests or some even some integration tests in multiple environments, it's a lot easier to do that with cloud then it was to do traditionally.

Slobodan: Exactly. So, especially when you're using serverless applications, and when you have some kind of infrastructure as a code, like CloudFormation or something else, Terraform and things like that. But you can, most of the time, It's just one command that will speed up your new environment, and it will take you like 2, 3 minutes, maybe 10. But it's not like hours or days anymore. So it makes sense, and you don't pay for environment that no one is using, so it makes sense for you to have different environments for each developer, but also different environments for your manual testers. So every person in your QA team can have their own environment, but also, if you have a big feature that you want to test for a few weeks or something like that before you release it, you don't block your test environment. Instead, you can spin up a new environment, and you can still have your test environment for some other smaller futures that you want to ship in the meantime.

Jeremy: So, are there certain things in serverless that you see I need to be tested differently? Or maybe the question is, what is it that we're testing? You mentioned integration because obviously you're hitting up, maybe against a DynamoDB table or different SQS queues or different SNS topics. Are there specific things that you would test in a serverless application that you wouldn't test maybe in a normal application?

Slobodan: So it depends. You're testing things that you think can be risky. For example, when I'm in Claudia.js core team and, when my friend Gojko started Claudia.js, those early days of serverless, it was really hard to start with serverless. So for serverless applications at that moment that you had, like, 10 lines of NodeJS code that will do something and then, like if you want to automate upload, it was like 200 lines of bash script that will do that. It was obvious that risk park is not just in your code anymore, it's also in the process of set up because you need to upload the code, which is not that hard. But you need to set up all the permissions and do many different things to connect everything and to be sure that everything works together as it should. So, yeah, risk is shifting from your code to different parts. I don't think you should build your own deployment library anymore because we have so many awesome libraries that are well tested. But you need to use something that is really well tested to deploy your application. So you need to be sure that your integrations are working fine, that your permissions are working fine and many other things, many other problems that you didn't have before. Like microservices in serverless. And there are certain risks, of course, that you want to cover.

Jeremy: Yes, so let's, I want to talk about those risks. Actually, maybe this gets us into how do we actually test these applications? But maybe we start first, I mean, you mentioned deployment services or deployment frameworks. I mean, obviously, Claudia.js is one, Serverless Framework, SAM for AWS, and there's a whole bunch of them that are out there now. Architect framework. There's a lot of them. And again, the preference for which one you choose is based on a number of factors. But, maybe additional tools for actually doing the testing, right? We just use traditional Jest and Mocha and things like that, right?

Slobodan: Yeah, for our applications we mostly use Jest for writing tests. Before that, we used Jasmine, but Mocha or anything else will work for JavaScript. The same for Python, you can just use, or any other language, you can just use the tools that you used before. And of course, if you want to run end-to-end tests and things like that, there are so many cool things that you can use such Cyprus, for example. It's really cool because before that we're telling you, that's basically another like, I don’t know, another set of skills that you need to have to run your end-to-end tests and things like that. Right now, you can do like, testing, so end-to-end testing by just writing JavaScript, which is awesome for us, because we, our full stack is basically JavaScript now with Node.js, some Typescript and many other things. So, yeah, basically, there are some new tools that can help you such as, I know there were a few. I don't know the names, but there were a few, like testing and CI libraries emerging with Serverless. But most of the time, you can just use your preferred tools and it will just work.

Jeremy: Awesome. All right, so let's get into these risks, right? Because that's one of the things I think when you, when you're talking about testing or testing strategy, we need to know what to test and that's an important question. And you kind of use the word ‘risks’ here, which I think is perfect. It's a good word, because again, you have liabilities, right? Every time we write a line of code, we have introduced some sort of technical debt or some sort of liability that we are now responsible for. If that ever changes, that could break a whole bunch of things. So maybe outline some of these risks, so as you see them.

Slobodan: So, yeah, in our book, my friend Aleksandar come up with four risks, and I really like that explanation on and, like, list of risks that that you need to cover. First one was configuration risk, which is basically if you have the right permissions to do something and if your application is configured to talk to the right database and things like that. Then you have technical workflow risk, which is basically, if you’re handling your errors correctly, or if you're returning the correct response to API gateway or some other tool that you're using or service that you're using. Then you have business logic risks, which is basically related to your code. If your code is working the way it should and things like that. And finally you have some integration risks. With configuration you configured the right services, but integration risks will tell you and will cover the part of these services working together, so if your Lambda function is writing the right way and can talk to your dynamodb in the right way and things like that. So, in order to build a serverless application, in my opinion, you need to consider all those four risks and to make sure that you're testing all of them. And some of them can be tested with like unit tests like part of business logic and, of course, things such as technical workflow risks and things like that. But for some of these things, such as configuration risks and integration risks, you need to have integration tests, of course, but sometimes you even need to have, like end-to-end tests to be sure that everything is working together fine and that your users will be able to use the application.

Jeremy: Right. And actually, I mean, I think that's a really important point where you know, when we write traditional applications and again, I don't know if that's the right word to call them ‘traditional applications’, like you said. But you know, you usually have your databases set up for you. So you know what the URL of the database is and you know the username and password and so forth. But with infrastructure as code, we're spinning up, you know, a separate database or a separate DynamoDB table for, you know, just for this Dev environment or for Developer A's environment or whatever. And so you run that risk of you spin something up, and you might be able to access it locally because you have a profile that you're running on with admin permissions or something. But then once the code starts to run, all of a sudden you can't access the database, right? Or you can't,  or when an error happens, you're not catching that error the same way in the cloud that you might be catching it locally or whatever. So you have all of these different risks. So it is super important that you do spin up an environment to actually test it in the cloud. Otherwise, you could run into all kinds of problems like that.

Slobodan: Yeah, of course. And I saw that so many times, sometimes you try to cut the corners and just deploy something to your test environment and just see if everything works. And then you end up with, like, now digging through cloudwatch logs and many other things to be able to figure out what’s, I don’t know, what’s failing and things like that, right. But yeah, you don't want to waste your time that way. You just want to be sure that things are working together in a way that they should.

Jeremy: Yeah. And so when you're using things like DynamoDB or SNS, you know, and you're writing tests against those, what's your strategy for doing that? Are we using real, do we want to make sure we're always using real cloud versions of those? Do we want to mock those locally? Do we want to run local versions? How's your, how do you deal with that stuff?

Slobodan: So in the early days, when I started working in serverless, I tried to just do the things that I did with non-serverless applications. So my first try was like to install DynamoDB locally and use it as a local database and test against it and things I got. But then there were so many different services such as like Cognito or SNS or many other things that they can, cannot just install locally. So there are some things that can simulate them. But these are simulations. You don't want to run your tests against simulations because they will not give you the right results. And then the second, my second try was basically to mock these things. So I tried to find some complex mocking libraries and things like that that will mock everything and return realistic results and things like that. And even that is like leading to so many errors. And some things are not mocked. You have some special things in your code or we're using the old library, so you need to send some pull requests and who knows what. So these things become more and more complex, and in the end we just decided not to do that. Instead, we want to run our tests locally. Unit test mostly. And whenever we want to test, to run integration tests, I want to test my code against real DynamoDB, which is on AWS or real SNS or real services that are on AWS and that they're working in the cloud. So basically, yeah.

Jeremy: And so what I do is, in my local applications or my unit tests, I do like to run mocks, but I hate those services that do the mocking for you. I mean, I think there's one, like AWS mock service or something like that. I actually just try to capture the actual response from the AWS event, and then I usually save that and use that as a way to quickly run unit tests. But then when you, which is great for just running tests locally and try to figure out if you're making some business logic happen, you don't wanna have to always be calling SNS or DynamoDB in order to do that. But I do find that once you move past those and you go to the integration test, you do want to test it. But is this something where when you run the integration test, do you have to get test every single function? Or do we only just have to really make sure that we're able to connect to the services?

Slobodan: Yeah, that's a great question. And that leads us to another big topic. And this is architecture because you can try to test everything against the real. So, for example, in our code in Vacation Tracker, I mentioned these 80 functions, and I think at least half of these 80 functions are posting some messages to SNS. If I try to test every of these 40 functions against a real SNS that will take ages because for SNS, you need to set up like something that will listen to these messages and things like that, so my test shoot would run for, like, hours or something like that. So instead of doing that, what we do is using ports and adapters or hexagonal architecture, to write our code in a way that we can just basically split everything into different ports and adapters and then we can basically test our business logic against different adapter. So before we do that, we should probably explain ports and adapters and hexagonal architecture because some of the people that are listening to the show will not be, will not know about that. So basically, it's a simple architecture that looks similar to some other architectures, for example, clean architecture and a few others. But the idea behind it is simple. It allows your application to be run and tested in isolation of its eventual environment. So basically your function can be run as a local function, and it can talk to some different, and each of your functions. So your business logic exposes some ports. Let's say that I have a function that receives some messages from Slack and then posts some other message to SNS and saves some data to a database. Instead of connecting to real Slack to get the message to listen to these messages directly from Slack and posting directly to SNS and saving directly to DynamoDB, I want to have three different ports in my code. The first port will be event port that will be able to listen to some different events, and I will have a different adapter for different things, I will have an adapter for Slack that will work in production. But for local testing I will have some kind of local adapter that they contribute from my command line or something else. Then for posting a message on SNS, instead of posting a message directly to SNS, I will have a port for that that will help some interface such as message.post or something like that. And I will have adapter for SNS for production and different adapters for local testing. Maybe just sending a JavaScript event or something like that and, again, for the database, something will assume that data will be stored to a database with some method of database adapter and then database adapter can be DynamoDB adapter that will save the data to DynamoDB, or MongoDB adapter, maybe, or just in-memory database adapter that will save some data in memory and allow us to test everything.

Jeremy: All right, so that was just a ton of information. So, let's break it down a little bit. So we start with let's start with the idea of the ports, right? So when we're building those applications or we’re building some bit of business logic, obviously it needs to interact with something else, it needs to connect to the database like you said, it might need to send a message. Might need to receive a message. The problem is, is that if we build that functionality directly in there and say, you know, maybe we include the AWS SDK with DynamoDB service in there, and the function is supposed to query the database, grab some data, and then maybe send it off to SNS. If it does all of that in that one function, you’re married in a sense, to those services. And you can't, they're not easily testable. That's basically what you're saying, right? So the idea of these ports are to say that we're going to include sort of like, almost like a generic library, maybe like dependency injection. Is that another way to sort of think of it?

Slobodan: Yeah, something like that. But it's called ports and adapters for a good reason. Whenever you're traveling, for example, if you travel to Europe, you want to charge your laptop or your phone. And here in Europe, we have different power plugs for the walls, so you're not able just to plug in your computer in a wall in your hotel room and charge it. Instead, you will need a different cable to do that. But you don't want to buy new cable every time you travel to different countries, because you will end up with a lot of different cables, and it's really expensive and a non-scaleable way to travel. Instead, you have a smaller adapter that will just convert your power plug. Do something that is compatible with a power socket in your hotel room, and that will be small enough that you can carry in your pocket or wherever you want. And it's really cheap. Instead of just buying the cable that will cost you more than $100 this will cost you like $2 or something like that. It's the same with your code. Basically, your business logic doesn't really care. What's your database? You want to save some user? Or let's say that inside our Vacation Tracker we want to save vacation requests. Why would my code even know about my database? It will want to save that vacation request or leave request somewhere, and that needs to be saved in some database. But it doesn't really matter for my code if that database is DynamoDB or I don't know what, maybe MongoDB or something else, as long as that saved and can be read from the database, that's it.

Jeremy: Right. And that's this idea. Like you said, this message.post or the interface. So you would build, your adapter essentially would be a sort of like in the case of the database, like a data access layer, or how we would maybe think of it that way where, you know, DynamoDB might have ‘get item’ or ‘put item’ or something like that. But you would genericize that and expose an interface that says item.get or db.get, db.save or something, and then your individual functions can plug into that and then that way, later on, you could actually change that, and your business logic or your code wouldn't have to change. You would just have to make sure that you had a compatible interface that you could swap out for, I should say compatible adapter, with the same interface that you could swap out later on down the road.

Slobodan: Exactly. So in our code, everything looks much more cleaner now because we have some lambda.js file or lambda.ts file. If you're using TypeScript, which, it's responsibility is basically to require those different adapters or repositories. How we call them, for example, DynamoDB repository to create instances of these repositories and pass them to our main.js or main.ts function, which is basically our business logic. And inside that function, main function that is business logic, we don't require any other repositories and things like that. We get all of them through, like arguments, and we just use them. We assume that whenever we have notification, we assume that that notification library that we got will have notification.send method that will just send some texts as a message to some topic or whatever. Basically, that helps us to like, move away our business logic from integrations and everything. And on the other side, it also helps us to write tests and everything because, as I said, we have a lot of functions that they're doing similar things, like sending messages to SNS or something like that. So instead of testing each function against that, we can now have integration and local unit test for that SNS repository that is posting to real SNS and has that, again, SNS topic and see that that works. And then in our business logic, instead of SNS repository, we can send different notification repository, which can be like a local notification just JavaScript event or something like that. And we can just test against that because that's much faster. And it's still integration between your business logic and SAM adapter.

Jeremy: Yeah. And then, actually, I think that's a really good point about the unit testing, too, because especially if you're doing mocking and you're trying to do some of those things you're always overriding or your stubbing API calls and things like that, and that just gets a little bit complex and kind of messy. And it's easier to, or I think would be easier, using this style of architecture to just swap it out so that when your function calls that you've already got that sort of set up to send back the right messages based off of, you know, the different inputs that it sent. So that seems like a really smart thing to do and then the thing about the SNS topic, or like you're doing this cloud integration piece, that's another thing where if you're, you don't want to test this probably in production, right? You want to test this in some sort of staging environment. But if you're even trying to do integration tests for individual functions, or isolated business logic, you could just change your adapter so that that's using a cloud resource. But it's just using some test resource so that it doesn't, you know, doesn't mess up anything else in your production environment.

Slobodan: Exactly. So I think the best example for that was, a few weeks ago, on a Twitch series that Heitor Lessa did for like, Built on Serverless or something like that. He was showcasing, like with different people, how to build real world applications on serverless and I think you helped him with unit tests.

Jeremy: I was on there, yes. 

Slobodan: Then after that, I was working with him on integration tests, and we actually did this exactly the way that you mentioned. So we didn't want to spin up a new environment every time we want to run integration tests, because we want people to be able to run their integration tests when they're doing development on one function or something like that. So whenever you're doing your development, you run your unit tests, you check your integration test, and after that, you push your code to somewhere where something will do end-to-end tests before it goes to production. But with integration tests, let's say we want to test some DynamoDB table. Basically what we want to test in integration tests is that our code will work fine with DynamoDB table, because with unit tests, you can just assume that everything you send to DynamoDB table will be saved. But maybe you're using attribute that is called status or something like that. Some reserved word, and then it will fail with the real DynamoDB table. So what we did was create a DynamoDB table just for tests on the beginning of the test suite, then running tests against that new DynamoDB table. And then when all tests are finished, then we just destroyed that table. So we don't need to deploy all the environment and things, so that we just want inside the code to create some temporary table test against that and kill it in the end. And that's it for integration test.

Jeremy: That's great. All right, so maybe this would be helpful if we could give some examples of, like why would this be beneficial? Like, how has it benefited you? And I know you have a story about MongodDB in there that maybe you could share.

Slobodan: Yeah. So, the first thing we already mentioned thing with SNS and the issue with testing SNS, posting to SNS, is not with posting the message. That's really fast. And I would be able to post as many messages as I want, but the issue is actually validating if that message was posted. So if we want to validate that then I need to have a Lambda function. Or maybe as the SQS that’s subscribed to that SNS topic so I can see if something was posted to that SNS topic. So that takes a lot of time to set up. And it's a complex thing to set up. So I don't want to test that in each of my functions. But the other thing that you just mentioned is with MongoDB and DynamoDB. When we started our product, the early version used MongoDB and some Express application for part of the app and the rest of it was serverless. And we slowly migrated from, first, we removed Express. And then we tried to remove MongDB, and we still have some parts of the application that are talking to some MongoDB. But most of it is now on DynamoDB. But, it took us a lot of time to migrate everything because of, like, writing everything and testing databases itself. But when we had different repositories for DynamoDB and MongoDB with the same interface- so whenever we wanted to say save leave request, we had db.saveLeaveRequest that got the same params and returned the same value in the end. We were able just to swap these two things. We went to our Lambda.js files. And instead of requiring MongodDB, we just required DynamoDB repository, and our business logic just worked because we don't change the business logic itself. We just change the repositories in the end. So we were basically able to swap the databases for a lot of things without, like, changing our business logic.

Jeremy: Awesome. And so just question, too, in terms of how specific you would get with some of these commands. So, I like to take with DynamoDB, for example, I like to basically take every access pattern which might be ‘getUsers’ or ‘getUsersByXyz’ . I like to write a specific function for each one of those. Is that something that you would do as well? Would you get very specific?

Slobodan: Yes, we do the same. The only issue that we found with that approach is that after some time you have a lot of different small functions that are talking to your database. So basically, we were using classes, some kind of class that is talking to, let's say, DynamoDB. But we have a base class. And then on top of that, we have a few different classes that they're extending that and providing interface to do, for example, things related to leaves, things related to users and things like that. Because otherwise we were in the situation that we got so many functions, and it was really hard for you to, like, see if the thing that you want to do already exists or not inside your code base. So, yeah, that's tricky.

Jeremy: Yeah, I have run into that as well. So, all right, so what about, you have an example, too, with Vacation Tracker. So you use Slack right now, but that's something with this architecture, to be really flexible if you wanted to migrate to something besides Slack or you wanted to add more services too, right?

Slobodan: Exactly. We started with Slack because that was a way for us to validate the MVP. And now we have, like, more than 200 teams paying for our product, so we kind of validated that, the idea, we know that people are ready to pay for this kind of product. And in the past few weeks we were getting more and more questions related to Microsoft Teams because it seems that Microsoft Teams is, like, getting more and more popular now. So, one of the things that this architecture will allow us to do, and we're actually considering doing that in the next few, like, weeks is extending our product not to work just with Slack, but to work with Slack and Microsoft Teams. Because with this approach, we have interfaces such as like postMessageToSlack, like post, not even post messages, like, but post, for example, request,  to that communication channel, and our communication channel can be Slack. Or we can basically add another communication channel that is different in the end. So we need different repository and different adapter. But the port is the same for our business logic that just, like, posting the request somewhere into someone's chat, and that chat can be Microsoft Teams or, who knows, maybe in future, some different things. We even have web applications, so, it's like it doesn't matter for my business logic, where that request is going. But, adapters are there to post that message to different channels and things like that.

Jeremy: Yeah, because all that logic is hidden. All that logic is hidden inside those adapters. And like you said, your business logic doesn't care. You know, and you can split that adapter, you could probably even put an adapter on an adapter, in the sense where you might have, you know, write to one adapter that then decides, maybe based on who the client is, whether it's going to go to Slack or whether it's going to….

Slobodan: We are exactly doing that, actually, because each adapter is basically using hexagonal architecture in its own. So, whenever I want to test different functions inside the adapter, I'm still applying the same principles as I do for my business logic. So, if you want to test your posting messages to a database and you just want to test it if, I don't know to do a unit test of that, you don't want to check that against a real DynamoDB. Then again, you can pass different, like small mini adapters inside and check if you'll get the response with the right format and things like that. It's especially important for, for example, Slack because most of the time you just need to send some kind of JSON and on the other side, Slack will just show something. And it's really hard for us to do integration tests for that. We can do some end-to-end tests, but not integration tests. So it's really important for us to be able to just read that JSON and validate against different things there, documentation or something.

Jeremy: Yeah, and I think that the other thing that this does too, and I'm hoping that we are moving past the, I'll call it a fallacy of vendor lock in, right? But I do think it's kind of important because the other piece of this is data lock-in too, right? Sort of like, once you choose a database technology and you write all these applications that interface with that, you know. Let's say you start with MySQL, and then all of sudden you realize, ‘Hey, MySQL doesn't scale when you get to, sort of, Cloud scale.’

Slobodan: It's even with, even with your programming language. Imagine that you started your application with, for example, Java or something like that, and then you realize that you're cold starts are too big or something like that. So you want to move to Node.js or go where cold starts basically don't exist anymore. So you're locked in with your language. Not just with, or with your framework, or with many other things. Our Express.js application was some kind of lock in, so lock in is kind of real. But Cloud lock in is basically not that real because I never heard about, like, company that moved from one cloud vendor to another unless they had some really specific use case and a lot of long history of bad decisions and things like that. And yeah.

Jeremy: Or maybe, maybe like a start up credit or something. A reason why people move.

Slobodan: Exactly. I don't think start ups, so even if you have a lot of credits or things like that, that will not help you if you need to rewrite your application because development time is much more expensive than, like, your infrastructure.

Jeremy: Right. So maybe, that might be a great way to end this. Sort of, like, what’s your advice? I mean, like, start early, you know? Or think about testing right from the beginning? And what's your advice?

Slobodan: Definitely. You should start thinking about testing early, because if you start thinking about testing at the moment when you have some issues and things like that, it will take you a lot of time to optimize your application and be able to test everything. Most of the time, people are trying to find complex ways to test their code without, like, changing their code. So you want to keep your code the same as it was and just find a really complex way to test it with different mocks and stubbing libraries and things like that. But my approach is completely different. I always want to change my code a bit to allow myself to write tests easier, because that will help me in future, to just move to, let’s say migrate, to other things, not really to other Cloud provider, but to migrate to, for example, from one database to another. Or even more important, my application is growing, so sometimes I just want to switch to another version of another service that we're building or things like that. So basically, it's okay to change your code a bit. To help yourself and your team to write easier tests and to be sure that you can check that your code is working in a way that it should.

Jeremy: Awesome. All right, well, thank you so much Slobodan, for joining me and sharing all this complex knowledge about hexagonal architectures because it’s definitely something that I think people should learn and definitely approach. Why don't you tell the listeners how they can find out about you and you have a ton of things going on, so, you know, feel free. Tell us all about how we could get in touch with you and these other things you're working on.

Slobodan: Yeah, so you mentioned complex knowledge. It's like complex things about simple things, because hexagonal architecture is quite simple in the basic of it, it's really simple. And then we add a layer of complexity around everything, of course. So, thanks for having me. And yeah, of course, people can find me on Twitter. My Twitter handle is @slobodan_. I'm Tweeting a lot about, like, serverless and things like that. And we're also organizing a Serverless Days Belgrade conference in September. So, if anyone of your listeners want to come to Belgrade, Serbia in September, weather will be really nice and we have a nice lineup of people. You can check that on serverlessbelgrade.com. And beside that, you can check Vacation Tracker, our awesome start up application, vacationtracker.io or my other company, Cloud Horizon at cloudhorizon.com. And yeah, I should probably mention Claudia.js, which is claudiajs.com, which is still one of the best frameworks for beginners because it's not a real framework. It's just the deployment library. For complex applications, you should probably use something more complex such as AWS SAM or now CDK, or, of course. Serverless Framework and things like that. And I also mentioned the book that I wrote with my friend Aleksandar Simović. The book is called Serverless Applications with Node.js. And we actually have 5 free e-book copies that we can give away to your listeners. And finally, you can read more about hexagonal architecture and many other things on serverless.pub website, where I write some things with Alexander and Gojko Adzic. There are a lot of great articles written by the two of them on that website, so feel free to visit it.

Jeremy: Awesome. So we appreciate those codes, by the way. So I think what we'll do is, I'll share this in a Tweet, but if you share the podcast or maybe go and leave a review on iTunes or something like that, we'll enter people in and give away some of these codes that they can read your book. All right, awesome. So thank you so much for being here. I will get all of this stuff into the show notes too, so that, you know, there's a lot to take in here.

Slobodan: Thanks for having me in this awesome podcast.