Episode #83: Serverless and TypeScript with Tim Suchanek
January 11, 2021 • 61 minutes
On this episode, Jeremy chats with Tim Suchanek about why serverless developers should think about TypeScript, the benefits of type safety, how to equate TypeScript features with existing programming paradigms, how it can benefit edge computing, and much more.
Jeremy: Hi, everyone. I'm Jeremy Daly and this is Serverless Chats. Today, I'm speaking with Tim Suchanek. Hey, Tim. Thanks for joining me.
Tim: Thanks for having me, Jeremy.
Jeremy: You are a TypeScript lead at Prisma. Why don't you tell the listeners a little bit about your background and what Prisma does.
Tim: Yeah. First of all, thanks for having me. It's an honor to be here. I have listened to several episodes already. Prisma is basically a database tooling company, you could say, where our core is implemented as open source. Everything we build is available for everyone, and everyone can contribute. What we basically focus on right now is a database client, database access, but we're also working on schema migrations. What this database client is doing is mostly giving you type-safe access to your database. We do that in TypeScript.
The way Prisma is architectured, we can also implement clients in different languages like Go or Java. We have the core of the query engine is written in Rust, and TypeScript is basically a layer on top to give you type safety for your database. With type safety, I mean if you for example say, "I want to select a certain field," then you will also in your types, in your code will have the guarantee that this field will be there. I believe that still until today this is the only client out there giving you really this kind of experience. How this works is through code generation.
We employ code generation quite heavily, and you define declaratively your schema. You say, "I have a user. A user has a post and so on or has posts." Based on this schema definition, we then generate the whole client in TypeScript. This is now in GA since 2020. We have been working on this for two years, and in general Prisma already exists for nearly five years. That's what we're doing. Obviously, if you use your database, you oftentimes in 2020 you use serverless, and so we see many users, we see a lot of adoption rising there. While still many users are using this in a containerized fashion, we see a big growth also how this is being used in Lambda and serverless.
Jeremy: What about your background? How did you get into TypeScript?
Jeremy: Yeah. It was a very difficult change for me. It was a huge mind shift for me, but anyways, all right. I don't typically fool my listeners here, but I think we're going to fool them a little bit because even though this is a serverless podcast, we're going to talk a lot more about TypeScript. Of course, we're going to link it back to serverless here, and maybe we'll start that in the beginning, but there are so many really, really cool things that you can do with TypeScript. If you're building serverless applications, it's going to help you. I'd like to start by maybe just getting your thoughts on why TypeScript is going to be an important thing for serverless.
I think the main point is really developer experience and also giving you safety. If you look into the whole serverless world, then you oftentimes deal with APIs, third party APIs. Let's say we're dealing with AWS SDK. It's huge. The AWS SDK, it's impossible to know everything, and it's also sometimes hard to find exactly what you want in the docs and in examples. In that case, the types are really helpful. I recently looked into Timestream DB, the new serverless time series database from AWS which became GA a couple of weeks ago. Because it is so new, there was not so much content around yet, so I just checked out the types of the AWS SDK. That really helped me to understand some more edge cases details of the API, what is even available. Maybe they didn't document it. I these days even use types oftentimes to explore an API.
Jeremy: Yeah. No. I totally agree with exploring the APIs because the problem with AWS documentation is it's often very complete, but you have to keep digging and digging and digging to find sometimes the right thing that you need, and just to have it pop up and tell you what methods are available or whatever. The other thing is with the JSDoc stuff, that is something that is ... I've actually been working on another library that I'm trying to add all the JSDoc stuff into as well as converting it to TypeScript, completely to TypeScript, which we can talk about in a second. I found that that is a really, really helpful way for you to do ... It almost forces you to do documentation on your own code so you document your own methods and things like that, but then when that is converted to types, it helps just from the user perspective and gives you that really, really good developer experience.
However, later they added the allowJS effect to the TypeScript config, and with that you can basically start turning file by file into a TypeScript, and also you can make the type checks very loose so to say, you can start with any type, and then you step by step introduce types. I think with that, you can introduce it quite easily. What most developers I guess even in Node.js Already have some kind of compile step. If they don't, yes, you now need to introduce a compile step because you just run the tsc command basically which takes all of your input and it just ... It's a little bit a bundler, not really a bundler, it's rather transpiling.
If you for example use generators, if you use Promise's asynch functions and you want to turn that into ES5, then you may need to transpile that. TypeScript at the same time basically covering Babel, so oftentimes if you have Babel in your project, probably you don't need that anymore if you switch to TypeScript. TypeScript, the compilers for example, also able to parse React like JSX syntax. I think oftentimes, it's getting this compiler into your build pipeline is probably not the biggest thing, especially if you start with file by file, the TypeScript compiler will be very fast in the beginning. It can get a bit slower later if you have really, really big projects, and also depending on how you define the types, how much advanced TypeScript stuff you do because somewhere the compiler needs to run something somewhere, but overall, I suggest really directly starting to introduce the compiler in your code base and file by file convert that over to TypeScript.
Tim: With TypeScript, you have a specific keyword built into language that is is. The keyword is called is. With that, you can have let's say a function that returns if that is indeed that type or not. It returns a boolean, but it has a special meaning now, a semantic meaning in the types because you now can do your if statement. Let's say it is my car, and if this if statement is true, TypeScript knows that everything you do in that if block is definitely accessing your car. This way, you really can build, you can make this connection between runtime and compile time because the reality you not always know the type of it. You need to sometimes validate user input or input from other APIs, and this is a very useful way to discriminate the types. Putting that into a separate function alone already is it a blah, blah, blah, type, this kind of patterns is not possible to do that without a function in TypeScript.
However, we have great libraries for that. One of them is called IOTS, and there you can basically define a TypeScript type in runtime. You get a programmatic API, you can say tdot and then string, and you can construct and nest your types, and it will give you a valid TypeScript type that will make a valid TypeScript type for the compile time, but also gives you a runtime checker. You basically take the whole TypeScript compile time checking into runtime. If you now get an input and you don't know is it a car, you can use one of these libraries IOTS which are really useful for validation here to validate if things are proper.
If you look at the whole validation space, in general it's really exciting what is changing there. If you look into the libraries that people used to use like Yup for example, some people may be familiar with that, they in hindsight added the TypeScript on top which works reasonably well, but now we have native, TypeScript native libraries coming, Zot is one of them, IOTS, and they give you the full package because obviously it's awesome if you can already validate in compile time if possible. And then, later in your code once you have run this check from the library IOTS, you have to guarantee in your code yes, this object has these properties. You don't need to write any code there anymore.
Jeremy: Right. Which is interesting. Maybe we can move on to the project you and I first got connected on, which is the DynamoDB Toolbox.
Jeremy: I spend probably, I don't know, 80% of my time coding on writing runtime checks for the data. Make sure that this particular ... That when you're passing options, that this particular option is a valid option, and then making sure that the type and so forth. I had thought about maybe using joi to do that. Is it pronounced joi? J-O-I, whatever.
Jeremy: But then, I was like, "That just seems like a lot of extra work, and I'm basically doing the same thing, and some of them don't need to be as complex." That is something that would be really, really cool to have is to simply say, "How do I take my TypeScript checks and bring those onto the runtime to enforce them on that side?" Before we get into the DynamoDB Toolbox for a second, maybe we can take a second and talk about something like Deno, some of these runtime for TypeScript things, because those seem really promising. What are your thoughts on those sort of things? I know there's some performance issues.
Tim: Yes. Deno is a really exciting project. It's really awesome, and we are also getting more requests now at Prisma to add Deno support. We don't have it yet because we use the Child Process API quite extensively, and that's just a little bit different like the process model in Deno. We get more and more requests. I still didn't get an answer from anyone because we asked, "Do you use it in production?" Not that many answers yet, but I think that's coming. That is definitely quite expensive to see. If you run the TypeScript compiler, and also a few weeks ago at the TypeScript conference I gave a talk about pushing the ... It was a bit a catchy title. Pushing the compiler to the limit. It was both in terms of complexity, but also in terms of quantity, of how many types we are even having in there.
Now, they are really looking to move this to Rust, which is hardcore. If you look into the compiler code, I used to contribute a few little things, it's crazy. You have one file, 20,000 lines, and that is only Anders Hejlsberg who came up with C# and Delphi, he's the only guy who's allowed to touch that file. It's really like you need to have this extreme context in your head to be able to work on that. I even heard from someone who did an internship at Microsoft that they wanted to move the TypeScript compiler. It was an internal experiment at Microsoft. They wanted to move it I think to .net. There is still some artifact on GitHub somewhere of that experiment, but they gave up. They realized this thing is too complex.
Also, there are for sure type systems that we stop, we don't do any generics like Go that kept it simple. TypeScript, they are just going hardcore on the features. It's insane what kind of features they have. They have sometimes features that even Haskell doesn't have, and Haskell is one of the most powerful type systems. It will be a challenge to move that over to Rust. Coming back to Deno, I think Deno is a really exciting project, and also having not to deal with an overhead of adding TypeScript, it's beautiful because you can just write it and you don't need to deal with the extra compile step anymore.
In practice, I also have to say the compile step is not really a problem for me anymore because I'm 99% of the time using TS Node which is just a little CLI utility that injects the ... so in Nodejs you can overwrite the require system, and what they do, if you require a TypeScript file they quickly transpile it and then they run it basically. That's what TS Node is doing, and that way you can directly say TS Node, then the TypeScript file, and you can just run it so you don't really need to care about this extra transpilation step all the time.
Jeremy: Yeah. The way I found you, by the way, I was building this DynamoDB Toolbox library. I've been building this for over a year now, and I had a lot of people saying, "You got to move to TypeScript, you got to move to TypeScript." I said, "You know what? You're right. This is the time we're going to do it." I spent several weeks importing this thing over to TypeScript. The biggest challenge though was that what I wanted to be able to do was I wanted people to be able to define their schema for entities in the DynamoDB table, and then be able to have that autocomplete basically for them when they got returns from whatever operation they did, whether it was a GAT or a query or something like that.
The way that I figured out this could be possible was you could have them define their own type. If they're building in TypeScript, have them define their own type or schema and pass that into the library, and then that would carry that through. Essentially, they would have to do that twice. They would have to define the structure, define the schema for a particular item, and then they would have to write a TypeScript interface or something on top of that that would allow them to basically retype it or add types to that, whatever. I was like, "That just seems like a lot of work."
I went down this rabbit hole and I came across a talk that you did called "Generics, Conditional Types and Mapped Types," which was absolutely fascinating. What I'd love to do is just quickly tell me or tell the audience what it was you talked about that type in terms of what you were building for Prisma.
Prisma Client, again, you define your schema in a DSL that we came up with at Prisma. We call it the Prisma Definition Language, and it's for the people ... I think it's actually quite similar to a TypeScript definition, or for the people who use GraphQL, it's similar to a GraphQL SDL, schema definition language definition. You basically can just say I have a model user, and then you can point to another model and you can command click it in VSCode and you have this nice experience. That is how you define the whole schema, how you define relations.
And then, another route you have is CLI, that's also written in TypeScript. That generates the whole Prisma Client implementation basically for your particular schema, so all the capabilities it has, query, update, delete. Maybe you have a JSON column in there, and so we give you a specific JSON filter. All of this is generated depending on your schema on your database that you use where you support SQLite, MySQL, Postgres, MariaDB and mSQL. MongoDB is coming, but MongoDB is a completely separate paradigm because it's no SQL. Right now, the SQL database is still more feasible to support them.
What we were looking into is how can we reduce the amount of code generation in order to give people a nicer developer experience, but still have the type safety, and in particular about querying data. Let's say I say I only want the ID. How can I achieve that in a type-safe manner? You could say you for example have ... Let's just build our own database client now. You could have a folder that is called queries, and you could have a file there that you call the user ID query.JSON, and then you just have a JSON definition of that, of what you want, and then you would run a CLI that looks into that and that generates the types or generates what is available. And then, in your code, I have the ID available.
How about you could skip this step of extra generation? How about you can build that into the type system? That is what we did with Prisma Client. Depending on the object which is the query that you write for Prisma Client is just a JSON object. How about we can, depending on the shape of that object, have different types for the output of the query function. In other words, if I'm adding to that object now the name field as well, then suddenly I have this available and the return type of this function, let's say Prisma user query for example, this would be really awesome if that is possible.
We looked into that topic last year January, and we knew there's a mechanism in TypeScript that might make this possible, but we didn't know if that's actually possible. This mechanism is called conditional types. The conditional type really, there's some languages also call it dependent types, there are not many languages first of all which have such a concept. I think C# has it, Haskell has it. The idea is really that you say depending on the input I have, I have a different kind of output. You use generics in that case. Generics, I just refer to it as a variable on the type system, and you can say let's say if the input is a number, the output will be a string, and if the input is a string, the output will be a number. Not sure if that function makes sense, but this kind of combination you can do.
You cannot just say if the input is a string, the output is a string, that you can oftentimes do with generics or you can pack let's say an array around it. You can provide more let's say complex conditions. We looked into that topic, and another important ingredient to be able to implement such an API that maps the input to a specific different kind of output are mapped types. TypeScript has a so-called structural typing, and that means you can define types just by defining a certain structure of the type, and the structure can be let's say an object type for example.
What we are doing now, coming back to Prisma Client, that was a quick discourse into more advanced TypeScript types, Prisma Client, why do we need something like that? Also, to the listeners, we don't have any code here that we are showing on the podcast, but I still hope it's kind of understandable. The idea is now in the mapped type, you give Prisma Client an object type that queries let's say ID and name, we loop through these keys ID, name, and we now map that to a different type. In Prisma Client as an input, we always get a boolean. You just say ID true or ID false if you wanted to be part of the payload, of the result, and we can now on a type system level check is it the concrete boolean true or false, and based on that we can calculate the return type. That is basically how all of this comes together.
Again, when we were looking into that back in January, it was not clear at all, January 2019, it was not at all clear if this is even possible because we did not see anyone out there doing it. Actually, I asked a bunch of questions and the issues in the TypeScript repo, and the answers rather sounded like, "No. What you're doing here is stupid. Don't do it." We did it anyway, and actually that's also what I mentioned in the talk at TS Conf, for a long time Prisma Client was not really type-safe. There were edge cases when you were doing some specific definitions of types that you could break the types basically, which we then also later could fix with some more TypeScript trickery.
What I really want to say here, I think TypeScript might be a bit scary. If I'm now talking about all of these crazy types, do I need to understand them to use TypeScript? No, not at all. I rather think this is something that belongs into a library. This is not something I'm doing in my application code. If I am writing an application, I've never used this kind of types. However, also yesterday at the meetup you saw people for example implementing a game engine, game framework in TypeScript. If you are on that level, library level, there you can put a lot of complexity into it because it's like a surface, it's hidden. I believe that with this, having the complexity rather in the library, we can keep our application code more simple.
Let's say I'm just defining an entity type in a database and I say, "Well, it's got an ID, it's got a name, it's got a date, it's got a client number, something like that." The problem is that if my query doesn't return all that data, then when I'm using TypeScript, it's going to say that it's there even though it's not going to be there. This magic of saying we can dynamically generate types so that even at the coding stage when you're writing the code, that you know whether or not a certain thing is going to be available. To me, that just blows my mind. You're totally right, though, because I think about how I write TypeScript when I'm doing a simple project versus how I write TypeScript when I'm doing a library. Those are two very, very different things.
Jeremy: It's not always true. These things don't always hold true, but it is a really good way to wrap your head around it.
Now, what they did in TypeScript, you can now type them. It's a bit insane. In TypeScript, there's a keyword that is called the infer keyword. What you basically do, you can define a crazy type setup, you can say there's a Promise and there's a function around it, and three more functions around it, and you say in the third argument of the fifth function, you put the infer and then you just call it T for example. If the type that you got in has exactly that form or five functions nested, then you can now take that type of that argument and you can do something with it. You can type check it, you can do certain things.
You can do the same with these template string types, so you can say that if there's for example a template string that has a specific structure, let's say it has ... What they even did, they implemented a split, a string.split on a type level. What is a split? You have a left side, you have a right side, and you need to have split in the middle. They just defined that as a type, and you can do that in TypeScript now, and you can even make that dynamic what it's splitting on so you can again have that as a type parameter, and so you can say now, "I have a left side and a right side." What TypeScript is doing it splits as soon as it finds that pattern. It's a little bit like how Regex is working in greedy fashion.
What you can then do, again, you can take what you split, you can take the third part of it, you split into three parts the first one, the middle you may throw away in a normal split implementation, the third part you again call split on it. That's the powerful thing here. You can define a split function on a type level. What people even started with now is implementing a JSON parser in TypeScript on the type system level. What does that mean? I have a string. Also important to understand why does this even work, in TypeScript you not only have the string type, but you can also have a type that is a very specific sting literal, the string with a specific content that can also be a type. You can say only the type ASD is possible.
If you now have a Foo and Bar, if you have a union type between these, you can basically build your own Enum. You can say it can either be Foo or Bar. It can be the string with a concrete content Foo or the concrete content Bar. What you can do now, you can make these even more complex. You can also do union types in the string, template strings, which then distributes them. You can do combinations and that kind of stuff. People are still exploring where this is useful. I think where it gets useful is if for example in a timestamp API, let's say I am allowed to use the ISO 8601 standard for timestamps, and you can obviously do a runtime check, but now that you can actually do a compile time check if that string, that particular string that you pass in is even valid. What that parser is then doing is it's basically people implemented Regex on the type system level. This kind of things, they are not making or breaking it, but we get more and more these nice things to make the developer experience more awesome.
Jeremy: Yeah. That's amazing. All right. A couple more things about TypeScript specifically, and then I want to bring it back to serverless for a minute. With TypeScript, one of the ... Maybe this is just a debate that I see because I'm reading all these different forms on Reddit and all kinds of things, but when you're building your types, when you're adding things like interfaces and types and things like that, how do you structure your documents? I know I have multiple classes and things like that. Do you include interfaces and types? Do you put those in the same file? Do you separate them out into other files? How do you structure your TypeScript project, and where do you put those definition files?
What we are mostly doing at Prisma still is keeping it simple and having the types collocated with a code, so that means the types are just at the head of the file which is useful to understand the requirements that that function has. As soon as you don't have a simple let's say two argument function anymore and you want to really have five, six, seven arguments, it's useful to have an object as an input type. There, especially also in React, you need to do it anyway when you have prop types, you will just define that type at the top of the ... It's quite useful in my opinion to have that at the top of the file so you see the data requirements, what that function, what that class needs in order to run.
I think collocating it is one way I saw that, and also the @types directory, and then also if you're writing TypeScript, oftentimes the types are embedded. I think this location of types we're mostly talking there about interfaces and object types, but oftentimes you have the types embedded in your code if you write it in TypeScript upfront so that you have your return type typed or you have your parameters typed in there. That's I think even more readable if possible.
Jeremy: Awesome. I'm in the collocation camp. I love to collocate my types in the files. I just find it easier. The second you separate them out and you put them somewhere else and then you have to go look them up, even if I have to import a type into a different file, I just import the other file that has the type in it, and it seems to work pretty well for me. I don't know if I'm doing it right, but it's good to hear that you do a similar thing at Prisma.
Tim: Yes. Exactly. Yeah. What I quickly wanted to mention here is that this really helps us and the team as communication. We have ESLint rules activated. For the people who are familiar with ESLint, there was the effort of TSLint done by Palantir, and they merged that into ESLint, and now it's a plugin for ESLint because they just realized they cannot build such an awesome project again, so now it has access to the ESLint AST, and ESLint has the capability to pass a TypeScript. There are some rules that even force you to make the types a bit more explicit. That helps us in the team. In the moment that you are writing the types, especially coming from the front end, or in general I see that a lot with the front end developers, types are in their way or it feels like, "I don't want to write this type. Why do I even need this?"
I have the feeling the acceptance for this in backend in Node.js Is a bit higher still because also to be fair, the React tooling and so on, it's already quite awesome. ESLint, you could nearly call it TypeScript compiler because it understands the code already quite deeply. But anyway, we have a rule in our setup that forces you to explicitly write down the return types. Why is that interesting? For you in that moment, it's trivial. You just hover with your mouse and you see it's whatever, the TypeScript language servers in VSCode will tell you this is a number. Where this is really useful is for the PRs. If you have the PR review in the GitHub UI, you don't know. You don't have your intellisense. In that case, the types are really useful to just understand what's the contract.
After all, a type system is a contract, what gets in, what gets out. Also, then later if you are refactoring and you are not aware what was the type actually and you think you didn't break anything, with having the return type explicit, you just know what is expected of this function. We found that quite useful.
Jeremy: Yeah. The other thing too that's great besides ESLint is Jest. I use Jest for all my unit testing. With the TypeScript there, it just works really, really well. All right. Let's bring it back to serverless, and then I'll let you go and get back to your TypeScripting. The cloud edge, edge computing seems to be something that is getting more and more popular. We've got Fastly and Cloudflare workers and things like that. I know a lot of them run on the V8 engine, but there's also all this stuff with assembly script and WASM. What are your thoughts on that?
They cannot do that many things. You cannot run native code in there. However, let's say the gate into a native code is WASM WebAssembly. Also, crazy news that Fastly basically hired basically all the WASM people.
Jeremy: Yeah, I saw that.
In Fastly, you can also theoretically do TypeScript, but rather a subset of it called AssemblyScript. They are pushing it really hard. They are also sponsoring the AssemblyScript project. What is AssemblyScript? The idea is really they selected a subset of TypeScript, it's the same Syntax, you feel familiar with it if you use it, that is able to compile to WebAssembly. The Fastly runtime is a different piece. It's completely different design than Cloudflare. They claim I think they have 34 microseconds boot up for the runtime, so if you give them your WebAssembly file, then they can boot that on the edge in 34 microseconds, which is impressive. That is traditionally only Rust.
Tim: Also, their first toolchain, what they're supporting there in the edge beta, I had a look at that, that's only Rust. However, now they're adding the AssemblyScript support, which is exciting because this opens up this extreme let's say advanced edge computing, which normally if you are let's say a solo developer, a small team, you would never look into that. Now, you suddenly have access to that and you can write very fast edge functions. The difference between Fastly and Cloudflare, there are also a bit because I looked into that quite a bit because I'm working on a side project related to that. In Cloudflare, the function is not a Lambda function. It's a function it's always triggered. That's basically the outer edge. Even before the caches hit, that function is always called, and then in that function you have access to the cache API and can get something out of the cache and return it, so the cache there is basically powering Cloudflare.
In Fastly, it's different. Fastly has Varnish in front of everything, which is this quite old cache implementation, but very legit and just works well. You can now talk to Varnish, you can communicate, you can configure Varnish on top, and then if Varnish didn't have the cache, then you can write your function behind that, and that can now also be with AssemblyScript, basically TypeScript. I have to be fair here, it's not really yet TypeScript. Most of the packages will not work. I had a look into it. It's not yet there. It will take a while. Yes, it has the same Syntax, but it's still way to go. If I would use WebAssembly at the edge, I would probably rather do Rust today because the tooling there is much more advanced, but it's still really exciting to see that they are pushing forward this. Cloudflare I think is a great addition as well, and I am actually using it as a cheap alternative to API Gateway because you have pricing of 50 cents per million requests. I think API Gateway, you're somewhere at $2.50 or something, $2.50.
Jeremy: I think it's $3.50 for the REST, and a dollar for the HTTP APIs if that's not confusing enough.
Tim: Yeah. Exactly. Now, you are down to 50 cents basically with Cloudflare because what you can do, there are packages out there that you can directly call your Lambda function with AWS API from a Cloudflare function so you can now have that as your gateway, which is really exciting. Again, all of that can be type-safe if you write it in TypeScript.
Jeremy: That's amazing. Yeah. No. I think the whole WebAssembly thing and edge computing, that to me is this next evolution of serverless computing. What's great about the WebAssembly too is that again, running that on a browser, there's so many more things that you can do there and package so much more compute there. The less that the internet is required here to connect to a little bit of data, but again everything powered in that toolchain is just absolutely amazing. Really exciting stuff. Tim, thank you so much for sharing your knowledge here because honestly, I've learned so much from you just through the video that you did and our previous conversation. I am very, very happy that I found you because it completely changed the way I think about TypeScript. I appreciate that. I hope this did something similar for people listening to this. I will put all of your talks in the show notes. If people do want to get ahold of you, what's the best way for them to do that?
Tim: I guess just on Twitter @timsuchanek. Yeah. I guess in the show notes maybe there you can link to it, but it's just T-I-M-S-U-C-A-N-E-K. The name comes from Czech Republic. My heritage is German, but that's where it originally comes from. Anyway.
Jeremy: Awesome. And then, if people want to check out Prisma they go to github.com/prisma, correct?
Tim: Yes. Or Prisma.io. Exactly.
Jeremy: Awesome. All right. I will get all that stuff into the show notes. Thanks again, Tim.