Jeff Triplett is a consultant at REVSYS, a Director of the Python Software Foundation, and President of the Django Events Foundation of North America. We talk about testing Django applications, TDD, and Python/Django community involvement.
Carlton Gibson 0:05
Hi, and welcome to another episode of Django Chat. I'm Carlton Gibson. I'm joined as ever by Will Vincent.
Will Vincent 0:09
Hi, Carlton.
Carlton Gibson 0:10
And today we have a special guest, Jeff Triplett.
Jeff Triplett 0:14
Hi, Jeff. Hi. Thanks for having me.
Will Vincent 0:16
When I thought of this podcast, one of the very first people I wanted to have on or do it with, was you, Jeff. So I'm pleased that we're finally making this happen for Episode 20, or whatever it is. So I was trying to think of how to describe your role in the Django community, but I thought I'd punt it to you because you do so many things. So if someone said, Who are you in? What What do you do in Django? Where would you start?
Jeff Triplett 0:40
Probably that I don't say no, which I'm working on. I bet a lot better to say no for a while. But yeah, I am a Django and Python developer and Lawrence, Kansas, which is the home of Django. I do a lot for the community, but I am a PSF board of director. I'm the president of def net, which is the number nonprofit behind Django con us also been the former chair. I'm a kind of an advisor organizer for the conference to now. I'm a Python fellow, which is not the same thing as Carlton's Django fellowship. I'm also a consultant and a partner at revolution systems in Lawrence, Kansas, which is what pays my bills. So all
Will Vincent 1:19
right, it's you say Rab sensor revolution systems. How should we refer to it?
Jeff Triplett 1:23
revolution systems is what everybody knows it as we just kind of say rubs this, you know, shorthand. So it's like definite the nonprofit I'm a part of is the Django events Foundation, North America. It's just easier to say def No. So Rebecca says the same.
Will Vincent 1:37
Okay, great. So, given that you are a professional consultant, one of the things I want to ask you about is testing. So this is something that Carl and I are asked frequently by readers, excuse me by listeners. And, you know, I think as a consultant, you have sort of a unique view because you get to see a lot of different code bases, existing projects, as well as new projects from scratch, whereas most people are kind of working on the same codebase for years at a time, and so they don't necessarily have a chance to think about how they do test. So I'm curious if maybe the three of us can just talk about testing. And I don't know where we start. If it's on a new project or an existing one, maybe start with an existing one, because we've had that conversation, where I mean, one thing I'm really interested in when you
Carlton Gibson 2:19
get a client, and you get a new project, and then you're like, we've got a, perhaps he hasn't got tests, right, let's start there as the archetype or what do you do? You get a okay.
Will Vincent 2:31
And says, here's my project, but I've got no tests. What do you do?
Jeff Triplett 2:34
Well, that's not uncommon, but it's also not super common. So usually, we're kind of somewhere in the middle. And especially like, as a consultant, you have a problem and you're coming to us either because you want some you want to build something or, you know, you've got fires every time you deploy code. So starting from no tests, the first thing I want to do is basically lay down a simple layer of tests. And so I want to I want to be effective because I do not like writing tests. I don't know that many people who really like writing tests, one of my co workers, Lacey, William Central, she loves writing tests for some reason. And so you open up a test file, you can see thousands of lines of code, not what I do. But what I will do is, I think the most, the most effective way to start is to look at your views in Django. If you're doing Django rest framework stuff, you can also do endpoints, which is also a view in Django. So I'll start there. And what I'll try to do is just a really thin layer of test everything that's a view, record the status codes, and just see what they are. And so you'll get a lot of 200 status codes, which are going to tell you that everything is working in some way, it may not be working 100% but at least Django is telling you things are fine. Then I'll go through and and also record like 405 hundred error tests or status codes, which is going to tell you that maybe there's a piece of data missing to be able to see that view and to have all the information come up. And so once we kind of very quickly go through and just get some kind of test coverage. And then that way at least we have a test suite. So if I go in and delete a bunch of code or I make changes, it'll at least tell me if the status codes are going to be the same. And then once we have that reassurance, then we'll go back and start looking at, you know, why is there a 400 error on this or 404 in this particular review, that may be there, like I'm trying to hit a blog article or something, it just doesn't exist. So then that gives me room to be able then to do like, generate some data or use a, like a factory library, which will, you know, create fake data, and then I can start testing against that to see, you know, does this view work as its as expected. After that, then we'll hit the 500 level code, or 500, excuse me status codes, and start making sure that you know, we have the parts of the view that need to be there for the application to not blow up. So if it's a blog or something, there might be categories or might be tagged as there might be authors. There might be some other bits of information there that's needed just to construct that view and to make it make it testable. Once this is done, another pass I like to go back and do is Django has a really nice feature called a certain num queries inside the test framework. And so we'll use that then to start like measuring how many database calls each view has. And this is usually what's pretty critical. Like if you call me because your homepage is slow. When I start looking at how many database queries are on it, you may have 200 database queries, or you might have 2000. So that's kind of a good place to start. And if like every view starts to have like a ton of database queries, that may mean that you have like a part of middleware that runs in every request that's going to be hitting the database more than it should. And so in this, once you kind of have this base level of tests to know the status goes are good, you know where the database queries are, then we can start making performance decisions. So we've had a lot of like nightmare stories where we had a middleware one time where a client was hitting their database in pulling up and try to do I think this was like their earlier prefetch related days or something or Yeah, I think it's pretty fresh, really? Because pre
Will Vincent 6:01
priced electroplated was there first and then prefetch was added, I think, right?
Jeff Triplett 6:04
Correct. So basically prefetch will load up everything associated with a table. So it might do like a lookup early lookup. And so this was a case where somebody had, let's say, 250,000 categories. And so every single request made to anyone, and he paid on their website was reloading this table, and they weren't caching it either. Yeah. Yeah, yeah. Yeah. So if you cache it, it wouldn't have been bad. It still would be bad, because that's a lot of data.
Will Vincent 6:28
Well, yeah. Send every request. It's a first step. Yeah.
Jeff Triplett 6:31
So I think this has been fixed in Django now. So you can actually like limit the number of records that are going to be accessed. So that was like a pretty blurry like, that was an easy one. Like, let's just eliminate this one middleware cache this one request, this access, so that only gets hit one time. And let's limit actually how much data it's pulling in. And then your website's going to be a whole lot more performant than it was before. But without tests without looking at database queries. There's no way you would ever know that that's actually what the problem was.
Will Vincent 6:58
Yeah, that's super nice. Really good. For you, I assume that to see like the number of database queries, almost everyone's using Django debug toolbar, is that what you're
Jeff Triplett 7:07
we would use, I actually never use it. I think it's a great tool. I just go straight to the test on that, because it's gonna tell me more. And you know, most of the time I'm dealing with back end code. So if I'm working on API's, it doesn't really bother me much. And I think it's a great tool. It's just where do you spend your time? And for me, most of it is just running tests. Yeah. There's actually sites that I've developed features for and API's that I have probably maybe looked at the looked at the actual front end that it maybe once or twice.
Carlton Gibson 7:35
Yeah, I use debug toolbar, not for the templates, because what it'll do is show you the exact templates that will render the includes that came in the context that was available in each one. It's like, yeah, this is that's quite handy, but that's a template layer rather than what Jeff's talking about.
Will Vincent 7:48
Yeah. And where does where does logging fit into all that? Or does it
Jeff Triplett 7:53
It depends on what type of logging really because that's gotten so I mean, so for some people, logging is just printing You know, to the console, we we've dealt with some pretty big systems where, you know, we're basically spitting out JSON via Django struct log. And that's getting consumed by, like another process. So especially like in the Docker world, like we're collecting the logs sometimes and aggregating it to, like another system where we can go in and check and see what's going on. So we can leave clues about metrics, have this user logged in, or this user did something else? So it kind of depends on what you mean by logging, like, fair enough?
Will Vincent 8:29
Yeah. Well, so how, so what about if you have a Greenfield project? What is how do you how does rev sis think about testing in that capacity? Do you have an internal framework that you use on new projects or what what's different versus an existing project that you get?
Jeff Triplett 8:48
Well, if I mean from the beginning, then we don't really have an excuse to not have good test coverage. So, you know, and again, like I tend to start with views, and part of it really depends on what the clients like what's their business objective. Like right now we're working with a with a client is a friend of mine. And it's basically like a brew pub type service where you can walk into a restaurant, you can order food, they can bring it right out to you. And you can do everything through your phone, which is kind of nice, because if the restaurants busy, you don't have to worry about waiting. If, you know, five of you sat down and a sixth and seventh person sit down after you've already made your order, then they can come back and bring you food or drinks or whatever.
Will Vincent 9:26
Yeah, this is super common in Asia in particular, I think this is like the norm plus, yeah, not so much in the US
Jeff Triplett 9:33
yet. Yeah. So for that test, you are for that that situation. We're starting with rest. Basically, we've done all the modeling. We'll write a little bit of model tests for it. But we're basically looking at what's the business logic, you know, is the business logic, or is the main core part of the application going to be inventories? Or is it going to be the ordering systems? And then we try to prioritize like, what is the bread and butter of the application that has to work perfectly and then that's what we're going to make sure that we have 100 And test coverage on. And so from there, we just kind of distill it down. If you have about pages, you probably don't need anything more than just the simple tests and know that this thing doesn't blow up. And like I was talking about database queries earlier, we just want to make sure that somebody doesn't add a template tag to pull in news articles or something and you end up not getting a slide related on and all of a sudden, you've got a couple of thousand database hits on your homepage, because that's really frequent.
Carlton Gibson 10:24
Yeah, it's worth testing the 200 does this template actually render you know, some somebody mistyped a template, they use a template tag, that's just you know, that you never loaded the page you didn't check it was running. But if you've got a unit test, it just loads it and check that it returns something sensible. That error will be picked up.
Jeff Triplett 10:43
Yeah, for sure. So I definitely tend to like err more towards performance when it comes to testing business logic. This is something
Will Vincent 10:49
Carlton I have talked a lot about. And I always like to ask guests, how do you think about where to put that because in Django, you can put it in the views but you can also if you can, you can put it in the managers or in models. I'm wondering what sort of how do you think about, you know, that question of where to put the logic? And obviously, that can change over the project as you build things out. But do you? I mean, do you subscribe to the fat models, slim views approach or
Jeff Triplett 11:14
I kind of I really don't like either one.
I mean, it kind of depends. And part of being a consultant is people are very opinionated. So some people will have thousands of lines of business logic inside of models. And it's hard to follow. And so I usually like a mix of in between, with views, especially with endpoints and stuff we're doing, like rest API's. If you put too much of the business logic in like a traditional Django view, then you're gonna lose that whenever you're using, like Django rest framework. And so I tend to try to do as little as possible with a web request. So sometimes that means we're taking the actual logic and maybe putting it in salary. So I had an application I worked on a couple years ago. They would literally get millions of hits, sometimes during really active like, news and stuff like that. And so people might come and do a bunch of donations. And so what we would do is batch order those, those requests up, and we've put them in salary to be processed. Because we don't want like any blocking that go on when you hit a form, or when you hit an endpoint, we want to try to get the request, you know, we want to try to get a response back to the user as quickly as possible. So in that case, we want the view to not really do much, but do a quick validation, check, get their information, batch things up, put it inside a salary, make the actual request go through, and then we can like email them later to give them the, you know, to congratulate or thank them or whatever. So it really just kind of depends on the application. I know signals aren't really popular. I tend to not want to put a lot in signals. But I do tend to use signals as an easy way to hook into parts of an application to call celery. And that's a pattern that I'm pretty comfortable and happy to use.
Will Vincent 12:55
And then pi test Do you you guys You had you guys that's funny. Did you want to plug your your you guys site?
Jeff Triplett 13:05
Yeah so this is one of those things that just kind of it's I guess passive aggressively annoys me. But uh Lacey and I Lacey Williams Henschel we are coworkers and we worked on Django con together for four or five years. And one thing that we get a lot of especially being in business meetings now that we work together is you know, there might be four or five guys in a room and then there will be you know, one woman and then someone will repeatedly use the term guys over and over and over again. And it's kind of one of those once you hear it, you can't unhear it type of you know, statements or phrases and it's really common and so a lot of people will say like, it's okay to use guys because it's gender neutral. Well, you know, if I were to just use Hey girls, or Hey, gals, in a room full of men, people would notice and you know, it's there's just you know, there's there's better ways to speak and so it's an easy way to make people feel more inclusive and not make people feel exclusive. So We wrote a site, I'm gonna have to look it up. I think it's Hey, guys, cc.
Will Vincent 14:03
Yeah, we'll put it in the show notes.
Jeff Triplett 14:05
And it's just kind of a quick, easy way to send the people in, let them know like, you know, I use it as like an email footer. And so it just kind of reinforces it. So once we've started using, like, y'all, I've had clients thank us after the fact and we don't correct them. It's like, I'm not gonna stop you in a meeting and say it. It's just kind of through osmosis of the more that we can say this, the the more that, you know, people will pick up on it after a while.
Carlton Gibson 14:29
It's just about changing cognitive biases, right? It's not necessarily that there's a conscious prejudice, but there's this kind of cognitive bias and this kind of paternalistic language that we use, and if we just think about it, we can start to change it. And perhaps we change the attitudes that created it. And
Will Vincent 14:47
I made that that statement on purpose. Yeah. But I was going to ask y'all, you use PI tests for your tests. Yes, yes. Maybe talk about why Cuz I don't think everyone does. Um, what do you like? Another another thing to add in, but basically, once you've learned it, it has a number of shortcuts and sort of the selling points of it.
Jeff Triplett 15:11
Yeah, I think it's faster is probably why I started using it. And it does have a lot of really neat plugins. And so I don't know that I have like a bunch of go twos that I can mention off the top of my head. But there's some like different formatting, it kind of has the right hooks involved. So I can like pop into a stack trace at the right places, which, which is really helpful if a test is failing, because I don't want to run a test suite that's going to take a while, you know, if I have to wait five minutes, which your test suite shouldn't, you shouldn't have to wait that long anyways. But I want to be able to like be able to debug and look at the code through tests to try to figure out you know, what's going on, and Python just seems to work well. It's a pretty good standard. You can basically just install it and run it. In most Django apps I've used I really haven't even had to do much to actually just be able to, you know, install it and run it. Just go So actually, I think in a couple dozen projects, I've never actually had to change any code to work with it. ref says does have an open source project called Django test plus, and it kind of adds some nice wrappers around the Django test suite. And I believe the last couple of versions actually has some better pi tests support for it, too, is pi test tends to like this very, like functional view type where you can make easy test functions, and you don't have to have like a class and stuff and Django test plus actually, well, pi tests kind of has some nice wrappers around not needing like a, why would you say like a class based test approach. And so I tend to since I learned a class based test approach, that's what I tend to do with Django just because I don't know, I've got a pattern that I like, but I do plan to spend some more time writing a, like a PI test, native looking tests,
Carlton Gibson 16:47
sort of hidden halfway land in between I kind of use PI tests as a runner, and I love playing a search, you know, just being able to use the SEC, the plain Python assert statement instead of self what was it assert equal? What was the parameters for that? That, you know, that kind of remembering what the assert statements are. But I still like the unit test style classes because they helped me group tests. And perhaps it's just because I'm not quite into the PI test idiom yet. But, you know, I've, that's kind of where I'm at with it.
Will Vincent 17:14
Yeah, I find that test because I'm asked a lot by people learning Django to cover testing. And I, I think it's a it's a, it's a somewhat hard thing to teach because you you rarely see it in any sort of linear fashion, you'll see, you know, maybe an open source thing just has a bazillion tests, and it's hard to make sense of it yet. Most people, they just throw something up, maybe won't have tests. So tests are, I think of them as they're really confusing, until you learn kind of the standard tests that you write every time kind of you were mentioning, you have your standard views test, and then they get really boring, because you're just kind of tossing them at everything. So that's sort of the the leap is like you want to get to the point where you go, Okay, I can just, you know, kind of copy these tests I've written elsewhere to this project, but until you get to that point, It is this feeling of Well, I don't even know how to get this page to work. And now I'm supposed to write tests. And oh, I'm supposed to do test driven development, write the test first. It's like, well, I don't know where I'm trying to go. That's kind of the mindset I see of beginners. And so maybe With that said, I take it you don't do a pure test driven approach, or how do you think about that?
Jeff Triplett 18:18
I don't, I felt like kind of my job, especially with like a Greenfield project is I want to model the project. And I want to see the data first. And so I actually I kind of do a Django admin first approach. So I'll write the models and then I pop them in Django admin, because I feel like everything else in Django kind of derives from that, that pattern. So what I start doing Django, Django rest framework, even a lot of the filter options, a lot of the way the data fits together, the admin for me kind of models that very well. And so I once I can see it and visualize it. And it's pretty easy for me sometimes to discuss with other developers like how to make model changes that maybe they're struggling with, just because I will go into the admin I'll create the models and all that racked with it. And so it's easier to see like a many to many in this situation may not be the best, you know, the best way to model something, especially when you start getting into like Django rest framework because dealing with many of the minis, especially when you're doing like nested serializers. So you've got, you know, multiple layers of models deeper, you know, model references in JSON. It's it's hard to work with. And it's really, you know, you've got these really cool rapid design tools with Django and Django rest framework. And as soon as you start getting into nested serializers, it just all kind of falls down and you're doing everything by hand. So, Carlton, you work on Django rest framework.
Carlton Gibson 19:37
Just been busy nodding away to everything you say, Jeff,
Jeff Triplett 19:41
can you fix that?
Carlton Gibson 19:42
No. And I'll tell you why. Like it's there's a limit to what's auto respectable within you know, reasonable effort unless you know, you have to be. I'm fairly sure that for some specific framework in this specific setup, you can create some code which will in perspective much better than what's available in Django rest framework. But that will only apply to that particular project. Right? And particularly with nested serializers. Like that there's different conventions, are they meant to be read only? Are they meant to be writable? If they are writable? How do you handle conflicts? Do you do you abort the whole request? Do you write the ones which don't conflict and then report the errors on the one that do? How does that look? What's the format of those serializers? Those are decisions that a framework like Django rest framework can't make for you. Not yet, not in this stage and then the technolog where the technology frontier is. Yeah, what else can I say? I you know, I nodded everything you said, Yeah, it does get complex especially you know, if you've got a major major with many too many with a through model and you want to serialize some attributes from the through through model on the nested Sir, this, you've just got to make those decisions. And yes, I think ultimately you have to write serializers by hand. But
Will Vincent 20:54
well, speaking of complexity, another thing I want to ask you about is what are your feelings on Docker in terms of Is it? Do you use it all the time on projects? Do you try to convince clients of that they should switch to it? If they're not? Where do you fall in that spectrum? And I say this, you've, you've actually taught me a lot about Docker over the last few months. So I know you know how to use it, but I'm curious what your preferences are.
Jeff Triplett 21:13
I like it a lot. Actually. I think it's overused. And sometimes people, you know, they see Docker as a hammer, and they just try to use Docker to solve everything. Jessie frizzle, who I'm a huge fan of, she's written some really amazing go code. And I think she's written pretty much everything. Like I've seen discussions before spiral online where people are saying this part of coop is broken. And she's like, no, I fixed it. And they're like, well, this is a bug. And this is something that's a bug in Docker and she'll point to literally her commits and like Qube or Docker, or they'll then complain that this is a problem and a particular like Linux library, and she'll point to the code that she fixed also in Linux, and it's just amazing to see people like that who you know, can who have worked on and work on go code and everything.
Will Vincent 22:00
Didn't you just have another one? I think I just wanted to plug her because she's awesome. So I didn't she just have a tweet a couple days ago that said, maybe containers were a mistake. I saw
Jeff Triplett 22:10
that probably because of that overuse. And I guess I think why I was thinking of Jessie is because she has this awesome GitHub repo of Docker files. And so she runs chrome and Docker on her laptop. And so everything she runs is is Docker based, which is amazing. Wow. I like Docker. My clients are kind of mixed on it. But what we can do with containerization is just absolutely amazing. And I don't mean me as like a consultant, I just mean technology wise, like when I compile a Docker container, when I make an image, I know that that's kind of a contract that it's going to run if I give it the right environment, environment variables and link it to the right, you know, context. So like, if I know this thing needs Postgres and it needs Redis as long as I plugged the right values in and tell it where to go, it's gonna run and I've already tested it, so I know how it's gonna perform. When you start using like a Some of the container services and I don't want to plug in the particular one because they're all great and crap at the same time. But it's really powerful to say like, I can start this running on one node. And if I need 100 of them, I can spin that up in a couple of minutes. And that's amazing, right? And you can also it's more, you can take it from one provider to another much more easily, because it is just containers. It's also a lot easier, I think, working with other developers, because it's no longer Oh, well, you know, use use homebrew but you're on Windows or you're on Linux. So you need to figure something else out. Like once I have a Docker compose set up, and I run tests on it. I know as long as you have Docker installed on your machine, it's gonna run. And that can also run on like GitHub or get lab, I guess, get lab since GitHub doesn't have a test runner thing yet, but, you know, I can I can use ci tools to reliably run and test it. And so that, to me is pretty great.
Will Vincent 23:54
Yeah, I mean, I'm almost tempted with with true beginners who are struggling to who want to do Django to install Python. To say, well, let's just jump to Docker. But you know, I can give them a recipe that works. But then it it's a little much for them. But it does, you know, fundamentally, it is the answer, which is a reproducible environment that all team members can have. And you can almost entirely mimic production locally, too, which is pretty nice to do.
Jeff Triplett 24:20
Yeah, it definitely can be, but like, let's say you take a beginner, how are you going to tell them to deploy Django to digitalocean? Let's say, I tell.
You tell them to use Heroku. Or what's the
Will Vincent 24:32
what's the and I know you're punting to me? So let me just say that I hate deployment.
Jeff Triplett 24:39
Everybody does. That's
Will Vincent 24:41
well, so I, I think the issue is that most people, they don't have to deal with deployment. They're in a company or in an environment where someone else has figured it out or you iterate on it. You just kind of like hold your breath and get through it and then it works. It it's Yeah, it's a pain. I mean, I tell people, I usually recommend Heroku or Python anywhere just because they're pretty straightforward. And they have free tiers, you know, actually with Heroku. So in my book, which should be out by the time this is released Django for professionals, I'm finishing up the deployments chapter. And a big question I had is we use Docker in the book, do we deploy with Docker as well, because Heroku just added this pretty recently. And I think it's simpler when I show you exactly how to do it. But I don't know exactly if it's simpler than I think it is then doing it the old pro c file way. But there's just a lot of for her, okay. I mean, Heroku in particular, there's just a lot of steps, right? I mean, it'd be nice if you know, Carlton, or someone had a perfect drag and drop deployment for Django, but, you know, collect static, you got to configure static files. It's just a lot, a lot of things. It's hard to go from a basic crud app to deployable worthy project without Taking a whole ton of shorter shortcuts, allowed hosts deployment checklist. So, yeah, so that's a, that's a rambling way of saying I haven't fully figured out exactly how to handle it other than I know the two extremes. And as I become more comfortable with Django, I'm less comfortable, just sort of waving my hands at stuff and, and not covering it for people. But I am still a big believer in an iterative learning path. And so I don't want to overwhelm folks with proper deployment. So
Carlton Gibson 26:32
it's tricky, but not sure there is no, there is no really simple solution. I'm still a big fan of, you know, run a VM. And if you want to use Docker, just use Docker on the VM. And if you need to scale it, get a bigger VM for you know, to keep a lot of these things simpler. That only goes so far. And eventually you do need, you know, hundred units and then you need, you know, Kubernetes or whatever, because, you know, VMs aren't gonna go that far.
Will Vincent 26:56
Yeah, I know what what do you think? I mean, we actually just just had a whole episode. On deployments where I think we we gave some of our thoughts. I mean, basically the advice is pick something, learn it, and then you have to focus on it. That's a great problem to have. And then you can take other steps.
Jeff Triplett 27:10
Well, I think that's the only point for me is if you ever tried to like set up a, let's say Django app and not pick on Django, any Python app, or really almost anything, it is a pain when you say, PHP kind of has an advantage, because you can, in theory, copy PHP code and run it remotely on services that run PHP. But if you try to host your own, because if you have a website that needs to scale at all, you're not going to use that model. And so you're going to go and you're going to have to install I don't even know now Apache, you're going to install a bunch of libraries and a bunch of stuff and the average person's not going to be able to do it. And so one thing that I like and I think you could actually do this kind of tutorial, if you go and fire up a Digital Ocean and digitalocean is the sponsor you now I think we plugged them like four or five times but if you go to digitalocean if you installed Docker, you can actually run Run your base image on port 80. And you don't have to install really anything else. And so that to me is pretty powerful to just apt get install, Docker, run your image container, you've got to do some authentication stuff that you'd have to do anyways, even if you're doing a get checkout. So one thing I've liked is through like get lab or through Dockers registry, you can push images from your local machine to that you can run another Docker process, I use one called Watchtower, and all it does is it looks for new images, it pulls in and restarts the service that's running. That's about the most simplistic architecture I can think of for how to run something if you're rolling at your own. Now that's built upon years of knowing, like, here's how these applications and these parts put together. And that's really the challenge. It's not that this stuff is so technical, you can't teach somebody. It's also that, you know, I kind of did all the crappy parts with Linux for a long time and I don't want to do that. I also want to know what's running behind the box because I don't know that I want to hand the container to, you know, a cloud provider and trust what's going on. Behind the scenes, or even though like, if I'm paying X amount of dollars, how do I, why do I get one Docker processor? How many Docker processes do I get? And I think that it's been kind of hard for cloud providers to be able to explain and sell that to people in a way they understand that what stroke is probably done the best job of that was also difficult is the costs, you know, you don't, it's really hard to see how things scale
Carlton Gibson 29:21
and short of like putting them on there and trusting it and then finding you've got a big bill, it's, you know, which you often read, you know, I got $70,000 on Firebase, or whatever it is, like, Ah,
Will Vincent 29:33
you know, how to get all that. I would say that, that that is true, but like that is the problem you want to have. You want to have such crazy traffic that you have a huge bill, and then you it's worth the time to dive in and, and solve it. And I would add, I mean, this comes from even at scale, you know, if you can throw hardware at a problem, any problem you can throw money at. I think that's a good problem to have. I mean, you know, in a startup setting, like yeah, you need to find the money but it's so much easier to throw money. Then some of these trickier problems. So if it's Oh, we just need more machines. Okay, like, that's, that's a good problem, right? Like, you're always gonna have problems. So yeah, you don't want a $70,000 Firebase bill, but kind of got to blame yourself if somehow you rack that up, right. I mean, we just sit in there and you didn't see the analytics. You didn't see the millions of traffic.
Carlton Gibson 30:21
Yeah, I mean,
Will Vincent 30:22
I don't know. You know, I never again, I would take that I would take that problem. Because either you just don't know, either your company goes out of business, because you don't know you're doing anyways. Or you're growing so fast. You can find the money somehow. Like it's solvable.
Jeff Triplett 30:36
Yeah, you know, like going bankrupt is a is a good fix for it.
Will Vincent 30:42
Yeah, but I mean, on deployment separately, I don't know, you know, for any of these providers that are listening. I mean, I was just going through the Heroku docs yesterday, and I've done her who so many times and I was finding the container part confusing. Like I would love to have a simple Django project and just show how to get my words deployed on all day. These major services, but the challenge is that they don't have affiliate fees. So it's just totally giving it away. It takes a lot of time, and they change all the time. So it's just not worth it. But yeah, if anyone wants to, wants to sponsor sponsor the show how to deploy and you will answer me, I would love to I would love to do that. But anyway, separate moving on.
Carlton Gibson 31:27
But as I wanted to talk about your community work, Jeff, I want to talk about the work you've done. So we're
Will Vincent 31:31
gonna jump right to that. Yeah. Cool. Dockers, I want to talk a little bit more.
Carlton Gibson 31:35
Okay, go into one more note.
Will Vincent 31:37
Okay, I just I, I've seen you talk about Jeff, how you're able to evaluate the time a project will take. And I would just love any sort of stories you can tell about the mismatch between what clients think and kind of what you see and how you have that conversation where someone comes to you with some outrageous request. How do you how do you kind of talk them down from that cliff and, you know, what words do you use When they think something will happen, you know, can be done in a month and you know, it takes 12 months, right? Because that that's gonna happen all the time. It doesn't,
Jeff Triplett 32:09
strangely enough, okay, I know I started the podcast saying I don't my other
Will Vincent 32:13
consultants Brando have that all the time. So you're different?
Jeff Triplett 32:17
Well, I mean, it's just, it just is what it is. I mean, so let's say a client comes to us, and they say, We want this really awesome service. And it's something that you know, is taken years years to build. So let's say somebody comes on and they want GitHub or something, we could say, Sure, that's multi multi millions of dollars to do and you know, we're a small company, so we don't have the, you know, like the labor or the the workers to do this thing. But you can't just come in and say, copy every feature of a multi million dollar or Billion Dollar Startup realistically and put it up in two weeks, like it's just not going to happen. So it really depends on Can we just evaluate is this even doable? Part of it is if you come to us with a bad business model, and Frank doesn't more of us than I do. So I can, I can make it sound like a better story. But we do discuss that as well. If you come with us to a really bad business model, and we know that you're not going to be able to be successful at it, or it's just a bad idea, then we want to also get paid. And so if we can't get paid to do the work because of your bad business, then there's no reason for us to do it. We don't want to be in the collections business, we want to be in the right applications and you know, do what we do. So for me, it's realistically a, you have a feature or you have an MVP, so the site that we just finished that's been are we it's going through like a beta process and stuff while we're doing the brewpubs. They had a three month deadline basically, and they knew they have investors and these are the marks they need to hit. So if they say, we need to be able to sit down at a restaurant at 6pm on a Monday, which is what they did, and order a couple of beers or a cheeseburger. That's that's what we know that the goal is, so if we need to scrap other features, because The inventory management maybe doesn't matter as much, because the order has to go through. So for us, it's really a matter of prioritizing. These are the features. And when people come to us and say we want to add this other thing or an investor or somebody thinks would be really cool if we could do XYZ, it's just reminding them of the focus. And that is you, you know, this is the idea you have sold, this is what you need to do. We can do these things. But the more things that you add, the more that deadline is going to get pushed back. And so I can do that, I guess without sounding like, I don't know, I guess I'm being a consultant. As long as I have. I can tell people No, and they thank me for it were when I was probably eight or 10 years ago, they probably didn't thank me, they'd been rude or maybe I didn't know how to communicate it. So maybe eight or 10 years ago, you might not have had the authority that you have now, so that you may i don't know i'm just sort of extrapolating from my own experience, where you find yourself in scenarios where unrealistic deadline after unrealistic deadline after unrealistic deadline and you're saying that this can't be done and here's why. But it's not that that's not necessarily being listened to. And, you know, software estimation is notoriously difficult, and managers notoriously don't listen. So do you think that the later one, thinking specifically about this software estimation problem and like working with clients?
Carlton Gibson 35:21
I mean, and I guess the point I started watching from was when you said eight or 10 years of, you know, eight or 10 years ago, did you find you're able to make that point? Or were you just did you find yourself? No, not listen to that?
Jeff Triplett 35:35
Um, yes.
So eight or 10 years ago was the Lawrence journal world and we basically I was working for one of their startups that was
basically.
So we I worked on Ellington CMS, which was the first Django app or actually, I think it was technically the second Django app, but it's the biggest at the time. And so we roll this thing out to hundreds and hundreds of newspapers, and so We kind of got I got used to unrealistic deadlines. I got used to trying to do all nighters, not because they pushed us to do this thing. But it just was like self induced pressure to do it. And then at some point, I just decided I'm not doing that I'm not going to stay super late, I'm not going to do all nighters, I can tell you what I can do realistically, because I just don't want to do it, like life is too short. So if a client wants to fire me for not, which never happens, because I won't work the weekend for them, then it's not a good fit. And it takes a lot of privilege to be in that situation to be able to say that but clients tend to respect us more for that, because they don't want to be they don't want to be stressed out because we're working on a feature an hour before they're supposed to show it, especially if like their income is based on this, this demo being successful. Most of the time, when we deal with startups, that's a little more normal that, you know, they're kind of going from like funding round the funding round. That's probably not most of what we Do but, you know, I think it's kind of a trust to that. Can we deliver it? When we say we're going to deliver something we're going to do it we're going to deliver. And it's just a matter of what features Do we need to cut out? I think with working on the projects like you do you probably understand is it better to do the feature wrong or not? 100%? Or is it better just to land the feature just to say you did it with something like Django it's better to just take the time it days, it's got to be a good feature is no point there's no point putting in a broken feature. So if it takes you no longer to get in, that's fine because Django is an upstream dependency of all these, you know, end projects, but in a commercial environment, sometimes it's like no this features got it just got to ship on this day. And because the marketing department sold it to the client, before the estimation was ever done.
Will Vincent 37:54
It Okay, so I've,
Carlton Gibson 37:56
yeah, you know, it strikes me that happens all the time in software. I mean, You know, he's not great, obviously.
Jeff Triplett 38:03
I mean, I think it's just because we're not afraid or I'm not afraid to just cut out features or cut out what we need to get done is going to change based on their feedback and what else they need. So, but we're pretty upfront about it. So kind of it's a if we take a project on and we know it's gonna be three months, can we deliver this in three months? And, you know, we we can whittle that back. And and I don't know, I don't sit down with any like task tools that measure productivity, or try to figure out how to do it. When I look at models and kind of know, can I get this MVP up in a couple of weeks, and if I can get the models done, the rest, you know, the DRF stuff, it's just that the tools that we deal with are so mature now that it's pretty easy to kind of guesstimate, I guess, in my mind what it's gonna take to get something at least, you know, plausible
Carlton Gibson 38:48
and passable. Yeah. And that's the key point about estimation, right is that it's all there's this idea, this meme this trope that software timelines aren't estimated, but with mature technology that you And you've done it a lot of times and yeah, they've kind of really are because you know how it will take a day or so to put these views wrapped around this is exactly that, can you model it right. And if you've got the models, and that business layer, the views of a known commodity,
Jeff Triplett 39:13
and you know, DRF makes it so easy to like, well, there's a pattern that I've kind of grown into recently that I like, where we'll do a lot of Model View sets. And I'll basically take all the models and make an admin version of it. And that's gonna require like a staff or super user or something. And that that's easy and quick for us to do. If you're a front end person, that's not the most useful thing. If you're writing like an admin or something for like a JavaScript based admin, then, you know, it may be a pain, but technically, the job is done, you can hit everything. And then from there, that's where we go back to that, like, what's the bread and butter of your application. And we'll start writing like very particular views. So you can place an order or so you can get like a full menu with all the nested everything that you want. And then that way, we can work on optimizing those views, but by default, I do push back on front end people, sometimes they say, this isn't a very performant way of doing things. So if you just want to save an extra request, don't, I'll make it easy. So you can do batch requests, which I would love to see in DRF, by the way,
Carlton Gibson 40:14
well, there's a bunch of things we can do to make lives easier, too, that aren't always nested. Okay, so the thing I'm really hoping for with the whole async thing, the ASCII and whatnot, is that we can have a way of really simp building simple kind of proxy endpoints which hit you know, because the natural, the natural format of a REST API is flat, right. So you have one request for ingredients on for drinks, one for you know, whatever. And but the requests that front end developers want to make is one request fetching, you know, all of these and all of those and all of those and combining them. If I'm really hopeful that the async stuff will enable us to build quick proxy endpoints that will really serve front end developers really well. So I don't know if that's going to come in DRF per se, but it will be part of the Django async move I think
Jeff Triplett 41:02
that's what I'm hoping I think that'd be great. As somebody who does a little bit with Django con, I think we have a few fun surprises.
Will Vincent 41:13
So, alright, so speaking of Django cons and community, let's let's dive in to that area. And maybe we could start with. So when, when we met at pi con, you mentioned you are off to the Python board meeting, what's the titles for that? And then we'll get to the Django specific stuff. So you so you're involved with, you're sort of like the Django representative on Python right now. Um, I wouldn't say that
Jeff Triplett 41:37
because Katie McLaughlin, and if I said her name correctly, there's other Django people on the the Python Software Foundation Board. And so a year ago, pi con, I just a couple of people came to me that were directors and they said they were going to be stepping down and asked me to run. And so I thought, sure, why not. I really didn't think I would get on the board. And then I think I was on a flight From Chicago coming home and my phone blew up when I landed and apparently I'm on the board. So does that entail they were getting and I don't think any what I was getting into so it's been good. It's it's basically a lot of meetings, a lot of email. The the Python Software Foundation is the nonprofit behind the Python language. And so what they try to focus on is the education and the community in growing the community. So like, if you have events, you can hit the PSF for money to try to help and spread the word about Python. I'm still new enough to it that I don't kind of have the spiel down so I'm still trying to learn at all so hopefully none of them listen and then cringe whenever they
Will Vincent 42:42
well Katie, we already interviewed her and she'll, her interview will be coming up before yours. So that's at least one other. Awesome. So what? So education community events, are there other discussions about the language itself and features or is that hounded handled by a separate part of organization for Python.
Jeff Triplett 43:02
So I know that's changed recently. And there's a government's board. So they used to have a BD FL model, which doesn't really scale like one person just doesn't scale. And so I don't know the timeline, but that recently is changed into a council, which I guess they discussed the PI con. So the board itself has nothing to do with the actual language as far as features and stuff goes, when they're going to have meetups or they want to try to like get the all the C Python devs together to work on stuff. I think they do it like once or twice a year. Yeah, so anything we can do to aid c Python for their meetups, or their I guess it's like an in person meeting they'll do a couple times a year, at least once a year. If we can help fund that to try to get everybody together because not everybody works for a big corporation and can afford to go then really anything we can do to support the community and support that movement is good for Python. And that's good for Django and good for all the projects we'd like to work on.
Will Vincent 43:57
Yeah, well, and so just as Python has the Python Software Foundation Django has the Django Software Foundation, which you are what's I know you're part of it. Do you have a specific title as well, within that these days?
Jeff Triplett 44:09
I'm just, uh, I don't even know what I am. I think I'm just a contributing member. I don't know, Carlton, do you know what that's called? DSF member,
Carlton Gibson 44:17
I don't know if there's categories of that as a DSF. Anyone can join the DSF. By the way, if you're a Django user, part of Django community, go to the Django project comm website, look for the Django Software Foundation isn't a form you can fill in and you cannot, you know, apply to join this not it's not like there's an entry criteria other than, you know, just that you're real person, I guess. But join that join the DSF there's a mailing list for that, which you know, it's private in the only DSF members are on it, but it's not private in the sense that you can't be on it. And that's about this the Django community and you know, there's it's not particularly high traffic, so do join. I don't think there's, well then there's definite too. Yeah, definitely. Something else because it definitely is responsible for organizing Django code and writing that.
Will Vincent 45:05
And I know you've done maybe if you want to thump your chest a little bit about your involvement with Django cons, cuz I know you've done a lot.
Jeff Triplett 45:12
Yeah. So I guess I've been a principal organizer for I think this is the fifth year, six years, something like that.
Yeah, I don't even know where to start with with. How do you?
Carlton Gibson 45:23
How do you How did you so I mean, I guess you've been working with Django forever. So you How, how did the bowl of organizing Django code land in your hand,
Jeff Triplett 45:31
I went to the first Django con because I was working at Lawrence journal world, and I've kind of been the most of them, I've probably been the half or 75% of them. After, after, like, everything started off as a community model, a couple of things didn't work, I guess out very well. And so it kind of became a commercial model, which worked for a while and then that didn't work out. So I just happened to go to one where things really weren't working out at all, and so on. I just kind of volunteered to try to like make sure that, you know, there were speakers and other stuff going on. So I guess you could say I was the last minute program chair with about a week or two weeks notice. And so I just, I don't mind begging and asking people for help. So I asked a couple of people if they would speak, I think Russ spoke like three or four times at the conference. So thank you, Russ. That was amazing. And then we basically made sure that Django con happened. Russell, Keith McGee at the time was the DSF. President. He asked me if I'd be interested in maybe, you know, trying to at least get the next Django con going and even if I wasn't going to be involved with it. One of the speakers who stepped up and really helped a lot was Craig Bruce. And so he said, if you're in I'm in, we could maybe create a nonprofit and start this. Stacey Heisler who was at the time I think the DSF, maybe treasure, she volunteered as well. And so the three of us started the Django events foundation in North America. And our goal was just to make sure that Django Con us happened. And then after that things kind of were going a little better every, you know, the community got really behind it and was really, you know, helping us grow and do good stuff. And so, you know, when we were just basically trying to have a conference that you know, evened out every year, or hopefully it lose a bunch of money, we started taking the extra profits, putting some of that back into the DSF, putting those back into other Django events and Python events that we feel, you know, support the community. And so that's kind of how that ball got rolling.
Carlton Gibson 47:28
And so and one thing that's sort of really noticeable about the Django community see one about what outreach you know that to make sure that the speaker's representative and, you know, everybody can attend or you know, that it's not just not just white men in a room for one of a better phrase. So how, how did that get going or, you know, what's the, what's the birth of that and that kind of important stuff, because for me, it's one of the features of the Django community that makes it worth hanging around for
Jeff Triplett 48:00
So the first Django con us was in Santa Clara. And there were a couple of women who worked the Lawrence journal world least one woman worth that launched in the world. And then a couple of the more notable Notable people who were women in the Django community, were there. And it was really odd for me walking in and knowing four or five and maybe 10 women of 200 people. And so it was just a weird experience. It was mostly white men. And so from there, when I looked at, let's say Django con when some of the years were going pretty well, representation wasn't very good. Like we probably the first and I'll just throw out a number, maybe the first nine or 10 years of Django con, I don't know that we had a single woman speak as a keynote speaker. And so looking at that, whenever I was the program chair and started doing like my first actual running Django con. I wanted to make sure that we prioritize having a couple of women speak. And so I think by the time I had my first two keynote speakers of the three realize these are both women, which was a very intentional, and then I was looking for recommendations for another speaker. And so I said, you know, God informant, you know, got just this list of people that we could maybe pick from. And and I picked a couple of people that like I was, you know, fans of or friends of mine who I really thought a lot of because maybe they helped me and technology kind of get my start. And so when we had the opportunity, and I realized like this could send a really powerful message to have like three women keynote speaking. And so it was partially by design, and partially just kind of happenstance, I guess. And once that happened, then I think it started sending kind of a message that this is a little different than your average conference. We're going to prioritize something very different. And I think it's like the Ruth Bader Ginsburg, and I'll get the quote wrong, but it was like, How many?
I think somebody asked,
I should probably like, look this up. It's like how many women is enough to have on the Supreme Court? And I think she basically said all Yeah, like it would not be a problem because there have been all men Do that. So all you know, we didn't have any rules that like x, y, z needs to be all men or women or people of color, what we did do is kind of look at what we've done in the past and notice, and then the side that we need that we need to kind of change the way maybe talks or pick, so let's make it a community model. Let's go to a blind, you know, type process, then let's go to a double blind blind process that way, you know, when you start taking out speaker names, and you start removing the BIOS from, you know, proposals, it really changes the people at the top, so you don't always get the same people all the time. And so we do have some people who are very spectacular, and even though it's double blind, even though they don't put their bios and they're always going to be the top 10 as far as reviews go, because they're really good at what they do, I guess. So. So I think changing that emphasis was a little better. And then from there, we kind of learn from mistakes like you know, because we had that first year where we had three women It was a very valid criticism that we had three white women. And so we should you know, we have a lot of underrepresented people in our community, we can do a better job. And we've just kind of iterated on it over the years. And that's the thing is iteration isn't it? doesn't you know, it's going from here to there? No.
Carlton Gibson 51:17
We're terrible. Let's give up.
Jeff Triplett 51:19
And and I think as far as like some of the people who are doing amazing things in tech, you know, like, I talked about Jesse Zell earlier, like, if I get my pick of who I want to speak at a conference, then why would I not ask Jesse Zell to be a keynote speaker. And she's not this year, by the way. But you know, there are people that are just doing such amazing things that the people who I started following on Twitter definitely changed a lot. And so every year when we ask for keynote speakers, we just look at who we have. And like this year, I don't pick them. I haven't picked them in a few years, but I'll give recommendations on you know, maybe people that I think would be good, and we do get a lot of white men still but it just kind of comes down to who's going to give the right talk at the right time. And you mentioned async earlier, like how cool would it be? to how they Python async kind of keynote for this year. So maybe that happens.
Will Vincent 52:04
Yeah. Well, so as we come to the close of this interview, what else should we discuss? So you have I know a number of side projects, all three of us are, I guess, Jenga developers of a certain age with kids, which What should we discuss that be interesting to people?
Jeff Triplett 52:22
So I think, if I do one thing, well, I think it's driven by the idea of the campsite rule, which is leave the campsite better than you found it. And I think that works for relationships, that works for community projects. I think it works for clients. And that's the one thing that I think that the more time that you spend making the campsite better, or, you know, putting effort into making the community better. I think people who also identify with that are also going to put their efforts into trying to do that. And so if I had any, like parting wisdom, it's the follow the campsite rule. And then the kid stuff i think is great. If you want to talk about that, or
Will Vincent 53:01
Yeah, whatever you guys, let's talk about I mean, side note, we can edit it out if we don't like it, or we can include it people tend to like that.
Carlton Gibson 53:09
I mean, the thing for me with kids, it's it's just about energy levels and maintaining those and realizing that there has to be a rhythm to life that if you don't maintain it's unsustainable and when you're young, and you can perhaps have a line on the weekend or whatever. You can maltreat yourself and somehow get away with it. But actually, it has a cost which perhaps you don't notice. But once you've got youngsters because on the weekend, they get up at exactly the same time as they do every night. There is no line, there is no break, there is no day off. So if you don't care for yourself, you can't perform and you can't perform as a parent and you can't perform work and you can't keep going not just this week or this month, but all year and all next year for the next 20 years when they they're going to need you. And so for me, it's a that that's this kind of lesson. Perfect lesson I learned as a parent was about self care and the priority of self care. And it's much more important than some project deadlines, fake and all the rest of it. That was that would be where I would begin.
Will Vincent 54:10
What I would say, I mean to make an analogy to, it carries over into into projects and coding. So all the things that you know, you need to do you stay up to date with Django ad tests, you just have enough experience where you go, I'm going to prioritize that over maybe this new feature or this rush deadline, because you just know how it's gonna play out. And you've seen the record play and, you know, again, hopefully, this is wisdom. And you know, it ultimately, it just makes you more, it does make you more efficient. I think maybe the challenge is that it's can be hard to communicate that efficiency to someone else who doesn't have that knowledge. But it's certainly I mean, I feel like I'm, every year I'm a better Django developer and it has nothing to do with the number of hours I put in, but it has a lot to do with to your point. Jeff initially of knowing what to say, know about and having that framework for making decisions about, you know, personal time and then also professional time.
Jeff Triplett 55:09
I think I've learned that if I'm not gonna say yes to something that I can't put the energy in to do it. So I'm either going to be all in or I'm going to be all out like I don't I don't like the middle on that. Maybe my kid has taught me that a little bit because my kid just does not care. My kid doesn't care if I have a deadline. You know, he's happy to come to my, you know, computer he loves like Amazon echoes and Alexa's. So he will point and say, boss, and that's really all he cares about is he wants to hear wills on the bus. He just doesn't care about deadlines. And so there's just something about that, that I definitely and I guess I'm the only person here which is a kid. You all have multiple kids, which is just I don't even know how you do it one at a time. The energy stuff you're absolutely right about and I think it's because my kid doesn't care if I'm tired. Let's go outside. Let's do these. things and so that's that's kind of nice, actually. But
Will Vincent 56:04
yeah, it's a good slap across the face of perspective on a 24 seven basis, which, well,
Jeff Triplett 56:11
I think it's helped me too, because I know there's this idea of like flow, and you know, you're in flow and, you know, kids just again, they don't care about flow. And so I've had the kind of learn that I need to be more concise. And if I do, like, if he has taken a nap, maybe that's an hour, maybe that's two hours. But it definitely changes that I've got two hours, whether I'm going to do something around the house or laundry or whatever, I definitely approach it very differently. And to me, it's not an art what we do, it's, it's a job. So I've don't really worry about like writer's block, or coding block or plumbers block, you know, there's, it's just, I've got this amount of time, I'm either going to get this done or I'm not going to get this done. And so I kind of pick and choose what I'm going to work on, if I have the luxury of working on something based on what I think I have available in that time. So I think That's the one last thing about having a kid that was amazing for me is that, you know, flow was just non existent when you're a parent, I felt like,
Will Vincent 57:08
Yeah, well, you're just so deliberate and focused, when you have the chance to be that you can't entertain thoughts of. I mean, it's sort of like, sometimes you'll see people on YouTube with their morning routines and their evening routines, and, you know, people will fill the space that they have, but kids fill the space for you. So when you do have those gaps, you say, right, I'm, you know, I can get a lot done in 20 minutes, actually, on this coding thing if I have to, as well like,
Jeff Triplett 57:35
Yeah, and it's just a
Carlton Gibson 57:37
low, sorry, not as well, the thing, if you spend time with your kid, you call you know, you're spending time to get your right brain, your creative, your brain is busy working on the problem that you couldn't solve anyway. And you come back to the computer later on, and you've got that 20 minutes and it's like, yeah, there's a solution for you. And that was kind of free work in a way because you didn't have to sit at the kitchen. pewter laboring for it. You were playing with your kid. And it's like, Ah, you know, the life pays you back for being present. And it's like, Thank you.
Will Vincent 58:10
Well, I think maybe we could end on that one, Carlton. Well said,
Carlton Gibson 58:14
Jeff, thank you. That's awesome. Thanks for coming on the show. And thank you all for listening as ever. If you've got any questions, you can find us or as chat Django on Twitter, and we're at Django chat talk calm. All right. Bye bye bye.
Carlton Gibson 0:05
Hi, and welcome to another episode of Django chats. I'm Calvin Gibson. I'm joined as ever by Will Vincent. Hi, Carlton. And today we have a special guest, Jeff Triplett.
Jeff Triplett 0:14
Hi, Jeff. Hi. Thanks for having me.
Will Vincent 0:16
When I thought of this podcast, one of the very first people I wanted to have on or do it with, was you, Jeff. So I'm pleased that we're finally making this happen for Episode 20, or whatever it is. So I was trying to think of how to describe your role in the Django community, but I thought I'd punt it to you because you do so many things. So if someone said, Who are you in? What What do you do in Django? Where would you start?
Jeff Triplett 0:40
Probably that I don't say no, which I'm working on. I bet a lot better to say no for a while. But yeah, I am a Django and Python developer and Lawrence, Kansas, which is the home of Django. I do a lot for the community, but I am a PSF board of director. I'm the president of def net, which is the number nonprofit behind Django con us also been the former chair. I'm a kind of an advisor organizer for the conference to now. I'm a Python fellow, which is not the same thing as Carlton's Django fellowship. I'm also a consultant and a partner at revolution systems in Lawrence, Kansas, which is what pays my bills. So all
Will Vincent 1:19
right, it's you say Rab sensor revolution systems. How should we refer to it?
Jeff Triplett 1:23
revolution systems is what everybody knows it as we just kind of say rubs this, you know, shorthand. So it's like definite the nonprofit I'm a part of is the Django events Foundation, North America. It's just easier to say def No. So Rebecca says the same.
Will Vincent 1:37
Okay, great. So, given that you are a professional consultant, one of the things I want to ask you about is testing. So this is something that Carl and I are asked frequently by readers, excuse me by listeners. And, you know, I think as a consultant, you have sort of a unique view because you get to see a lot of different code bases, existing projects, as well as new projects from scratch, whereas most people are kind of working on the same codebase for years at a time, and so they don't necessarily have a chance to think about how they do test. So I'm curious if maybe the three of us can just talk about testing. And I don't know where we start. If it's on a new project or an existing one, maybe start with an existing one, because we've had that conversation, where I mean, one thing I'm really interested in when you
Carlton Gibson 2:19
get a client, and you get a new project, and then you're like, we've got a, perhaps he hasn't got tests, right, let's start there as the archetype or what do you do? You get a okay.
Will Vincent 2:31
And says, here's my project, but I've got no tests. What do you do?
Jeff Triplett 2:34
Well, that's not uncommon, but it's also not super common. So usually, we're kind of somewhere in the middle. And especially like, as a consultant, you have a problem and you're coming to us either because you want some you want to build something or, you know, you've got fires every time you deploy code. So starting from no tests, the first thing I want to do is basically lay down a simple layer of tests. And so I want to I want to be effective because I do not like writing tests. I don't know that many people who really like writing tests, one of my co workers, Lacey, William Central, she loves writing tests for some reason. And so you open up a test file, you can see thousands of lines of code, not what I do. But what I will do is, I think the most, the most effective way to start is to look at your views in Django. If you're doing Django rest framework stuff, you can also do endpoints, which is also a view in Django. So I'll start there. And what I'll try to do is just a really thin layer of test everything that's a view, record the status codes, and just see what they are. And so you'll get a lot of 200 status codes, which are going to tell you that everything is working in some way, it may not be working 100% but at least Django is telling you things are fine. Then I'll go through and and also record like 405 hundred error tests or status codes, which is going to tell you that maybe there's a piece of data missing to be able to see that view and to have all the information come up. And so once we kind of very quickly go through and just get some kind of test coverage. And then that way at least we have a test suite. So if I go in and delete a bunch of code or I make changes, it'll at least tell me if the status codes are going to be the same. And then once we have that reassurance, then we'll go back and start looking at, you know, why is there a 400 error on this or 404 in this particular review, that may be there, like I'm trying to hit a blog article or something, it just doesn't exist. So then that gives me room to be able then to do like, generate some data or use a, like a factory library, which will, you know, create fake data, and then I can start testing against that to see, you know, does this view work as its as expected. After that, then we'll hit the 500 level code, or 500, excuse me status codes, and start making sure that you know, we have the parts of the view that need to be there for the application to not blow up. So if it's a blog or something, there might be categories or might be tagged as there might be authors. There might be some other bits of information there that's needed just to construct that view and to make it make it testable. Once this is done, another pass I like to go back and do is Django has a really nice feature called a certain num queries inside the test framework. And so we'll use that then to start like measuring how many database calls each view has. And this is usually what's pretty critical. Like if you call me because your homepage is slow. When I start looking at how many database queries are on it, you may have 200 database queries, or you might have 2000. So that's kind of a good place to start. And if like every view starts to have like a ton of database queries, that may mean that you have like a part of middleware that runs in every request that's going to be hitting the database more than it should. And so in this, once you kind of have this base level of tests to know the status goes are good, you know where the database queries are, then we can start making performance decisions. So we've had a lot of like nightmare stories where we had a middleware one time where a client was hitting their database in pulling up and try to do I think this was like their earlier prefetch related days or something or Yeah, I think it's pretty fresh, really? Because pre
Will Vincent 6:01
priced electroplated was there first and then prefetch was added, I think, right?
Jeff Triplett 6:04
Correct. So basically prefetch will load up everything associated with a table. So it might do like a lookup early lookup. And so this was a case where somebody had, let's say, 250,000 categories. And so every single request made to anyone, and he paid on their website was reloading this table, and they weren't caching it either. Yeah. Yeah, yeah. Yeah. So if you cache it, it wouldn't have been bad. It still would be bad, because that's a lot of data.
Will Vincent 6:28
Well, yeah. Send every request. It's a first step. Yeah.
Jeff Triplett 6:31
So I think this has been fixed in Django now. So you can actually like limit the number of records that are going to be accessed. So that was like a pretty blurry like, that was an easy one. Like, let's just eliminate this one middleware cache this one request, this access, so that only gets hit one time. And let's limit actually how much data it's pulling in. And then your website's going to be a whole lot more performant than it was before. But without tests without looking at database queries. There's no way you would ever know that that's actually what the problem was.
Will Vincent 6:58
Yeah, that's super nice. Really good. For you, I assume that to see like the number of database queries, almost everyone's using Django debug toolbar, is that what you're
Jeff Triplett 7:07
we would use, I actually never use it. I think it's a great tool. I just go straight to the test on that, because it's gonna tell me more. And you know, most of the time I'm dealing with back end code. So if I'm working on API's, it doesn't really bother me much. And I think it's a great tool. It's just where do you spend your time? And for me, most of it is just running tests. Yeah. There's actually sites that I've developed features for and API's that I have probably maybe looked at the looked at the actual front end that it maybe once or twice.
Carlton Gibson 7:35
Yeah, I use debug toolbar, not for the templates, because what it'll do is show you the exact templates that will render the includes that came in the context that was available in each one. It's like, yeah, this is that's quite handy, but that's a template layer rather than what Jeff's talking about.
Will Vincent 7:48
Yeah. And where does where does logging fit into all that? Or does it
Jeff Triplett 7:53
It depends on what type of logging really because that's gotten so I mean, so for some people, logging is just printing You know, to the console, we we've dealt with some pretty big systems where, you know, we're basically spitting out JSON via Django struct log. And that's getting consumed by, like another process. So especially like in the Docker world, like we're collecting the logs sometimes and aggregating it to, like another system where we can go in and check and see what's going on. So we can leave clues about metrics, have this user logged in, or this user did something else? So it kind of depends on what you mean by logging, like, fair enough?
Will Vincent 8:29
Yeah. Well, so how, so what about if you have a Greenfield project? What is how do you how does rev sis think about testing in that capacity? Do you have an internal framework that you use on new projects or what what's different versus an existing project that you get?
Jeff Triplett 8:48
Well, if I mean from the beginning, then we don't really have an excuse to not have good test coverage. So, you know, and again, like I tend to start with views, and part of it really depends on what the clients like what's their business objective. Like right now we're working with a with a client is a friend of mine. And it's basically like a brew pub type service where you can walk into a restaurant, you can order food, they can bring it right out to you. And you can do everything through your phone, which is kind of nice, because if the restaurants busy, you don't have to worry about waiting. If, you know, five of you sat down and a sixth and seventh person sit down after you've already made your order, then they can come back and bring you food or drinks or whatever.
Will Vincent 9:26
Yeah, this is super common in Asia in particular, I think this is like the norm plus, yeah, not so much in the US
Jeff Triplett 9:33
yet. Yeah. So for that test, you are for that that situation. We're starting with rest. Basically, we've done all the modeling. We'll write a little bit of model tests for it. But we're basically looking at what's the business logic, you know, is the business logic, or is the main core part of the application going to be inventories? Or is it going to be the ordering systems? And then we try to prioritize like, what is the bread and butter of the application that has to work perfectly and then that's what we're going to make sure that we have 100 And test coverage on. And so from there, we just kind of distill it down. If you have about pages, you probably don't need anything more than just the simple tests and know that this thing doesn't blow up. And like I was talking about database queries earlier, we just want to make sure that somebody doesn't add a template tag to pull in news articles or something and you end up not getting a slide related on and all of a sudden, you've got a couple of thousand database hits on your homepage, because that's really frequent.
Carlton Gibson 10:24
Yeah, it's worth testing the 200 does this template actually render you know, some somebody mistyped a template, they use a template tag, that's just you know, that you never loaded the page you didn't check it was running. But if you've got a unit test, it just loads it and check that it returns something sensible. That error will be picked up.
Jeff Triplett 10:43
Yeah, for sure. So I definitely tend to like err more towards performance when it comes to testing business logic. This is something
Will Vincent 10:49
Carlton I have talked a lot about. And I always like to ask guests, how do you think about where to put that because in Django, you can put it in the views but you can also if you can, you can put it in the managers or in models. I'm wondering what sort of how do you think about, you know, that question of where to put the logic? And obviously, that can change over the project as you build things out. But do you? I mean, do you subscribe to the fat models, slim views approach or
Jeff Triplett 11:14
I kind of I really don't like either one.
I mean, it kind of depends. And part of being a consultant is people are very opinionated. So some people will have thousands of lines of business logic inside of models. And it's hard to follow. And so I usually like a mix of in between, with views, especially with endpoints and stuff we're doing, like rest API's. If you put too much of the business logic in like a traditional Django view, then you're gonna lose that whenever you're using, like Django rest framework. And so I tend to try to do as little as possible with a web request. So sometimes that means we're taking the actual logic and maybe putting it in salary. So I had an application I worked on a couple years ago. They would literally get millions of hits, sometimes during really active like, news and stuff like that. And so people might come and do a bunch of donations. And so what we would do is batch order those, those requests up, and we've put them in salary to be processed. Because we don't want like any blocking that go on when you hit a form, or when you hit an endpoint, we want to try to get the request, you know, we want to try to get a response back to the user as quickly as possible. So in that case, we want the view to not really do much, but do a quick validation, check, get their information, batch things up, put it inside a salary, make the actual request go through, and then we can like email them later to give them the, you know, to congratulate or thank them or whatever. So it really just kind of depends on the application. I know signals aren't really popular. I tend to not want to put a lot in signals. But I do tend to use signals as an easy way to hook into parts of an application to call celery. And that's a pattern that I'm pretty comfortable and happy to use.
Will Vincent 12:55
And then pi test Do you you guys You had you guys that's funny. Did you want to plug your your you guys site?
Jeff Triplett 13:05
Yeah so this is one of those things that just kind of it's I guess passive aggressively annoys me. But uh Lacey and I Lacey Williams Henschel we are coworkers and we worked on Django con together for four or five years. And one thing that we get a lot of especially being in business meetings now that we work together is you know, there might be four or five guys in a room and then there will be you know, one woman and then someone will repeatedly use the term guys over and over and over again. And it's kind of one of those once you hear it, you can't unhear it type of you know, statements or phrases and it's really common and so a lot of people will say like, it's okay to use guys because it's gender neutral. Well, you know, if I were to just use Hey girls, or Hey, gals, in a room full of men, people would notice and you know, it's there's just you know, there's there's better ways to speak and so it's an easy way to make people feel more inclusive and not make people feel exclusive. So We wrote a site, I'm gonna have to look it up. I think it's Hey, guys, cc.
Will Vincent 14:03
Yeah, we'll put it in the show notes.
Jeff Triplett 14:05
And it's just kind of a quick, easy way to send the people in, let them know like, you know, I use it as like an email footer. And so it just kind of reinforces it. So once we've started using, like, y'all, I've had clients thank us after the fact and we don't correct them. It's like, I'm not gonna stop you in a meeting and say it. It's just kind of through osmosis of the more that we can say this, the the more that, you know, people will pick up on it after a while.
Carlton Gibson 14:29
It's just about changing cognitive biases, right? It's not necessarily that there's a conscious prejudice, but there's this kind of cognitive bias and this kind of paternalistic language that we use, and if we just think about it, we can start to change it. And perhaps we change the attitudes that created it. And
Will Vincent 14:47
I made that that statement on purpose. Yeah. But I was going to ask y'all, you use PI tests for your tests. Yes, yes. Maybe talk about why Cuz I don't think everyone does. Um, what do you like? Another another thing to add in, but basically, once you've learned it, it has a number of shortcuts and sort of the selling points of it.
Jeff Triplett 15:11
Yeah, I think it's faster is probably why I started using it. And it does have a lot of really neat plugins. And so I don't know that I have like a bunch of go twos that I can mention off the top of my head. But there's some like different formatting, it kind of has the right hooks involved. So I can like pop into a stack trace at the right places, which, which is really helpful if a test is failing, because I don't want to run a test suite that's going to take a while, you know, if I have to wait five minutes, which your test suite shouldn't, you shouldn't have to wait that long anyways. But I want to be able to like be able to debug and look at the code through tests to try to figure out you know, what's going on, and Python just seems to work well. It's a pretty good standard. You can basically just install it and run it. In most Django apps I've used I really haven't even had to do much to actually just be able to, you know, install it and run it. Just go So actually, I think in a couple dozen projects, I've never actually had to change any code to work with it. ref says does have an open source project called Django test plus, and it kind of adds some nice wrappers around the Django test suite. And I believe the last couple of versions actually has some better pi tests support for it, too, is pi test tends to like this very, like functional view type where you can make easy test functions, and you don't have to have like a class and stuff and Django test plus actually, well, pi tests kind of has some nice wrappers around not needing like a, why would you say like a class based test approach. And so I tend to since I learned a class based test approach, that's what I tend to do with Django just because I don't know, I've got a pattern that I like, but I do plan to spend some more time writing a, like a PI test, native looking tests,
Carlton Gibson 16:47
sort of hidden halfway land in between I kind of use PI tests as a runner, and I love playing a search, you know, just being able to use the SEC, the plain Python assert statement instead of self what was it assert equal? What was the parameters for that? That, you know, that kind of remembering what the assert statements are. But I still like the unit test style classes because they helped me group tests. And perhaps it's just because I'm not quite into the PI test idiom yet. But, you know, I've, that's kind of where I'm at with it.
Will Vincent 17:14
Yeah, I find that test because I'm asked a lot by people learning Django to cover testing. And I, I think it's a it's a, it's a somewhat hard thing to teach because you you rarely see it in any sort of linear fashion, you'll see, you know, maybe an open source thing just has a bazillion tests, and it's hard to make sense of it yet. Most people, they just throw something up, maybe won't have tests. So tests are, I think of them as they're really confusing, until you learn kind of the standard tests that you write every time kind of you were mentioning, you have your standard views test, and then they get really boring, because you're just kind of tossing them at everything. So that's sort of the the leap is like you want to get to the point where you go, Okay, I can just, you know, kind of copy these tests I've written elsewhere to this project, but until you get to that point, It is this feeling of Well, I don't even know how to get this page to work. And now I'm supposed to write tests. And oh, I'm supposed to do test driven development, write the test first. It's like, well, I don't know where I'm trying to go. That's kind of the mindset I see of beginners. And so maybe With that said, I take it you don't do a pure test driven approach, or how do you think about that?
Jeff Triplett 18:18
I don't, I felt like kind of my job, especially with like a Greenfield project is I want to model the project. And I want to see the data first. And so I actually I kind of do a Django admin first approach. So I'll write the models and then I pop them in Django admin, because I feel like everything else in Django kind of derives from that, that pattern. So what I start doing Django, Django rest framework, even a lot of the filter options, a lot of the way the data fits together, the admin for me kind of models that very well. And so I once I can see it and visualize it. And it's pretty easy for me sometimes to discuss with other developers like how to make model changes that maybe they're struggling with, just because I will go into the admin I'll create the models and all that racked with it. And so it's easier to see like a many to many in this situation may not be the best, you know, the best way to model something, especially when you start getting into like Django rest framework because dealing with many of the minis, especially when you're doing like nested serializers. So you've got, you know, multiple layers of models deeper, you know, model references in JSON. It's it's hard to work with. And it's really, you know, you've got these really cool rapid design tools with Django and Django rest framework. And as soon as you start getting into nested serializers, it just all kind of falls down and you're doing everything by hand. So, Carlton, you work on Django rest framework.
Carlton Gibson 19:37
Just been busy nodding away to everything you say, Jeff,
Jeff Triplett 19:41
can you fix that?
Carlton Gibson 19:42
No. And I'll tell you why. Like it's there's a limit to what's auto respectable within you know, reasonable effort unless you know, you have to be. I'm fairly sure that for some specific framework in this specific setup, you can create some code which will in perspective much better than what's available in Django rest framework. But that will only apply to that particular project. Right? And particularly with nested serializers. Like that there's different conventions, are they meant to be read only? Are they meant to be writable? If they are writable? How do you handle conflicts? Do you do you abort the whole request? Do you write the ones which don't conflict and then report the errors on the one that do? How does that look? What's the format of those serializers? Those are decisions that a framework like Django rest framework can't make for you. Not yet, not in this stage and then the technolog where the technology frontier is. Yeah, what else can I say? I you know, I nodded everything you said, Yeah, it does get complex especially you know, if you've got a major major with many too many with a through model and you want to serialize some attributes from the through through model on the nested Sir, this, you've just got to make those decisions. And yes, I think ultimately you have to write serializers by hand. But
Will Vincent 20:54
well, speaking of complexity, another thing I want to ask you about is what are your feelings on Docker in terms of Is it? Do you use it all the time on projects? Do you try to convince clients of that they should switch to it? If they're not? Where do you fall in that spectrum? And I say this, you've, you've actually taught me a lot about Docker over the last few months. So I know you know how to use it, but I'm curious what your preferences are.
Jeff Triplett 21:13
I like it a lot. Actually. I think it's overused. And sometimes people, you know, they see Docker as a hammer, and they just try to use Docker to solve everything. Jessie frizzle, who I'm a huge fan of, she's written some really amazing go code. And I think she's written pretty much everything. Like I've seen discussions before spiral online where people are saying this part of coop is broken. And she's like, no, I fixed it. And they're like, well, this is a bug. And this is something that's a bug in Docker and she'll point to literally her commits and like Qube or Docker, or they'll then complain that this is a problem and a particular like Linux library, and she'll point to the code that she fixed also in Linux, and it's just amazing to see people like that who you know, can who have worked on and work on go code and everything.
Will Vincent 22:00
Didn't you just have another one? I think I just wanted to plug her because she's awesome. So I didn't she just have a tweet a couple days ago that said, maybe containers were a mistake. I saw
Jeff Triplett 22:10
that probably because of that overuse. And I guess I think why I was thinking of Jessie is because she has this awesome GitHub repo of Docker files. And so she runs chrome and Docker on her laptop. And so everything she runs is is Docker based, which is amazing. Wow. I like Docker. My clients are kind of mixed on it. But what we can do with containerization is just absolutely amazing. And I don't mean me as like a consultant, I just mean technology wise, like when I compile a Docker container, when I make an image, I know that that's kind of a contract that it's going to run if I give it the right environment, environment variables and link it to the right, you know, context. So like, if I know this thing needs Postgres and it needs Redis as long as I plugged the right values in and tell it where to go, it's gonna run and I've already tested it, so I know how it's gonna perform. When you start using like a Some of the container services and I don't want to plug in the particular one because they're all great and crap at the same time. But it's really powerful to say like, I can start this running on one node. And if I need 100 of them, I can spin that up in a couple of minutes. And that's amazing, right? And you can also it's more, you can take it from one provider to another much more easily, because it is just containers. It's also a lot easier, I think, working with other developers, because it's no longer Oh, well, you know, use use homebrew but you're on Windows or you're on Linux. So you need to figure something else out. Like once I have a Docker compose set up, and I run tests on it. I know as long as you have Docker installed on your machine, it's gonna run. And that can also run on like GitHub or get lab, I guess, get lab since GitHub doesn't have a test runner thing yet, but, you know, I can I can use ci tools to reliably run and test it. And so that, to me is pretty great.
Will Vincent 23:54
Yeah, I mean, I'm almost tempted with with true beginners who are struggling to who want to do Django to install Python. To say, well, let's just jump to Docker. But you know, I can give them a recipe that works. But then it it's a little much for them. But it does, you know, fundamentally, it is the answer, which is a reproducible environment that all team members can have. And you can almost entirely mimic production locally, too, which is pretty nice to do.
Jeff Triplett 24:20
Yeah, it definitely can be, but like, let's say you take a beginner, how are you going to tell them to deploy Django to digitalocean? Let's say, I tell.
You tell them to use Heroku. Or what's the
Will Vincent 24:32
what's the and I know you're punting to me? So let me just say that I hate deployment.
Jeff Triplett 24:39
Everybody does. That's
Will Vincent 24:41
well, so I, I think the issue is that most people, they don't have to deal with deployment. They're in a company or in an environment where someone else has figured it out or you iterate on it. You just kind of like hold your breath and get through it and then it works. It it's Yeah, it's a pain. I mean, I tell people, I usually recommend Heroku or Python anywhere just because they're pretty straightforward. And they have free tiers, you know, actually with Heroku. So in my book, which should be out by the time this is released Django for professionals, I'm finishing up the deployments chapter. And a big question I had is we use Docker in the book, do we deploy with Docker as well, because Heroku just added this pretty recently. And I think it's simpler when I show you exactly how to do it. But I don't know exactly if it's simpler than I think it is then doing it the old pro c file way. But there's just a lot of for her, okay. I mean, Heroku in particular, there's just a lot of steps, right? I mean, it'd be nice if you know, Carlton, or someone had a perfect drag and drop deployment for Django, but, you know, collect static, you got to configure static files. It's just a lot, a lot of things. It's hard to go from a basic crud app to deployable worthy project without Taking a whole ton of shorter shortcuts, allowed hosts deployment checklist. So, yeah, so that's a, that's a rambling way of saying I haven't fully figured out exactly how to handle it other than I know the two extremes. And as I become more comfortable with Django, I'm less comfortable, just sort of waving my hands at stuff and, and not covering it for people. But I am still a big believer in an iterative learning path. And so I don't want to overwhelm folks with proper deployment. So
Carlton Gibson 26:32
it's tricky, but not sure there is no, there is no really simple solution. I'm still a big fan of, you know, run a VM. And if you want to use Docker, just use Docker on the VM. And if you need to scale it, get a bigger VM for you know, to keep a lot of these things simpler. That only goes so far. And eventually you do need, you know, hundred units and then you need, you know, Kubernetes or whatever, because, you know, VMs aren't gonna go that far.
Will Vincent 26:56
Yeah, I know what what do you think? I mean, we actually just just had a whole episode. On deployments where I think we we gave some of our thoughts. I mean, basically the advice is pick something, learn it, and then you have to focus on it. That's a great problem to have. And then you can take other steps.
Jeff Triplett 27:10
Well, I think that's the only point for me is if you ever tried to like set up a, let's say Django app and not pick on Django, any Python app, or really almost anything, it is a pain when you say, PHP kind of has an advantage, because you can, in theory, copy PHP code and run it remotely on services that run PHP. But if you try to host your own, because if you have a website that needs to scale at all, you're not going to use that model. And so you're going to go and you're going to have to install I don't even know now Apache, you're going to install a bunch of libraries and a bunch of stuff and the average person's not going to be able to do it. And so one thing that I like and I think you could actually do this kind of tutorial, if you go and fire up a Digital Ocean and digitalocean is the sponsor you now I think we plugged them like four or five times but if you go to digitalocean if you installed Docker, you can actually run Run your base image on port 80. And you don't have to install really anything else. And so that to me is pretty powerful to just apt get install, Docker, run your image container, you've got to do some authentication stuff that you'd have to do anyways, even if you're doing a get checkout. So one thing I've liked is through like get lab or through Dockers registry, you can push images from your local machine to that you can run another Docker process, I use one called Watchtower, and all it does is it looks for new images, it pulls in and restarts the service that's running. That's about the most simplistic architecture I can think of for how to run something if you're rolling at your own. Now that's built upon years of knowing, like, here's how these applications and these parts put together. And that's really the challenge. It's not that this stuff is so technical, you can't teach somebody. It's also that, you know, I kind of did all the crappy parts with Linux for a long time and I don't want to do that. I also want to know what's running behind the box because I don't know that I want to hand the container to, you know, a cloud provider and trust what's going on. Behind the scenes, or even though like, if I'm paying X amount of dollars, how do I, why do I get one Docker processor? How many Docker processes do I get? And I think that it's been kind of hard for cloud providers to be able to explain and sell that to people in a way they understand that what stroke is probably done the best job of that was also difficult is the costs, you know, you don't, it's really hard to see how things scale
Carlton Gibson 29:21
and short of like putting them on there and trusting it and then finding you've got a big bill, it's, you know, which you often read, you know, I got $70,000 on Firebase, or whatever it is, like, Ah,
Will Vincent 29:33
you know, how to get all that. I would say that, that that is true, but like that is the problem you want to have. You want to have such crazy traffic that you have a huge bill, and then you it's worth the time to dive in and, and solve it. And I would add, I mean, this comes from even at scale, you know, if you can throw hardware at a problem, any problem you can throw money at. I think that's a good problem to have. I mean, you know, in a startup setting, like yeah, you need to find the money but it's so much easier to throw money. Then some of these trickier problems. So if it's Oh, we just need more machines. Okay, like, that's, that's a good problem, right? Like, you're always gonna have problems. So yeah, you don't want a $70,000 Firebase bill, but kind of got to blame yourself if somehow you rack that up, right. I mean, we just sit in there and you didn't see the analytics. You didn't see the millions of traffic.
Carlton Gibson 30:21
Yeah, I mean,
Will Vincent 30:22
I don't know. You know, I never again, I would take that I would take that problem. Because either you just don't know, either your company goes out of business, because you don't know you're doing anyways. Or you're growing so fast. You can find the money somehow. Like it's solvable.
Jeff Triplett 30:36
Yeah, you know, like going bankrupt is a is a good fix for it.
Will Vincent 30:42
Yeah, but I mean, on deployment separately, I don't know, you know, for any of these providers that are listening. I mean, I was just going through the Heroku docs yesterday, and I've done her who so many times and I was finding the container part confusing. Like I would love to have a simple Django project and just show how to get my words deployed on all day. These major services, but the challenge is that they don't have affiliate fees. So it's just totally giving it away. It takes a lot of time, and they change all the time. So it's just not worth it. But yeah, if anyone wants to, wants to sponsor sponsor the show how to deploy and you will answer me, I would love to I would love to do that. But anyway, separate moving on.
Carlton Gibson 31:27
But as I wanted to talk about your community work, Jeff, I want to talk about the work you've done. So we're
Will Vincent 31:31
gonna jump right to that. Yeah. Cool. Dockers, I want to talk a little bit more.
Carlton Gibson 31:35
Okay, go into one more note.
Will Vincent 31:37
Okay, I just I, I've seen you talk about Jeff, how you're able to evaluate the time a project will take. And I would just love any sort of stories you can tell about the mismatch between what clients think and kind of what you see and how you have that conversation where someone comes to you with some outrageous request. How do you how do you kind of talk them down from that cliff and, you know, what words do you use When they think something will happen, you know, can be done in a month and you know, it takes 12 months, right? Because that that's gonna happen all the time. It doesn't,
Jeff Triplett 32:09
strangely enough, okay, I know I started the podcast saying I don't my other
Will Vincent 32:13
consultants Brando have that all the time. So you're different?
Jeff Triplett 32:17
Well, I mean, it's just, it just is what it is. I mean, so let's say a client comes to us, and they say, We want this really awesome service. And it's something that you know, is taken years years to build. So let's say somebody comes on and they want GitHub or something, we could say, Sure, that's multi multi millions of dollars to do and you know, we're a small company, so we don't have the, you know, like the labor or the the workers to do this thing. But you can't just come in and say, copy every feature of a multi million dollar or Billion Dollar Startup realistically and put it up in two weeks, like it's just not going to happen. So it really depends on Can we just evaluate is this even doable? Part of it is if you come to us with a bad business model, and Frank doesn't more of us than I do. So I can, I can make it sound like a better story. But we do discuss that as well. If you come with us to a really bad business model, and we know that you're not going to be able to be successful at it, or it's just a bad idea, then we want to also get paid. And so if we can't get paid to do the work because of your bad business, then there's no reason for us to do it. We don't want to be in the collections business, we want to be in the right applications and you know, do what we do. So for me, it's realistically a, you have a feature or you have an MVP, so the site that we just finished that's been are we it's going through like a beta process and stuff while we're doing the brewpubs. They had a three month deadline basically, and they knew they have investors and these are the marks they need to hit. So if they say, we need to be able to sit down at a restaurant at 6pm on a Monday, which is what they did, and order a couple of beers or a cheeseburger. That's that's what we know that the goal is, so if we need to scrap other features, because The inventory management maybe doesn't matter as much, because the order has to go through. So for us, it's really a matter of prioritizing. These are the features. And when people come to us and say we want to add this other thing or an investor or somebody thinks would be really cool if we could do XYZ, it's just reminding them of the focus. And that is you, you know, this is the idea you have sold, this is what you need to do. We can do these things. But the more things that you add, the more that deadline is going to get pushed back. And so I can do that, I guess without sounding like, I don't know, I guess I'm being a consultant. As long as I have. I can tell people No, and they thank me for it were when I was probably eight or 10 years ago, they probably didn't thank me, they'd been rude or maybe I didn't know how to communicate it. So maybe eight or 10 years ago, you might not have had the authority that you have now, so that you may i don't know i'm just sort of extrapolating from my own experience, where you find yourself in scenarios where unrealistic deadline after unrealistic deadline after unrealistic deadline and you're saying that this can't be done and here's why. But it's not that that's not necessarily being listened to. And, you know, software estimation is notoriously difficult, and managers notoriously don't listen. So do you think that the later one, thinking specifically about this software estimation problem and like working with clients?
Carlton Gibson 35:21
I mean, and I guess the point I started watching from was when you said eight or 10 years of, you know, eight or 10 years ago, did you find you're able to make that point? Or were you just did you find yourself? No, not listen to that?
Jeff Triplett 35:35
Um, yes.
So eight or 10 years ago was the Lawrence journal world and we basically I was working for one of their startups that was
basically.
So we I worked on Ellington CMS, which was the first Django app or actually, I think it was technically the second Django app, but it's the biggest at the time. And so we roll this thing out to hundreds and hundreds of newspapers, and so We kind of got I got used to unrealistic deadlines. I got used to trying to do all nighters, not because they pushed us to do this thing. But it just was like self induced pressure to do it. And then at some point, I just decided I'm not doing that I'm not going to stay super late, I'm not going to do all nighters, I can tell you what I can do realistically, because I just don't want to do it, like life is too short. So if a client wants to fire me for not, which never happens, because I won't work the weekend for them, then it's not a good fit. And it takes a lot of privilege to be in that situation to be able to say that but clients tend to respect us more for that, because they don't want to be they don't want to be stressed out because we're working on a feature an hour before they're supposed to show it, especially if like their income is based on this, this demo being successful. Most of the time, when we deal with startups, that's a little more normal that, you know, they're kind of going from like funding round the funding round. That's probably not most of what we Do but, you know, I think it's kind of a trust to that. Can we deliver it? When we say we're going to deliver something we're going to do it we're going to deliver. And it's just a matter of what features Do we need to cut out? I think with working on the projects like you do you probably understand is it better to do the feature wrong or not? 100%? Or is it better just to land the feature just to say you did it with something like Django it's better to just take the time it days, it's got to be a good feature is no point there's no point putting in a broken feature. So if it takes you no longer to get in, that's fine because Django is an upstream dependency of all these, you know, end projects, but in a commercial environment, sometimes it's like no this features got it just got to ship on this day. And because the marketing department sold it to the client, before the estimation was ever done.
Will Vincent 37:54
It Okay, so I've,
Carlton Gibson 37:56
yeah, you know, it strikes me that happens all the time in software. I mean, You know, he's not great, obviously.
Jeff Triplett 38:03
I mean, I think it's just because we're not afraid or I'm not afraid to just cut out features or cut out what we need to get done is going to change based on their feedback and what else they need. So, but we're pretty upfront about it. So kind of it's a if we take a project on and we know it's gonna be three months, can we deliver this in three months? And, you know, we we can whittle that back. And and I don't know, I don't sit down with any like task tools that measure productivity, or try to figure out how to do it. When I look at models and kind of know, can I get this MVP up in a couple of weeks, and if I can get the models done, the rest, you know, the DRF stuff, it's just that the tools that we deal with are so mature now that it's pretty easy to kind of guesstimate, I guess, in my mind what it's gonna take to get something at least, you know, plausible
Carlton Gibson 38:48
and passable. Yeah. And that's the key point about estimation, right is that it's all there's this idea, this meme this trope that software timelines aren't estimated, but with mature technology that you And you've done it a lot of times and yeah, they've kind of really are because you know how it will take a day or so to put these views wrapped around this is exactly that, can you model it right. And if you've got the models, and that business layer, the views of a known commodity,
Jeff Triplett 39:13
and you know, DRF makes it so easy to like, well, there's a pattern that I've kind of grown into recently that I like, where we'll do a lot of Model View sets. And I'll basically take all the models and make an admin version of it. And that's gonna require like a staff or super user or something. And that that's easy and quick for us to do. If you're a front end person, that's not the most useful thing. If you're writing like an admin or something for like a JavaScript based admin, then, you know, it may be a pain, but technically, the job is done, you can hit everything. And then from there, that's where we go back to that, like, what's the bread and butter of your application. And we'll start writing like very particular views. So you can place an order or so you can get like a full menu with all the nested everything that you want. And then that way, we can work on optimizing those views, but by default, I do push back on front end people, sometimes they say, this isn't a very performant way of doing things. So if you just want to save an extra request, don't, I'll make it easy. So you can do batch requests, which I would love to see in DRF, by the way,
Carlton Gibson 40:14
well, there's a bunch of things we can do to make lives easier, too, that aren't always nested. Okay, so the thing I'm really hoping for with the whole async thing, the ASCII and whatnot, is that we can have a way of really simp building simple kind of proxy endpoints which hit you know, because the natural, the natural format of a REST API is flat, right. So you have one request for ingredients on for drinks, one for you know, whatever. And but the requests that front end developers want to make is one request fetching, you know, all of these and all of those and all of those and combining them. If I'm really hopeful that the async stuff will enable us to build quick proxy endpoints that will really serve front end developers really well. So I don't know if that's going to come in DRF per se, but it will be part of the Django async move I think
Jeff Triplett 41:02
that's what I'm hoping I think that'd be great. As somebody who does a little bit with Django con, I think we have a few fun surprises.
Will Vincent 41:13
So, alright, so speaking of Django cons and community, let's let's dive in to that area. And maybe we could start with. So when, when we met at pi con, you mentioned you are off to the Python board meeting, what's the titles for that? And then we'll get to the Django specific stuff. So you so you're involved with, you're sort of like the Django representative on Python right now. Um, I wouldn't say that
Jeff Triplett 41:37
because Katie McLaughlin, and if I said her name correctly, there's other Django people on the the Python Software Foundation Board. And so a year ago, pi con, I just a couple of people came to me that were directors and they said they were going to be stepping down and asked me to run. And so I thought, sure, why not. I really didn't think I would get on the board. And then I think I was on a flight From Chicago coming home and my phone blew up when I landed and apparently I'm on the board. So does that entail they were getting and I don't think any what I was getting into so it's been good. It's it's basically a lot of meetings, a lot of email. The the Python Software Foundation is the nonprofit behind the Python language. And so what they try to focus on is the education and the community in growing the community. So like, if you have events, you can hit the PSF for money to try to help and spread the word about Python. I'm still new enough to it that I don't kind of have the spiel down so I'm still trying to learn at all so hopefully none of them listen and then cringe whenever they
Will Vincent 42:42
well Katie, we already interviewed her and she'll, her interview will be coming up before yours. So that's at least one other. Awesome. So what? So education community events, are there other discussions about the language itself and features or is that hounded handled by a separate part of organization for Python.
Jeff Triplett 43:02
So I know that's changed recently. And there's a government's board. So they used to have a BD FL model, which doesn't really scale like one person just doesn't scale. And so I don't know the timeline, but that recently is changed into a council, which I guess they discussed the PI con. So the board itself has nothing to do with the actual language as far as features and stuff goes, when they're going to have meetups or they want to try to like get the all the C Python devs together to work on stuff. I think they do it like once or twice a year. Yeah, so anything we can do to aid c Python for their meetups, or their I guess it's like an in person meeting they'll do a couple times a year, at least once a year. If we can help fund that to try to get everybody together because not everybody works for a big corporation and can afford to go then really anything we can do to support the community and support that movement is good for Python. And that's good for Django and good for all the projects we'd like to work on.
Will Vincent 43:57
Yeah, well, and so just as Python has the Python Software Foundation Django has the Django Software Foundation, which you are what's I know you're part of it. Do you have a specific title as well, within that these days?
Jeff Triplett 44:09
I'm just, uh, I don't even know what I am. I think I'm just a contributing member. I don't know, Carlton, do you know what that's called? DSF member,
Carlton Gibson 44:17
I don't know if there's categories of that as a DSF. Anyone can join the DSF. By the way, if you're a Django user, part of Django community, go to the Django project comm website, look for the Django Software Foundation isn't a form you can fill in and you cannot, you know, apply to join this not it's not like there's an entry criteria other than, you know, just that you're real person, I guess. But join that join the DSF there's a mailing list for that, which you know, it's private in the only DSF members are on it, but it's not private in the sense that you can't be on it. And that's about this the Django community and you know, there's it's not particularly high traffic, so do join. I don't think there's, well then there's definite too. Yeah, definitely. Something else because it definitely is responsible for organizing Django code and writing that.
Will Vincent 45:05
And I know you've done maybe if you want to thump your chest a little bit about your involvement with Django cons, cuz I know you've done a lot.
Jeff Triplett 45:12
Yeah. So I guess I've been a principal organizer for I think this is the fifth year, six years, something like that.
Yeah, I don't even know where to start with with. How do you?
Carlton Gibson 45:23
How do you How did you so I mean, I guess you've been working with Django forever. So you How, how did the bowl of organizing Django code land in your hand,
Jeff Triplett 45:31
I went to the first Django con because I was working at Lawrence journal world, and I've kind of been the most of them, I've probably been the half or 75% of them. After, after, like, everything started off as a community model, a couple of things didn't work, I guess out very well. And so it kind of became a commercial model, which worked for a while and then that didn't work out. So I just happened to go to one where things really weren't working out at all, and so on. I just kind of volunteered to try to like make sure that, you know, there were speakers and other stuff going on. So I guess you could say I was the last minute program chair with about a week or two weeks notice. And so I just, I don't mind begging and asking people for help. So I asked a couple of people if they would speak, I think Russ spoke like three or four times at the conference. So thank you, Russ. That was amazing. And then we basically made sure that Django con happened. Russell, Keith McGee at the time was the DSF. President. He asked me if I'd be interested in maybe, you know, trying to at least get the next Django con going and even if I wasn't going to be involved with it. One of the speakers who stepped up and really helped a lot was Craig Bruce. And so he said, if you're in I'm in, we could maybe create a nonprofit and start this. Stacey Heisler who was at the time I think the DSF, maybe treasure, she volunteered as well. And so the three of us started the Django events foundation in North America. And our goal was just to make sure that Django Con us happened. And then after that things kind of were going a little better every, you know, the community got really behind it and was really, you know, helping us grow and do good stuff. And so, you know, when we were just basically trying to have a conference that you know, evened out every year, or hopefully it lose a bunch of money, we started taking the extra profits, putting some of that back into the DSF, putting those back into other Django events and Python events that we feel, you know, support the community. And so that's kind of how that ball got rolling.
Carlton Gibson 47:28
And so and one thing that's sort of really noticeable about the Django community see one about what outreach you know that to make sure that the speaker's representative and, you know, everybody can attend or you know, that it's not just not just white men in a room for one of a better phrase. So how, how did that get going or, you know, what's the, what's the birth of that and that kind of important stuff, because for me, it's one of the features of the Django community that makes it worth hanging around for
Jeff Triplett 48:00
So the first Django con us was in Santa Clara. And there were a couple of women who worked the Lawrence journal world least one woman worth that launched in the world. And then a couple of the more notable Notable people who were women in the Django community, were there. And it was really odd for me walking in and knowing four or five and maybe 10 women of 200 people. And so it was just a weird experience. It was mostly white men. And so from there, when I looked at, let's say Django con when some of the years were going pretty well, representation wasn't very good. Like we probably the first and I'll just throw out a number, maybe the first nine or 10 years of Django con, I don't know that we had a single woman speak as a keynote speaker. And so looking at that, whenever I was the program chair and started doing like my first actual running Django con. I wanted to make sure that we prioritize having a couple of women speak. And so I think by the time I had my first two keynote speakers of the three realize these are both women, which was a very intentional, and then I was looking for recommendations for another speaker. And so I said, you know, God informant, you know, got just this list of people that we could maybe pick from. And and I picked a couple of people that like I was, you know, fans of or friends of mine who I really thought a lot of because maybe they helped me and technology kind of get my start. And so when we had the opportunity, and I realized like this could send a really powerful message to have like three women keynote speaking. And so it was partially by design, and partially just kind of happenstance, I guess. And once that happened, then I think it started sending kind of a message that this is a little different than your average conference. We're going to prioritize something very different. And I think it's like the Ruth Bader Ginsburg, and I'll get the quote wrong, but it was like, How many?
I think somebody asked,
I should probably like, look this up. It's like how many women is enough to have on the Supreme Court? And I think she basically said all Yeah, like it would not be a problem because there have been all men Do that. So all you know, we didn't have any rules that like x, y, z needs to be all men or women or people of color, what we did do is kind of look at what we've done in the past and notice, and then the side that we need that we need to kind of change the way maybe talks or pick, so let's make it a community model. Let's go to a blind, you know, type process, then let's go to a double blind blind process that way, you know, when you start taking out speaker names, and you start removing the BIOS from, you know, proposals, it really changes the people at the top, so you don't always get the same people all the time. And so we do have some people who are very spectacular, and even though it's double blind, even though they don't put their bios and they're always going to be the top 10 as far as reviews go, because they're really good at what they do, I guess. So. So I think changing that emphasis was a little better. And then from there, we kind of learn from mistakes like you know, because we had that first year where we had three women It was a very valid criticism that we had three white women. And so we should you know, we have a lot of underrepresented people in our community, we can do a better job. And we've just kind of iterated on it over the years. And that's the thing is iteration isn't it? doesn't you know, it's going from here to there? No.
Carlton Gibson 51:17
We're terrible. Let's give up.
Jeff Triplett 51:19
And and I think as far as like some of the people who are doing amazing things in tech, you know, like, I talked about Jesse Zell earlier, like, if I get my pick of who I want to speak at a conference, then why would I not ask Jesse Zell to be a keynote speaker. And she's not this year, by the way. But you know, there are people that are just doing such amazing things that the people who I started following on Twitter definitely changed a lot. And so every year when we ask for keynote speakers, we just look at who we have. And like this year, I don't pick them. I haven't picked them in a few years, but I'll give recommendations on you know, maybe people that I think would be good, and we do get a lot of white men still but it just kind of comes down to who's going to give the right talk at the right time. And you mentioned async earlier, like how cool would it be? to how they Python async kind of keynote for this year. So maybe that happens.
Will Vincent 52:04
Yeah. Well, so as we come to the close of this interview, what else should we discuss? So you have I know a number of side projects, all three of us are, I guess, Jenga developers of a certain age with kids, which What should we discuss that be interesting to people?
Jeff Triplett 52:22
So I think, if I do one thing, well, I think it's driven by the idea of the campsite rule, which is leave the campsite better than you found it. And I think that works for relationships, that works for community projects. I think it works for clients. And that's the one thing that I think that the more time that you spend making the campsite better, or, you know, putting effort into making the community better. I think people who also identify with that are also going to put their efforts into trying to do that. And so if I had any, like parting wisdom, it's the follow the campsite rule. And then the kid stuff i think is great. If you want to talk about that, or
Will Vincent 53:01
Yeah, whatever you guys, let's talk about I mean, side note, we can edit it out if we don't like it, or we can include it people tend to like that.
Carlton Gibson 53:09
I mean, the thing for me with kids, it's it's just about energy levels and maintaining those and realizing that there has to be a rhythm to life that if you don't maintain it's unsustainable and when you're young, and you can perhaps have a line on the weekend or whatever. You can maltreat yourself and somehow get away with it. But actually, it has a cost which perhaps you don't notice. But once you've got youngsters because on the weekend, they get up at exactly the same time as they do every night. There is no line, there is no break, there is no day off. So if you don't care for yourself, you can't perform and you can't perform as a parent and you can't perform work and you can't keep going not just this week or this month, but all year and all next year for the next 20 years when they they're going to need you. And so for me, it's a that that's this kind of lesson. Perfect lesson I learned as a parent was about self care and the priority of self care. And it's much more important than some project deadlines, fake and all the rest of it. That was that would be where I would begin.
Will Vincent 54:10
What I would say, I mean to make an analogy to, it carries over into into projects and coding. So all the things that you know, you need to do you stay up to date with Django ad tests, you just have enough experience where you go, I'm going to prioritize that over maybe this new feature or this rush deadline, because you just know how it's gonna play out. And you've seen the record play and, you know, again, hopefully, this is wisdom. And you know, it ultimately, it just makes you more, it does make you more efficient. I think maybe the challenge is that it's can be hard to communicate that efficiency to someone else who doesn't have that knowledge. But it's certainly I mean, I feel like I'm, every year I'm a better Django developer and it has nothing to do with the number of hours I put in, but it has a lot to do with to your point. Jeff initially of knowing what to say, know about and having that framework for making decisions about, you know, personal time and then also professional time.
Jeff Triplett 55:09
I think I've learned that if I'm not gonna say yes to something that I can't put the energy in to do it. So I'm either going to be all in or I'm going to be all out like I don't I don't like the middle on that. Maybe my kid has taught me that a little bit because my kid just does not care. My kid doesn't care if I have a deadline. You know, he's happy to come to my, you know, computer he loves like Amazon echoes and Alexa's. So he will point and say, boss, and that's really all he cares about is he wants to hear wills on the bus. He just doesn't care about deadlines. And so there's just something about that, that I definitely and I guess I'm the only person here which is a kid. You all have multiple kids, which is just I don't even know how you do it one at a time. The energy stuff you're absolutely right about and I think it's because my kid doesn't care if I'm tired. Let's go outside. Let's do these. things and so that's that's kind of nice, actually. But
Will Vincent 56:04
yeah, it's a good slap across the face of perspective on a 24 seven basis, which, well,
Jeff Triplett 56:11
I think it's helped me too, because I know there's this idea of like flow, and you know, you're in flow and, you know, kids just again, they don't care about flow. And so I've had the kind of learn that I need to be more concise. And if I do, like, if he has taken a nap, maybe that's an hour, maybe that's two hours. But it definitely changes that I've got two hours, whether I'm going to do something around the house or laundry or whatever, I definitely approach it very differently. And to me, it's not an art what we do, it's, it's a job. So I've don't really worry about like writer's block, or coding block or plumbers block, you know, there's, it's just, I've got this amount of time, I'm either going to get this done or I'm not going to get this done. And so I kind of pick and choose what I'm going to work on, if I have the luxury of working on something based on what I think I have available in that time. So I think That's the one last thing about having a kid that was amazing for me is that, you know, flow was just non existent when you're a parent, I felt like,
Will Vincent 57:08
Yeah, well, you're just so deliberate and focused, when you have the chance to be that you can't entertain thoughts of. I mean, it's sort of like, sometimes you'll see people on YouTube with their morning routines and their evening routines, and, you know, people will fill the space that they have, but kids fill the space for you. So when you do have those gaps, you say, right, I'm, you know, I can get a lot done in 20 minutes, actually, on this coding thing if I have to, as well like,
Jeff Triplett 57:35
Yeah, and it's just a
Carlton Gibson 57:37
low, sorry, not as well, the thing, if you spend time with your kid, you call you know, you're spending time to get your right brain, your creative, your brain is busy working on the problem that you couldn't solve anyway. And you come back to the computer later on, and you've got that 20 minutes and it's like, yeah, there's a solution for you. And that was kind of free work in a way because you didn't have to sit at the kitchen. pewter laboring for it. You were playing with your kid. And it's like, Ah, you know, the life pays you back for being present. And it's like, Thank you.
Will Vincent 58:10
Well, I think maybe we could end on that one, Carlton. Well said,
Carlton Gibson 58:14
Jeff, thank you. That's awesome. Thanks for coming on the show. And thank you all for listening as ever. If you've got any questions, you can find us or as chat Django on Twitter, and we're at Django chat talk calm. All right. Bye bye bye.