‘Can AI tools Drive Engineering Success?’ with Sayak Saha, Director of Engineering at AUTO1 Group

Kovid Batra: Hi everyone! This is Kovid, back with another episode of Beyond the Code by Typo. Today with us, we have our special guest who loves to talk on podcasts, share his knowledge with the community. He has 16+ years of engineering plus leadership experience. And he’s one of the engineering directors at AUTO1.

Welcome to the show, Sayak. Happy to have you here. 

Sayak Saha: Hey. Hi, Kovid. Thank you for hosting me and looking forward for a nice conversation with you today. 

Kovid Batra: Thank you, Sayak. Thank you so much for taking out this time and willing to share your thoughts with the community. So, before we get started, there is a quick fireside chat, where we talk to our guests about their personal space so that the audience knows them well.

So, are you ready? Can we get started? 

Sayak Saha: Yeah, sure. 

Kovid Batra: All right, perfect. So, let’s start with your favorite travel destination. 

Sayak Saha: Okay. I love to travel a lot, but especially seas because you have a chance to, to mix, play sea sport and enjoy good food. I think my favorite destination is Langkawi and to be honest, any sea beaches. 

Kovid Batra: Cool, cool. We meet another beach person then. All right. Perfect. What about your heavy days? How do you unwind yourself at the end of the day? 

Sayak Saha: Okay. So, my day starts with my daughter. So in the morning, first thing to drop her to school. Okay, so that’s how the day gets started. 

Kovid Batra: Okay. 

Sayak Saha: And after that, it’s like reading some books or news and then having a coffee and starting the day with a walk. And then I don’t know how the day just gets going. 

Kovid Batra: No, that’s cool. And what do you do usually at the end of the day? Like, let’s say it’s a hectic day. How do you unwind? 

Sayak Saha: Okay. Generally, if it’s nowadays I started doing is like in between meetings, I am always keeping 5 to 10 minutes, uh, to do a a bit of reset because that helps you to focus in your next meeting or whatever you are doing. Okay and apart from that, I love to play chess and this is something I started this year. 

Kovid Batra: Oh, cool! 

Sayak Saha: So, I try to play at least two to three games a day. 

Kovid Batra: Cool, cool, cool. That’s nice. What was your last book that you read and could you share some learnings from there? 

Sayak Saha: Okay. The last book I read a little while ago, maybe a few months. So it’s ‘Measure What Matters’ by John Doerr. He is a famous venture capitalist and engineer. And this book majorly talks about how you set your priorities and you drive things and basically setting up OKRs and KPIs. We talk about OKRs, KPIs a lot, how this translates to engineering organizations, how you measure it, how you can define your stuff, right? So, this book talks in more detail about it. And there are actually classical examples, how Google, Microsoft, Amazon, they did it for themselves. Okay. 

Kovid Batra: Oh, okay. 

Sayak Saha: And John Doerr himself, he’s a board director in this company. So he has guided that how he convinces people and how he improvises OKRs and stuff. So this is a great learning because as engineers, we often see that there are company goals that, hey, I want to increase revenue stuff and all, but how this translates to engineering orgs is something that gets lost, and this book is a great guidance for that. 

Kovid Batra: Amazing. I think we’ll talk about this in the later part of the show also. I think I have a few questions there, but let’s get started. Thanks. Thanks for sharing your personal space with us, telling us about yourself. I think that was quite good.

We can move on to the main section. And in this, we would be talking about challenges that you see with a growing team. And of course, there would be a lot of other questions that might not be related to it. But, I think what we want to learn from you is what you have experienced in your career so far.

So, probably starting off with something that is related to the topic of challenges that we come across when we grow as a dev team, right? So, what are the important pointers that one should keep in mind when they are considering to scale teams? Because let’s say you have raised your Series A, and then you already have found a first-level of product-market fit. And now you know that, okay, this is the trajectory for next one to two or three years where you want to scale the dev team. What are those important things that you as an engineering leader would take care of? 

Sayak Saha: I think this is a very difficult and easy question both way, okay? Because everyone has done it probably at our experience the same thing in their course of life, right? But I think the major challenge, right, what happens is every company has their own culture and their way of doing work, right? And probably the example which you have mentioned that when you are a small company, you might have a small engineering team who are doing a great work, right? Every member in the team is super contributing. They are taking charges and stuff, right? So, when you scale team, you think that, “Hey, I will have another super-awesome team just be there. We’ll be performing in the same pace, the same motivation.” Right? Or maybe two, three, whatever. But, when you scale teams, right, that gel or that thing doesn’t happen that way because you have to build organizational hierarchies, the communication path changes and all. 

So I think in that way, right, there are probably top 3-4 things that we look into. One is find the magic what works for your team, what is working for your current team, what is the team composition, right? I mean team composition means that composition of the developers, like how many front end developers do you have? How many back end developers do you have? Or how many product managers do you have? Do you need an engineering-focused product manager? It depends product to product, right? Okay. So find the right composition for your team. That’s first. 

The second thing is that what would be the communication model for your team, right? In small organizations, generally, it directly comes from the founders that, “Hey, I want to try this experiment. Let’s do this.” Right? When teams are small, generally the founders sit with the developers, maybe in some cubicle and start saying, “Hey, let’s do some whiteboard, blackboarding, whiteboard sessions.” And we start doing it. But when you scale teams from maybe 20 to 50, 100, doing such whiteboard sessions always is not possible. So, how you set up the communication model to flow the ideas top-down and effectively that, that matters. Okay. 

I think the third part is we scale team with some hope that I will meet some goals, right? It’s important to set up those goals and objectives and set up clear expectations before the teams that, “Hey, I expect this from you.” Otherwise, the objectives are lost at the point that we have the big team and then start saying that the priorities change. What is like the ‘must have’ vs ‘nice to have’ somehow gets diluted when, when teams grow. So setting up the right OKRs, KPIs, these things matter, right? And I think the other part is that when you are small, your teams at times are very autonomous. So, it’s very important that how your team composition and this communication and leadership is structured so that the autonomy of the team is not lost. 

Kovid Batra: Right.

Sayak Saha: It should be very hierarchical and stuff, right? And apart from this, I think the other thing comes up, right? When your teams grow, there is an inherent challenge of bottlenecks, right, with the DevOps, IT, and, and others’ things, right? And the things whether we’re, we are supporting a 20-member dev team is much easier or much easier than when you have a 100-member or 200-member team. 

Kovid Batra: Correct. 

Sayak Saha: Which asks for automation. So, which means that an early stage, if you look for all those automation scopes that, that these are the areas, these are the things I need to automate, it’s not only about the CI/CD pipelines. It can be as simple as onboarding a team member, right, onboarding a team member and creating his accounts across all the systems, right, how you can onboard him, that simple. I mean, what are the dev tools you can support, how quickly a developer can onboard, because someone cannot sit next to him and all. So, I think building that automation around it. So I think these are the major things which people learn while doing like when you are really scaling the team. Definitely there are other challenges like hiring, the internal conflicts, growth and other stuff. But I think these are the major things that I have faced. 

Kovid Batra: That makes sense. Yeah. I think one point really intrigued me, like when you’re, when you’re scaling, you just have to make sure that the processes, the communication together should be like shaped up properly when the team is scaling, right? It is very important. But, how would you define that you don’t end up overdoing something, right? So, while you’re scaling the teams, that’s not the only goal. The goal is also to deliver the products, the features, right? At that point, you also have to be cautious about that you don’t end up doing or overdoing things, right? So let’s say, how would you approach this part where you have to ensure there is automation at the right places, there is a communication channel set up in the right mode? So, can you just give us a few examples how you dealt with some of this situation, like just to get more relatability in the situation?

Sayak Saha: See, to be honest, there are different ways to do this. There are different approaches, right? And when you get in leaders, like when you scale teams, when you get in leaders, it’s always not so good that you, you force some specific processes on them, right, that maybe you follow Agile, Kanban or this metrics and all. But what I feel is that when you say this entire process, right at the end of the, what is it? Like, it’s like, there is an idea which comes. Someone needs to define that idea that what I want to expect from it, maybe some mock up, screens and all. Then you define some KPIs from it. Then there’s a tech spec, design, testing and finally get it released doing some A/B testing, right? So, in order to define that what are these phases for your company, and based on the size of the idea, maybe a short, medium, large, how much time this idea should, should spend in each of these phases. Let’s say, ideation phase can be two to three days for a small scale feature, but maybe two weeks for a big size feature, right? If overall I’m having, let’s say 20 ideas which are cooking. 

Kovid Batra: Right. 

Sayak Saha: Different phases. Each phase, it should not spend beyond this period. If it is spending more than beyond this period, then that means the process is overkilling there and this requires some optimization. 

Kovid Batra: Makes sense. Perfect. I think that’s a very relevant example. 

Sayak Saha: Because finally if you look at any CEO or any Co-founder, right, what they looked at, how fast I can make this feature live. I mean, I have an idea, how fast I can do it. So if the expectation is to make it live in, let’s say one month, it means that how much time each of the slices, windows should have. 

Kovid Batra: Yeah, the process should support it basically, and that’s how you should plan for it. So I think from this, I’ll move on to like one important point that I have always been struggling with, right? This is more around collaborating effectively in these times. So, engineering teams collaborating with design and product teams to define smooth and effective product development. And there are always these fights, back and forth communication, the requirements have not been defined properly, this PRD is not up to the mark for us to like start working on it. On that note, I’m sure you would have also faced a lot of challenges. Just give us one good example from your journey where you actually ended up solving such problems and things were working smoothly. I know that would be very idealistic to say working smoothly, but still, yeah, what was your contribution in your team to solve for this process? 

Sayak Saha: I think the first and foremost is that let us accept the fact that things will not be smooth, right? When you’re moving fast, It’s very hard to accept that I will get something really in a very neat manner and stuff. This won’t work, to be practical because each of the actors in this process, right, whether it’s designers, developers, none of them know the entire thing, right? The designers know how a good interaction can be, right? Maybe the QAs or the PMs they know the product the best, how users will use it. The developers know how to code and the challenges of the system. 

Kovid Batra: Right. 

Sayak Saha: It’s good to build but what are challenges in my system, right? So, in a way to build anything, you need skills from all of these places. So, I think fundamentally the first and foremost, I think is like the team need to have a nice bondage and they should start feeling that it’s a shared success, right? Because unless each one of them help each other, right, this thing will never be successful. Okay. So, that initial gelling and building a team bondage, I think is the first and foremost thing. There is no other recipe for, for, for it. Okay. 

The next part what I think is like for each of these things, right, when the design is built, involve everyone in these phases, let’s say involve the devs, involve the PMs, involve the QAs. Run, run these mocks with them to understand because the mock phase is very important, ’cause when you run the mocks, you understand the bottlenecks of the system, right? And, and run it as a perfect user. Maybe at times it’s, it’s, it’s okay to run it with an actual end user and ask him for feedbacks that, “Hey, this is what we’re planning to build. What do you think?” Does it make useful for him, right? And, and even it’s useful that you run the mocks with the operations people, because finally you might have a product, but when the final reconciliation or stuffs are getting handled from operation standpoint, or maybe if some complaints are coming from customer support, right? How they see this data? Maybe there are like they should have sufficient logs and all these things so that they can debug those complaints. So, involving every stakeholder while designing a product, maybe the big products I’m saying or big features, it is very important because when you identify it early, it’s always important, right?

And second is developers, you will see they always have a mentality that “Hey, I’m not comfortable delivering something every two days or three days. Let me build everything in two weeks and I will dump you, or three weeks.” They, they try to, I mean, there is a general in, I mean, tendency of people that I will not make incremental commits, but I will make a single commit. I think this is culturally one of the things if a team can develop, it’s like having incremental commits and, and see that how it works with the big picture. That solves a lot of problems. 

Kovid Batra: Yeah, I think probably 70-80% work is done when people start believing that it’s a, like wholesome success, right? It’s not one team’s success. So, the main thing is done there. It’s more about, like the teams which are working smoothly, I should say I have seen that thing, which you just mentioned, like there should be a cooperation there, there should be a feeling of success together. So, I think only then things work out. You will have that patience. You will understand other standpoints. So, things would work better that way. Makes sense. I think a pretty good advice there actually. 

Sayak Saha: And also one last thing is actually, most of the times, right, if you see that teams, they got bit they’ve shared very like stiff deadlines and stuff, right, because there is a sense of fear. I think from leadership side, if, if they can be given a comfort that, “Hey, even if you fail with the commitment or if this feature fails or if the deadline slips, it’s okay if there is a valid reason.” So, I think when teams have this comfort, they just outperformed. So.. 

Kovid Batra: Yeah. 

Sayak Saha: Comfort needs to be provided to the teams, the comfort and trust. 

Kovid Batra: Totally, totally. Makes sense. Moving on to one important topic related to how we end up implementing technologies and now, because you have this ChatGPT coming in, all the AI wave going on, right? And there are a lot of changes happening. You have suddenly so many tools around you that already tech teams were having this tool fatigue, but now you see developers using things like GitHub Copilot and people are talking about it, let’s implement it. So, how do you use these technologies and how do you implement it when you already have such legacy code implemented? What is the overall process and as an Engineering Director or as an Engineering Leader, how do you make sure that it is utilized at the right point of time and in the right spaces? 

Sayak Saha: Okay. So first of all, if you look at, uh, with the introduction of ChatGPT and Generative AI in the last 6-8 months, it has been a total disruption, right? 

Kovid Batra: Yeah. 

Sayak Saha: All these days, what we have been hearing of AI from some, from some very specialized teams, this is no more a specialized team. Now it is everyone started using, exploring and every, every morning when you open LinkedIn, you will see that someone posting that these are the top 20 AI tools to be used, right? So, there is a, like a huge blast of ideas and usage. And to be honest, no one, I think everyone is adapting. So, there is no defined strategy as of now that if you follow this strategy, this will give you success or this results you can achieve. So I think at this point, what is important is identify first of all, for your organization, what are the bottlenecks and where you think AI can optimize, okay? So, let’s say the example which you said, we all are thinking that it’s code assistant, it can be Copilot, or AWS CodeWhisperer or anything, anything else, right, that we all think that this will help developers code faster, easy and stuff. Okay, we don’t know the real benefits yet. So, I think what the best thing is to run a pilot project within the company. And end up any of these tools and see how useful it is. So let’s take an example. Let’s say for example, if you are using any of these tools ideally what these tools are doing, so they are providing you two types of course, one, they already have a learning based on the course in the internet and other repositories, and this like code on that, right? So, this is a one set of output which they give you. It can be as simple as, “Hey, give me a code, to drop a file in S3.” or “Give me a code to read a file from S3.” You get all these things. Or, or it can be any other cloud providers or, or any other use case. 

The second is what, where it’s more tricky. There are organizations which are enterprise organizations. They might have a large code base of several years, legacy code. And many companies use their own customized libraries, right? So if I need to code, I need to code using those libraries. So, can this code assistant work with this? So, many of these code assistants are saying that their LLM models can learn the internal code base and suggest code snippets according to that, right? So, the best part is that it was a pilot project. See how this works. So one of the ideas which we are trying to do is that, I would not name the vendor that they have a metric which suggests how many snippets of code are accepted by the developer, right? It will suggest, but you need to accept it, right? Accept it. Overall I have X commits happening through these things, then it’s a productivity improvement, right? And the other part is that the second measurement is that the code which we are accepting or getting accepted, how much it is custom code, means which are very internal libraries or internal to the organizational code it’s suggesting versus very generic. Can we get a metric very detailed to that level, right? So this is another thing which we are trying to measure. 

And, and also a few aspects to look into is like many of these tools, this might be very accurate, but there is a software licensing issue, right? So, because they can pick code from Apache license or some other licenses. So as an organization, do you support all this licensing stuff? What makes sense? So, you need to do that right mapping also. 

Kovid Batra: Yeah, yeah, yeah. Absolutely! 

Sayak Saha: So this is one of the things which we are trying to adopt and learn. But in general, there are a couple of low-hanging fruits, let’s say, from operation side. They’re doing some translation service or some, someone collecting feedbacks and summarizing the feedback trains. So, these are pretty classical use cases, which ChatGPT is doing wonderful, right? So in, in this case, it’s important that people learn how to write effective prompts. So this is another skill set now. 

Kovid Batra: Exactly, exactly. Right. 

Sayak Saha: So prompt engineering is another important part. So, so this is, I think this is, these are more like operations or used to stuff, but what’s important, I think was what’s very, uh, there are other use cases coming up, let’s say you have an incident management, right? So, often we have repeated issues having the same cause. Can AI solve it, right? I mean, can I have, can I make my LLM learning happen on my all Slack messages for last three years and make it learn that in future if a similar issue happens, these are the steps to be done, right? So, so we do not spend time or at times when you have an issue later when you do an RCA, based on the chat history and stuff, can the AI summarize it, so we don’t spend time in building a RCA? Okay. 

So, AI is more like a tool now. So, how we use it matters. So I think one to, to summarize, I think what’s important is that we need to, to make the engineers learn what’s available in the market, these are the things available and make them decide that what’s more effective for them, instead of building a strategy that, “Hey, I want to onboard this 4-5 tools and this is what, what I want to achieve.” There are a lot of tools. Now it’s better that you, that the teams understand what’s best for them and then start getting a result out of it, because once you see the results, it matters. And then, definitely I think one of the things which is deviating people that there’s so many tools right now, jumping on a tool and tying up with some vendors or company probably is not the right thing. First internalize. 

Kovid Batra: Yeah, we should understand the need first. Like, if there is a real problem that needs to be solved and.. Correct. 

Sayak Saha: Yeah. So, understand your problem, maybe explore a few tools, build an internal team or a pilot team, see how it works for you, and then go for a commitment in terms of money, effort, and everything.

Kovid Batra: Makes sense. Totally. Perfect. I think with this like we are talking about GitHub Copilot and there are other, other tools out there in the market. There are a lot of times, not just with AI, but even before that, there is always a question of whether to build something. Let’s say you have a problem, you want to like solve it with productization, you want to have a product for that. How do you go about making that decision of building in-house or buying it from a vendor? How do you take that decision usually? 

Sayak Saha: Okay. So, probably I will frame this question a bit more. 

Kovid Batra: Yeah, yeah. Please go ahead. 

Sayak Saha: It is like build vs buy vs run, okay? Because when you maybe buy a software intruder, you are also running the software for a longer term, right? So there is a cost in running the software, whether it’s built in-house or out. So let’s come to that point. 

So first of all, like whenever you build a product or build a tool, right, I think what’s important is that every company has its core offering. Let’s say, if there’s an, it’s an insurance company or a fintech company, that’s their core, right? But, as a general IT company, nowadays we need to do a lot of generic functionalities. Let’s say, you need a, let’s say a message provider which is responsible for sending SMS or maybe WhatsApp messages, emails and stuff, right? So, do you need to build it in-house or do you want to tie up with some company and use their services for the same? There are many such things. It can be a payment gateway, it can be a, this kind of a messaging service, email service provider and whatnot. So, I think what’s important is that it’s a fundamental strategy that, that as a tech company, I will only solve my core problems or my core business cases and rest all I will outsource, or I will tie up with something, someone else, right? Because this helps in prioritizing your focuses. Okay. So, this is one of the strategies which can help, right? 

The next point is that, okay, you strategize it. The second point is that how much cost it takes, right? At times there are solutions which can be built in-house and it’s much cheaper to build. So, it might be worthy to build it in-house. I can take a small example, like a tiny URL. So if you’re building emails and you need a tiny URL, there’s a cost. It’s a very minimal cost, but you want to build it in-house, right? I mean, the cost value proposition, like is it really worth it? Okay. 

I think the third part is, which I was saying the run. When the organization is small, if you take a software, it often won’t pinch you much. Let’s say, you buy a software for some for 50, 20 developers, 10 euro per month, it’s great. But, when your organization scales to 500 developers.. 

Kovid Batra: Then it’s going to be.. 

Sayak Saha: It starts pinching you, right? So this is something strategy that if you really have such a growth strategy and you feel that this might be a pinch, it’s a decision, how you want to plan that, okay? And this has a cherry on the top in recent years, when because of business decisions, let’s say if you need to shrink your organization, this, this gives a different perspective because you, you make a commitment that, “Hey, I’ll take your license for 500 users for next three years.” But, let’s see if your organization shrinks to 300 users, then you’ll need to still pay the same money to the external vendor, right? So how, how you plan these things, so these are the aspects which I think one should consider. 

Kovid Batra: Yeah. Makes sense, totally. And last thing that I just want to discuss with you, like when you are using these tools, bringing in automation, using AI, ultimately we are trying to impact the overall productivity of the team, right? The question that arises from here is that how you are defining that productivity and even I should like take a step back. I should ask how do you define the success of an engineering team? Because productivity, I’m sure it’s, is a very core part to it, but in general, how do you define the success of an engineering team? How do you measure productivity? How do you measure success is what I would love to hear from you. 

Sayak Saha: It is a wonderful question. So if you look at right as an engineering leader, I mean, the best part is that you start talking with the other peers in your organization or in your company, and they might not be tech. So, everyone is trying to promote the good work of their part. It can be from finance, it can be from business and everyone. So, when you are in that kind of a forum, what matters is that finally how much revenue you are adding to the company through your team’s work or how you are building in optimizations and cost-saving initiatives to save cost, right? So I think these are the, for any two teams, these are the main two factors, which, which comes out, which drives benefits.

So now, the question is that how these two things can be related to engineering teams. So, what we do is that as an engineering team, we, we develop multiple features, right, features or products and stuff. So, so for any of this, right, when you deliver a feature, we start, we actually measure the ROI of these features that, hey, this feature, if it’s built, this is going to bring so much revenue for me, for my customers. This can bring in so much new customers or so much new business. This is, this is one way.

The other can be, let’s say it’s a, it’s an experience that if I build this experience, let’s say I limit my customer journey from 10 clicks to 7 clicks, it’s a better customer experience, right? If I can reduce my number of app crashes, it’s a better customer experience. It will improve my customer NPS. And maybe in turn, this will reduce my customer churn rate. So what I said that first is definitely increasing revenue. Second is customer churn. And third is the cost savings. Like when you build softwares and stuff, there’s an operational cost of running it, running the software, which is purely an OpEx cost. I mean, which involves OpEx cost spent by the developers. It is OpEx cost of the of the hosting the software. It can be in cloud and in-house. So, what are the optimizations can be done to, to save those costs around because, I mean, because as developer teams, they, they keep on adding features, right? And if we keep on in consuming the hardware resources, this cloud cost, your cloud cost be also linear and it would happen, right? Because every organization needs to limit the cloud cost, so that we limit our cloud cost to some budget. 

Kovid Batra: Correct. 

Sayak Saha: Right. So, what are the optimizations which can be done to limit those? I mean, there can be strategic decisions that, hey, we will build services with serverless vs microservices, based on the volume and other tech factors, right? So these are the aspects which needs to be taken into. So anything when we build, I generally try to categorize into these three buckets. Right. 

And, also the third part is, I think is innovation because innovation can give you something which you cannot maybe measure upfront, maybe the code assistant one, which we discussed, right? We don’t know the direct outcome of it right away before investment, right? So we should keep some bandwidth for innovation because otherwise, you might lose the train. It depends from organization to organization, but I believe at least 10-20% efforts to be spent on innovation so that we don’t miss the bus. 

Kovid Batra: Totally. When you say these goals are kind of, I would say broad goals to understand, and of course, everyone should have a realization towards it. First question that comes to my mind is that when developers are actually working towards building the product, and product is of course related to these business metrics of customer satisfaction, revenue, saving cost. Maybe saving cost is something that they can also see in the GCP bill or the Azure bill or AWS, right? But the other two factors are kind of indirect, right? Like when you write the code, you complete your sprint. You don’t really end up looking at the goal of customer satisfaction immediately because the feature is rolled out then you will get. So, the feedback loop is first of all, little time taking and then it is kind of indirect also. So, how do you, like motivate developers or like ICs for that matter to be feeling the same as a product guy would feel or a sales guy would feel in a business? What do you usually do to do that? 

Sayak Saha: Okay. I think I got it. But I think, I think that what’s important is, right, that every individual in an organization, they have some counters and metrics to measure for their work, right? So, what we do is let’s say, in my previous company, let’s take an example. We were managing a payment gateway, right? So, in the platform, if everyone is doing payments every day, it’s all good. I mean, it’s like, you’re not making a lot of changes in the gateway every day. So, the motivation level of people comes down that, “Hey, it’s a software. It’s running as usual. What new shall we do here? Or what, what’s the scope, right?” What we started doing is that we started developing the funnel, the gateways, how many people are adding their items in the cart, then clicking in the car details and finally making a payment, and then the successful payment at the end of the day, right? It’s a funnel with the full steps, right? So, we started measuring that in this full funnel, what’s my, uh, expected daily users in each of the stages of the funnel, okay, and average it out over a month. So, and every day, if any of these counters are dropping, then there is something wrong. Let’s say, if I expect 200K people to visit my payment site, right? Okay. So, if all of a sudden I have a 10K, then there is something wrong. So we had counters, like I expect so many successful payments per day, right? And this will be my success percentage. So end of the day, this report comes in that hey, my conversion should be 80%, payments should happen successfully. So all of a sudden, if it’s 67%, that means there’s something wrong in the platform and the developers are responsible for it. Okay. And this in terms comes to the operational metrics, right? The system uptime, the API, success rate, the TPS of stuff and all those things. So, what I try to mean is that if you give a user experience metric and then let developers decide that what are those internal metrics in the tech architecture they should measure. It can be the APIs, it can be the Google events, it can be anything. But, let them take that call. Don’t tell them how to do, but tell them what to do. 

Kovid Batra: Yeah. Cool. I think that makes a lot of sense. So I think this is great. Maybe one, one last thing that I just want to talk about is how are you looking at the in-general efficiency of your engineering team members. So, defining success with these goals and OKRs and initiative OKRs, right? So, that’s I think setting the fundamentals for anyone to be motivated and deliver towards what the business is actually needing. But, do you follow any practice? Like, I should change my question, actually. Do you follow any practices or do you do the efficiency measurement using metrics like DORA or something? 

Sayak Saha: We don’t do any specific those kinds of metrics. But, what we try to do is that, generally like we were saying that, that in a quarter, we have an expectation that in a quarter, we will deliver so many features, right? So many large size, so many yes, blah, blah, blah. Right. So we try to expect that if a team is able to deliver that, okay, those main signs, so that’s more important.

And then the other part is that, as I said, that since we try to just get the developers, PMs all together, the focus is more towards the outcome. 

Kovid Batra: Outcome. Makes sense. 

Sayak Saha: Right. And I think that motivates the team more instead of forcing that, “Hey, I developed so much story points, velocity.” Because at the end of the day, if there is no outcome, nothing matters. 

Kovid Batra: Right. No, totally makes sense. Cool. I think this is amazing, Sayak. It was a lovely chat. I think I learnt a lot and probably our audience, also learnt a lot from it. So, what we can do is we can have another session with you sometime, like where we can have a part 2 to this episode and talk about more such engineering topics. It was really, really nice talking to you. 

Sayak Saha: Thank you so much, Kovid, for hosting me and these were tough questions and all these questions are very close to my heart and it was lovely sharing with you. 

Kovid Batra: Your insights were really amazing. I must say. So, thank you. Thank you. 

Sayak Saha: Have a nice evening. Yeah, bye. 

Kovid Batra: You too. All right, see you.