In this episode of the groCTO Originals podcast, host Kovid Batra engages with Vilas, an accomplished engineering leader with significant experience at companies like Walmart, Netflix, and Bill.com.
Vilas discusses the concept of Developer Experience (DevEx) and how it extends beyond simply providing tools. Vilas highlights the importance of enabling developers with frictionless processes and addresses the multidimensional challenges involved. The conversation delves into Vilas’s journey in DevEx, insights from designing platforms and enabling developer productivity, and the necessity of engaging with key opinion leaders for successful adoption. Vilas shares personal anecdotes and learning experiences, stressing the significance of treating developer enablement as a product and encouraging collaboration.
The discussion concludes with advice for those stepping into DevEx roles, underlining the evolving significance of this field in the industry.
Kovid Batra: Hi everyone, this is Kovid, back with a new episode of groCTO podcast. Today with us, we have a very special guest. He’s an accomplished engineering leader, has been building successful teams from last 15 years at Walmart, Netflix, Bill.com, and with his expertise in DevEx and Dev productivity, he’s now very well renowned. So we found Vilas through LinkedIn and, uh, his posts around DevEx and Dev Productivity, and I just like started resonating with it. So, uh, welcome to the show, Vilas, great to have you here.
Vilas Veeraraghavan: Thanks Kovid. I am grateful for getting to meet people like yourself who are interested in this topic and want to talk about it. Um, so yeah, I’m looking forward to having a discussion.
Kovid Batra: Perfect. Perfect. But Vilas, before we get started, um, this is a ritual for groCTO podcast.
Vilas Veeraraghavan: Okay.
Kovid Batra: Uh, we will have to like, uh, know you a little more beyond what LinkedIn tells about you. So tell us about yourself, like your hobbies, how do you unwind your day? Something from your childhood memories that tells who Vilas today is. So, yeah.
Vilas Veeraraghavan: Okay. Okay. That’s, I was not prepared for it, but I’ll, I’ll share it anyway. Um, so I am a, the thing that most people don’t know about me, uh, is that I am a big movie fan. Like I watch movies of all languages, all kinds, and I pride myself on knowing, uh, most of the details around why the movie was made. Um, like, you know, I really want to get into those details. Like I want to get the inspiration of behind the movie. It’s almost like appreciating art. You want to get into like, why did this person do this? Uh, so I’m very passionate about that. Um, so that’s, that’s something that people don’t necessarily know. Um, and apart from that, like, I, I enjoy, uh, running and walking. It sounds weird to say I enjoy walking, but I genuinely do that. Like that’s my, that’s the place where I do most of my thinking, analysing, all of that.
Kovid Batra: Perfect. Which one’s the weirdest movie that you have watched and like found out certain details which were like very surprising for you as well?
Vilas Veeraraghavan: I don’t know if I would say weird, but you know, all of, every director, every film director has one movie that, you know, they have always yearned to make. So they, their entire career goes in sort of trying to get to that movie, right? Because it’s their magnum opus, right? That’s the, that’s the term that people use. Um, I always find that fascinating. So I always try to look for, for every director, what was their magnum opus, right? Uh, so for example, for Raj Kapoor, it was Mera Naam Joker, and that was his magnum opus. Like what went into really making that film? Why did he make it? Like what? And you’ll realize also that their vision, the director’s vision is actually very, um, pure in those, in a sense that they will not listen to anyone else. They will not edit it short. They will not cut off songs or scenes. It’s such a, uh, important thing for them that they will deliver it. So I always chase that. That’s the story I chase.
Kovid Batra: Got it. Perfect. I think that was a very quick, interesting intro about yourself. Good to know that you are a movie buff. And now like, let’s, let’s move on to the main section. So just for the audience, they know, uh, we’re going to talk about DevEx, dev productivity, which is Vilas’s main area of expertise. And his, his quote from my last discussion with him was that DevEx is not just, uh, some tools being brought in, some dev productivity tools being brought in. So I think with that note, uh, let’s get started, Vilas.
Vilas Veeraraghavan: Sure.
Kovid Batra: What according to you defines DevEx? Like let’s start with that first basic question. What is DevEx for you?
Vilas Veeraraghavan: Okay. So before I jump into that, I want to give you, give the context behind that statement I said, right? Um, it’s not about throwing tools at someone and expecting that things will get better. Um, I learnt that over time, right? I was a big fan of automation and creating tools to help people, and I would often be surprised by why people are not using them the way I thought they should. And then I realized it’s about the fact that their process that they are following today does not allow them to include this. There is too much friction that brings that. If they bring in a new tool, it’s too much friction. And then I realized also what the people, about management, all of that stuff. So it’s a very, it’s a, it’s a multidimensional problem. And so that, I just want to set that context because that’s how I defined DevEx, right? DevEx or I, as I like to call it more about dev enablement, is about making sure that your developers have the best possible path through which they can deliver features to production. Right? And so it’s, it’s not about productivity. I think productivity is inherent in the fact that if you enable someone, uh, you are providing them with the shortest paved road kind of thing to get to their destination. They will become productive. Uh, it’s sort of, uh, automatic extrapolation, if you will, from that. So that’s the reason why I, that’s how I defined DevEx. Um, but it’s important because that’s how, that was my journey to learn as well.
Kovid Batra: So I think, uh, before the discussion started, we were talking about how you got into this role and how DevEx came into play. So I think, uh, let audience also hear it from you. Like, we know like DevEx is a very new term. Uh, this is something that has been introduced very lately, but back in the day, when you started working on things, what defined DevEx at that time and how you got involved in it?
Vilas Veeraraghavan: Um, so back in the day when I started working in a software organization, the thing that drew me to, uh, what we would call ‘platform’ back then was the fact that there were a lot of opportunities to see quick wins from doing improvements for other teams. So for example, if I created something, if I improved something at the platform layer, it will not benefit one team. It will benefit all teams, multiple teams. And so the, the impact is actually pretty widespread and it’s immediate. You can see the, um, the joy of making someone happy. Like someone will come to you and say, “Oh, I was spending so much time and now I don’t have to do this.” Uh, so that drew me in, it wasn’t called DevEx. It wasn’t even called Dev Productivity at that time. Um, but this is I’m talking about like 2008, 2007–2008 timeframe. But then what happened over time was that, um, I realized that automation and creating the tools and all of that, uh, I realized how much of a superpower that can be for a company to have, uh, investment in that because it’s a multifold impact on how quickly people can get features. So how quickly you innovate, how efficient your engineering team is, how, um, excellent the, uh, how it says, the practices are within the engineering organization. They can all be defined by providing your engineers something that is, they can use every day and they don’t have to think and reinvent new ways and they don’t have to relitigate the same problem again and again.
Um, so that drew me in. Uh, so over time I’ve seen it evolve from just platform or like there used to be common libraries that people would write, which other companies, other teams would, uh, ingest and then they would release, uh, and we did not have, uh, continuous delivery. Uh, funnily enough, uh, we used to ship CDs, compact discs for those who are new to this process. Uh, so we would actually ship physical media over. So we would burn all the software on it and then we would ship it, um, to the data center and an admin would install it. So there was no concept of that level of continuous delivery, but we did have CI, and we did have a sense of automation within the actual pipeline, the software delivery pipeline. That is still valid.
Kovid Batra: There is one interesting question, like, uh, this is something that I have also felt, uh, coming from an engineering background. People usually don’t have, uh, an interest towards moving into platform teams, DevOps kind of things, right? You say that you are passionate about it. So I just want to hear it from you, like what drives that passion? Like you just mentioned that there is an impact that you’re creating with all the teams who are working there. Um, so is, is that the key thing or is it something else that is driving that passion?
Vilas Veeraraghavan: I mean, I feel like that is the key thing because I, I derive a lot of joy out of that, because I feel that when you make a change and sometimes, uh, the result, the impact of that change is not visible till it’s actually live and then people use it. I mean, for example, if you wanted to, let’s say you’re moving from a GitLab pipeline to, uh, using Argo CD for something or something like that. You’re doing a massive migration. It can be very troubling to look at it when you’re stepping back and looking at it as a big picture. But then when all of the change is done and you see how it has impacted, uh, you see how fast you’re running or you see, something like that, right? So I think it’s that, um, obviously is, which is a big motivator, but here’s the other thing, right? I think, uh, and this is a secret that I hope others also, uh, realize that it was right there all along. They just haven’t seen it. The secret is that by being in a space like DevEx, you actually solve multiple different domain, uh, domain areas, problems, right? So for example, at Walmart, I got deeply, I had a chance to deeply understand supply chain issues, like supply chain teams had issues that were different from maybe, uh, like teams that were doing more payment management. Uh, the problems are different, but when you look at the problem, uh, you have to understand deeply what that technology is. So you end up having a lot of really broad knowledge across multiple domain areas. And when you solve a problem for a domain area, you will be surprised to know, Oh, this actually solves it for five other areas as well. Right? So it’s, it’s a fascinating thing that I think people don’t realize immediately. So it feels less glamorous than something else, um, like a feature team maybe. Um, but in fact, it’s actually, in my opinion, uh, more powerful.
Kovid Batra: Got it. Is this the effect of working with large organizations particularly? Like, uh..
Vilas Veeraraghavan: It’s possible.
Kovid Batra: I’m not making any assumptions here but I’m just asking a question.
Vilas Veeraraghavan: Yeah. It’s possible.
Kovid Batra: Okay.
Vilas Veeraraghavan: Yeah, it is. I, I, yes. Uh, I, I will say that there is definitely a privilege that I’m, I should call out here, is that the privilege for me was to work, uh, in companies which allowed me the ability to like learn this, right? There was a lot of, um, bandwidth that was offered to me to learn all of this. Um, and Netflix was, is, is always good about a lot of transparency across organizations. Uh, so as an engineer, if you are working for a company like Netflix, you absorb a lot of information. And because you, if you’re curious, you can do more, you can do a lot, right? Um, obviously Walmart, fortune one, big, biggest company I’ve ever worked for. I think it’s, it is the biggest company in terms of size as well. Um, again, right, you have the ability to learn, uh, and you work your way out of ambiguity by defining structure yourself. Um, so same thing happens. I think I’ve been lucky in that way as well, um, to learn from all of these folks who worked there and obviously, amazing, talented people work in these places. So something, you keep hearing about it, you keep learning about it and then it makes you better as an engineer as well.
Kovid Batra: Makes sense. So, um, let’s, let’s deep dive into some of these situations where you applied your great brains around designing the platform teams, defining things for, uh, these platforms. So maybe, can you just bring up some examples from your journey at Netflix or Walmart or Bill.com where you had a great challenge in front of you? Uh, and what were the decision-making framework, uh, frameworks you, you, uh, basically deployed at that point of time and how things spanned out during the journey? So this might be a long question, but like, uh, I just wanted to, uh, dive into any one of those journeys if you, if you’re okay.
Vilas Veeraraghavan: Okay. I think we have had in the past, you’ve had Bryan Finster. So this was something that we traversed together along with many other people. Uh, we were all part of the same team, um, when we did this. Uh, so I’ll start with Walmart, uh, as an example. Um, I’ll, I’ll keep, keep it to sort of, I’ll go into generics and not give you specifics, but the challenge, uh, at a company like Walmart is that as a company, a big company, there is a lot of established practices, uh, a lot of established processes, established tools that teams use and businesses rely on, right? So each of these areas within the company is a business by itself. Uh, they are obviously wanting to get the best possible output for their customers. Uh, and they rely on a bunch of processes, tools, people, all of that, right? Um, if you now, going in, say that, “Hey, I’m going to introduce something that’s brand new.” Or if you’re going to change something drastically, you are creating unnecessary churn and unnecessary friction within the system, right? So in order for us to think about how we wanted to do dev enablement within Walmart, it is important to understand that you had to address the friction, right? If you are providing a solution that is replacing existing solution and doing just enough, that’s not going to cut it. It has to be a sea change. It has to be something that significantly changes how the company does software delivery, right? Uh, and so, one thing I’ll say is that I was very lucky to work for someone and for like leaders at Walmart that also understood this at that time. Um, so, for all those who are in the process right now, you cannot do it unless your leadership has that, you have buy in from that leadership, you have sponsorship from your executive teams. Uh, that helped us a lot.
Now, once you have buy in, you still have to produce something that is of value, right? And so that is where I’m saying this thing is important. So initially, uh, in my mind, uh, naively, my expectation was we build some amazing tools, right? And then we provide that to these teams and of course, they’ll be super happy, uh, the word of month will spread and that’s it. Right. All done. Um, what I found was in order to solve a problem where engineers were spending a lot of time doing toil, right? Like they were doing a lot of manual processes or repeated, uh, work throwing a tool at them was actually exacerbating the cognitive load problem, right?
Kovid Batra: Yeah.
Vilas Veeraraghavan: So now, while they maintain existing solutions, they have to now learn something new, migrate it, then convince their leaders and their teams to say, “Yeah, this is how we have to do things.” And then move forward. So you’re making that problem worse, that bandwidth problem, which is I’m a developer. I have certain amount of time to spend on feature delivery. I don’t have time for everything. So now I’m squeezing this into my, like 20 percent time, on my own free time outside of work to learn what this new thing is about. What that meant is that adoption would not succeed. So if adoption doesn’t succeed, then obviously, if your customers are not using you, you’re not, you’re a failed product, right? So what we realized was there are two other aspects to it that we had not thought about. One was process and the other one was people, right? So when I say people, I mean it could be management, it could be a key opinion leader within the space, right? That’s what we attacked. And you can obviously ask Bryan more about it. He is, he’s, he knows all about it. But the way that we attacked it was we created programs which were more grassroots, like more bottoms up view of saying, “Hey, we are starting to use these new tools. Come join us as we learn together. Let’s discuss what problems we have. Let’s talk about successes that we have. Let’s talk about how we want to do this well.” And we were open to feedback. So, inside my organization, uh, which is the dev enablement area, there was also a product organization. Uh, so we had product owners with each of the teams that are building these tools and the product owners had a pulse on the customer’s need.
So that is, that is how we found success over time. We did not obviously succeed at the start, and there was obviously, a lot of challenges we had to work through, but what happened is adoption only kicked up when we saw that we were able to, one, provide a solution that is X times better than where we were, right? So if you were to, if you were maintaining configuration, if you’re meeting five config, uh, different configs, now we just have to meet in one YAML file and that’s checked into GitHub or something like that, right? That’s a big difference productivity-wise. lesser errors. Uh, second thing is how many times do I have to look at the build? Uh, and then security review after the build and all that. So you say, okay, let’s do security scanning before the build. Uh, so even before you build a binary, you know if it’s safe to build it based on your code scan. Uh, things like that we did to improve the process itself. And then we educated our teams about it. All of our teams. We upskilled them. We gave them a chance to upskill themselves by giving them lots to, lots of references. We showed them like what the industry standards are. By showing them what the industry standards are, you created a need inside them say, “Hey, we need to be like that, right? Like, why can’t we do this?” And so that essentially became a motivating factor for most teams and most managers and directors and VPs started saying, “Hey, I want all of my teams to do exactly that.” Right. We need to be that kind of a team. And that introduced a lot of sort of gamification, right? Because when we, when you look at dashboards that look slick, right, and you’re like, “Hey, why can’t I do this? Why can’t my team do this?” It created a very natural tension, a very natural competition within the company, which served adoption well. Once the adoption was starting to grow and beyond a certain threshold, it became a very natural, or we didn’t have to go asking for customers, customers came looking for us. And so, that’s how we got to the point where there was more uniformity in how software is delivered.
Kovid Batra: Perfect. So I think it’s more around defining the right problem for the teams that you’re going to work with, defining a priority on those problems, how you were like very swiftly slide into their existing system so that the adoption is not a barrier in the first place itself. So the basic principles of how you bring in a product into the market. Similarly, you just have to..
Vilas Veeraraghavan: It is the exact same.
Kovid Batra: Yeah.
Vilas Veeraraghavan: Uh, platform, dev enablement, tooling, all of this. These are all products. Your developers are your customers. If your customers are not happy and they don’t use you, um, yeah, you are a failed organization then. That’s how it is. Right. So if you, if you feel like, um, just because you are part of a DevEx team, uh, what you say has to be the law of the land, it doesn’t work that way, right? The customers vote with their, with the time that they give you. Uh, and if that, if you find if, let’s say in an organization, you see that there are some tools that’s been released by the developer productivity or DevEx or enablement or platform engineering organization, but most people are using workarounds to do something. Then I hope the teams understand that there needs to be some serious change in the DevEx organization.
Kovid Batra: Cool. I’ll just go back to the first point itself from where you start. Is there any specific way to identify which teams are dealing with the most impactful problems right now and then you go about tackling that? Or it’s more like you are talking to a lot of engineering leaders around you and then you just think that, “Okay, this is something that we can easily solve and it seems impactful. Let’s pick this up.” How does that work?
Vilas Veeraraghavan: That’s actually a very, um, important thing to think about. And thanks for reminding me of that because I did ignore to say that. I didn’t say this the last time. Uh, you do need some champions and that’s why I said key opinion leaders, right? In the company, you need champions who can help do that early adoption and then find success. That comes from not just impact, which means, let’s say that someone is doing, uh, a hundred million dollars of business every year. Uh, and if they change something that they made to save a significant amount of money, that can be big impact, but it’s also about what their ambition is. So if I am a hundred million dollar business, but my ambition is I want to be a hundred million dollar business next year as well. They may not be able to be the, uh, they may not be the person who’s pushing at the boundaries, right?
Kovid Batra: Got it.
Vilas Veeraraghavan: They may be saying, “Oh yeah, it’s fine. I mean, everything is working just fine. I don’t want to break anything. I don’t want to touch anything. I don’t want to innovate. Let’s keep going.” But on the other hand, you will see, and this is common in many big companies is there’ll always be pockets of rapid innovation, right? And so, these folks who are in that space and their decision makers in those spaces, uh, them having a discussion with it, a really deep discussion, a very open discussion with them, uh, almost like a partnership, right? Saying, “Hey, I’m building this tool. Let’s imagine you have to use this tool. What would you want me to change in this so that it fits you?” And obviously, you’re going to take all of their input and decide which ones will be more useful to others as well. You’re not going to obviously, build something for just one team, but at the same time you get to know, like, you know, what is it that, what is it that is not getting them to adopt this right now? So you do need a set of those key opinion leaders very early in the process because they are also not just going to influence their team; they are going to influence other teams. And that’s how the word of mouth is going to spread. So that’s the first step. So it’s not just impact; its impact with ambition, which is where..
Kovid Batra: There should be some inherent motivation there to actually work on it, only then..
Vilas Veeraraghavan: I will, I will say one other thing, Kovid. Like if there is someone that, if there’s a team that doesn’t necessarily have ambition, but it’s doing more of a top-down, like get this done, right? I have often found that, uh, by leaders saying, get this done, it can sometimes backfire because the team feels like it’s an imposition on them. They may be very happy with their current state of tools, but it’s an imposition. Like now, why do you have to change this? Everything works just fine, right? You always have that inertia, like people, everyone doesn’t want change, and sometimes change might not be needed either. You might actually already be efficient, right? But that top-down approach doesn’t always work, which is why for us, I will say this, that for me, the greatest learning was how and seeing how much the bottoms-up approach worked at Walmart was actually very encouraging because I realized that you have to convince an engineer to see this for themselves. So it is very, that’s why I think opinion leaders are not necessarily VPs or they could be, it could be someone who’s well-respected in an area. It could be someone who is, um, like a distinguished engineer, uh, right, whose word carries a lot of value within an organization. Those are the, those are the people who, who tend to be those key opinion leaders, right? Uh, so top-down also doesn’t work. You can’t just be like, uh, your VP is ambitious, but you are not. That, that, that doesn’t work either.
Kovid Batra: Makes sense. Makes sense. All right. So I think when you have defined the team priority problem that you need to solve, then you start hustling, start building, of course, that phase has to be of a lot of to and fro, patience, transition, MVPs. Anything from that phase of implementation that came out to be a great learning for you that you would like to share?
Vilas Veeraraghavan: I’m thinking there was obviously a lot of learning. Uh, we, it was not, it is never a straight path, right, uh, when, when you’re doing something like this. But I think one thing that I, uh, evolved, uh, during that time was at the start, uh, I was definitely operating in a bit of a, “But this is the best way to do it.” Like I was, we were so convinced that there is no other way, but this to do it. That, uh, slight arrogance sometimes leads you down a path where you’re not listening to what people are saying, right? If people are saying, “Hey, I’m facing this pain.” And you’re hearing that across different organizations, different areas, and you dismiss it as, “Oh, it’s just a small thing. Don’t worry about it.” Right? That small thing can snowball into a very big problem that you cannot avoid, eventually. What I learned over time was I used to go into meetings being very defensive about what we already created and what, because the way I would look at it is, “Oh, well, that team can do it. Why can’t you?” And, uh, that was very naive at that time. But then I realized, uh, one of those meetings I went to, I, for some reason, I basically said, “Okay, fine. Tell me exactly how you would have solved the problem.” Maybe I was annoyed. I don’t know what, but I said, “Okay, how would you solve the problem if you were doing this?” And that person was so happy to hear that. And that person actually sat down with me for the next two hours and designed exactly how things could have been better, all of that. Like they, and I went, I was happy to go into detail, but it made me realize these are actually all allies that I should be adding to my list, right, as opposed to saying, “No, no, you have to use this. Like, what? Go away.” I, I, that was a big mistake I did. I probably did that for like six months. I, I will say that that was a bad idea. Uh, don’t do it. Uh, but after that it was, I, I was able to, the team was able to flourish because everyone saw us as partners in this thing, right?
So then we would go and we would say, “Okay, fine. You have this tool that we built, but don’t think about that. Think about what is the ideal tool that you need and let’s find out how much of this, this satisfies, right. And then whatever it doesn’t, we will accept that as feedback. And then we’ll go back and we’ll see and think about it and all that. And we will share with you what our priorities are. You tell us if this is making sense to you or not, and then we’ll keep this communication going.” That is a big evolution.
Kovid Batra: I totally relate to that. But I haven’t been like being back and forth on this thought of bringing in opinions and then taking a decision rather than just taking a decision and then like pushing it. I think it’s the matter of the kind of people you’re working with. You have to make a wise choice that whom you want to listen to and whom you don’t want to. Both things can backfire. I’ve actually experience both, uh, the same happened.
Vilas Veeraraghavan: Oh yeah. You don’t want to. Yeah, obviously, what, it goes without saying that there is gonna be some people who are, uh, giving you the right advice, right? And some people are just complaining because they are complaining. That’s it.
Kovid Batra: Yeah.
Vilas Veeraraghavan: Right? Uh, oh yeah, you have to separate that. But I’m saying there’s two ways to do this, right? Like when you, when you find that initial adoption starts hitting and all that, you can’t go into your shell and be like, “Okay, that’s it. My job is done. People will keep.” So that is what we, I felt like over a brief amount of time, right? When we said, “No, it’s all working just fine. Like, why do you, what are you complaining about?” And then I realized, I don’t know if maybe other folks in my team realized it earlier, but I realized it as a strategy. We needed to change that. And that put a very different face on our team because our team then started getting welcomed into meetings, which we originally were never a part of. It allowed us to see, uh, into their decision process because they were like, “Oh no, it’s important for you to know this because there is a lot of dependency on tools. We can’t change this process, but maybe we can adjust the tools and the settings to help us with this.” Right? So it was a very different perspective. And that learning, I was able to carry it into like other, uh, other initiatives, projects, companies, all of that. It has definitely served me well. Even now, if I’m listening to someone, I’ll usually say, “What would you do if you were in this space?” Right. And then let’s talk about it. Right. Very open. Um, but it is, it is important to have ego outside.
Kovid Batra: Yeah, totally. So I think it’s a very good point you just mentioned, like, uh, taking that constant feedback in some or the other form. But when you’re dealing with large teams, large systems, uh, I have got a sense that you need to have a system in place along with 1-on-1s and discussions with the people. So I’m sure you are focusing on making the delivery, uh, more efficient, faster, the quality should be better, less of failures, right? At the beginning of a journey, let’s say, any project, there must be something, some metrics that you define that, “Okay, this is what the current scenario is. And during the phase, these are our KPIs which we need to like look at every time, every 15 days or 30 days.” And then finally, when you are putting an accomplishment mark to your change that you have brought in, there is a goal that you must be hitting, right? So during this whole journey, what were your benchmarks? What were your ways of evaluating that system data? So that you are always able to like, most of the time it’s like, it’s for our own benefit. Like we know things are working or not. And at the same time you’re working with so many teams, so many stakeholders, you have some factual things in front of you saying, “Okay, this is what has changed.”
Vilas Veeraraghavan: Sure. Um, I’ll say this, um, we, the team used to do regular road shows, which means we would go around to different teams. We would have weekly and monthly meetings where we would showcase what’s coming, what’s happened, how this is a fit for, and we would try to always do something where you would demo this with the team that you’re talking to. We will demo it with something that they are doing, right, saying, “Hey, look, this is a build that you wanted to run. You want it to slow down all that. So you wanted it to speed up and it’s slow right now. This is how much we sped it up and all that.” So that is a roadshow thing. The reason I’m mentioning that is because that brings me to the metrics, right? Metrics, when we started, um, in the sense of day-to-day metrics, um, evolved over time, uh, till like, when I left, right? In the sense that at the very start, our metric was adoption, obviously, when we started creating the tool and sending it out. So for us, for us, it was an option. The mission statement for us was we wanted to get code into production in less than 60 minutes. So this was, when I say ‘code to production’, it is not just any code. It’s code that is tested. So, uh, which means we, we had to build it fast. We had to run unit tests. We had to run integration tests. We’ve also, uh, intended to run performance evaluation, performance testing, right? And then deploy it without having to go trouble the, the, the team again for details, right? Deploy it or, or at least make it ready to deploy. And then you obviously, have some gate that will say, “Okay, ready to deploy. Check.” Someone checks it and then it goes to product, right? We wanted this process to take 60 minutes or less. So that was the very mission statement kind of thing.
Kovid Batra: Got it, got it, yeah.
Vilas Veeraraghavan: But the metrics evolved over time. So initially, it was adoption, like how many people are using this tool? Um, it was about, uh, some common things, for example, um, a lot of folks within Walmart were using different code repositories, right? All of them, because they’re maintained by different parts of the organization. But because we unified those, we started checking, okay, is everything in one place? What is this amount of code that is maybe not in a secure space? Or something like that. Like that became an open thing to share. And we got a lot of partnership from our sister teams in InfoSec, in, uh, like all of these compliance areas, they started helping us a lot because they established policies that became metrics for us to measure. So just like I said, how secure is the code base? That is a great policy saying, “We need to have secure codebases that do not have high-level and medium-level vulnerabilities.” That meant we could measure those by doing code scans and saying, “Okay, we still have these many to go. We can point out exactly what teams need to do what.” And then we would slide in our tool saying, “Hey, by the way, this tool can do it for you if you just did this.” And so, immediately, it affected adoption, right? So, so that is how we started off with metrics.
Uh, but over time, uh, as we consolidated our, the space, we realized that, uh, I mean, once adoption was at like a 75, 80 percent kind of thing, we realized that we didn’t need to track it. I mean, then it’s like diminishing returns. It’ll take its time. The long tail is long. It’ll take time. Uh, at that time we switched, uh, to looking at more efficiency metrics. So which means we wanted to see how much is the scale costing us as a team. Like, are we scaling well to handle the load of builds that are coming to us, right? Are we, are the builds slowing down week over week for other teams, right? Things like that. So that is how we started seeing it because we wanted to get a sense of how much is the developer spending on things like long builds. So if you’re spent, if you’re like, “Oh, I start this build and I have to go away for an hour and come back.” It is a serious loss of productivity for that person. The context switch penalty is high, right? And when you come back, you’re like, “I forgot what I was even doing.” So we wanted to minimize that. So it became about efficiency metrics and that led to the goals and the strategy that we had to decide for the next year. Okay, we need to fix this one next time. So it was an adoption as much as saying, “Okay, make sure that we are still continuing on the, uh, what is the roadshows and things like that, but we’ll shift our attention to this.” So in the roadshows, we will call out those metrics. So you would start the discussion with saying, “Here is where we are right now.” There were publicly accessible dashboards, which is another thing that we believed truly as a DevEx team or a dev enablement team is every action that we take is very public. In a sense, it should be to all the organizations, public to the organization because that’s our customer, right? So we need to tell them exactly where we do, what we’re doing. The investment in money comes from these people, right? The other VPs or the execs are sponsoring this. So they need to see where their money is going. And so it was like transparency was key, and that’s why metrics were helpful. We showed them all the way from adoption to tuning to efficiency. That’s how sort of the thing went.
Kovid Batra: Cool. I think this was really, really interesting to know this whole journey, the phases that you have had. Just in the interest of time, I think we’ll have to just take a pause here, but, uh, this was amazing, amazing discussion that I’ve had with you. Would you like to share a parting advice or something for people who are maybe stepping into this role or are into this role for some time, anything you want to share with them?
Vilas Veeraraghavan: I want to, first of all, thanks, Kovid. This is, this is great. Uh, I, I really enjoyed this conversation. Um, and I also appreciate the curiosity you had, uh, to have this discussion in the first place. So, thanks for that. Um, message is simple, right? I don’t know how this happens, but DevEx never used to be cool in the past, right? In a sense that DevEx felt like one of those things that people would say, “Hey, you’re doing DevEx. You’re not necessarily releasing features.” But in reality, there were tons of features that, that the feature teams needed to deliver their features that we had to create before they did this. DevEx teams needed to be three to six months ahead of where the feature teams are so that when it comes to delivery, feature teams are not waiting on tools. We have to be giving it ready. So I believed it was cool back then, but I’m very happy to hear that DevEx is actually turning cooler because there is a lot of industry backing about it, right? Like, so there’s a lot of push, a lot of people talking about it, like yourself, uh, and we, like, we are doing right now. My only advice is, for those who are interested in it, I would suggest at least speaking to the right people so you know what the opportunities look like, right, before you say no. That’s all I ask.
Kovid Batra: Perfect. All right, that’s our time. Bye for now. But we would love to have you on another episode talking more about DevOps, DevX, dev productivity. Thanks, Vilas. Thank you for your time.
Vilas Veeraraghavan: Yeah. Thanks, Kovid. I’m happy to return anytime.
Implement DORA metrics, gain dev visibility & drive success in your teams.
Try Live Demo
