In this episode of the groCTO Originals podcast, host Kovid Batra talks to Venkat Rangasamy, the Director of Engineering at Oracle & an advisory member at HBR, about 'How AI is Revolutionizing Software Engineering'.
Venkat discusses his journey from a humble background to his current role and his passion for mentorship and generative AI. The main focus is on the revolutionary impact of AI on the Software Development Life Cycle (SDLC), making product development cheaper, more efficient, and of higher quality. The conversation covers the challenges of using public LLMs versus local LLMs, the evolving role of developers, and actionable advice for engineering leaders in startups navigating this transformative phase.
Kovid Batra: Hi, everyone. This is Kovid, back with another episode of the groCTO podcast. And today with us, we have a very special guest, Mr. Venkat Rangasamy. He's the Director of Engineering at Oracle. He is the advisor at HBR Advisory Council, where he's helping HBR create content on leadership and management. He comes with 18 plus years of engineering and leadership experience. It's a pleasure to have you on the show, Venkat. Welcome.
Venkat Rangasamy: Yup. Likewise. Thank you. Thanks for the opportunity to discuss on some of the hot topics what we have. I'm, I'm pleasured to be here.
Kovid Batra: Great, Venkat. So I think there is a lot to talk about, uh, what's going on in the engineering landscape. And just for the audience, uh, today's topic is around, uh, how AI is impacting the overall engineering landscape and Venkat coming from that space with an immense experience and exposure, I think there will be a lot of insights coming in from your end. Uh, but before we move on to that section, uh, I would love to know a little bit more about you. Our audience would also love to know a little bit more about you. So anything that you would like to share, uh, from your personal life, from your professional journey, any hobbies, any childhood memories that shape up who you are today, how things have changed for you. We would love to hear about you. Yeah.
Venkat Rangasamy: Yup. Um, in, in, in my humble background, I started, um, without nothing much in place, where, um, started my career and even studies, I did really, really on like, not even electricity to go through to, when we went for studies. That's how I started my study, whole schooling and everything. Then moved on to my college. Again, everything on scholarship. It's, it's like, that's where I started my career. One thing kept me motivated to go to places where, uh, different things and exploring opportunities, mentorship, right? That something is what shaped me from my school when I didn't have even, have food to eat for a day. Still, the mentorship and people who helped me is what I do today.
With that context, why I'm passionate about the generative AI and other areas where I, I connect the dots is usually we used to have mentorship where people will help you, push you, take you in the right direction where you want to be in the different challenges they put together, right? Over a period of time, the mentorship evolved. Hey, I started with a physical mentor. Hey, this is how they handhold you, right? Each and every step of the way what you do. Then when your career moves along, then that, that handholding becomes little off, like it becomes slowly, it becomes like more of like instructions. Hey, this is how you need to do, get it done, right? The more you grow, even it will be abstracted. The one piece what I miss is having the handholding mentorship, right? Even though you grow your career, in the long run, you need something to be handholding you to progress along the way as needed. I see one thing that's motivated me to be part of the generative AI and see what is going on is, it could be another mentor for you to shape your roles and responsibility, your career, how do you want to proceed, bounce your ideas and see where, where you want to go from there on the problem that you have, right? In the context of the work-related stuff.
Um, how, how you can, as a person, you can shape your career is something I'm vested, interested in people to be successful. In the long run, that's my passion to make people successful. The path that I've gone through, I just want to help people in a way to make them successful. That's my belief. I think making, pulling like 10 to 100, how many people you can pull out. The way when you grow is equally important. It's just not your growth yourself. Being part of that whole ecosystem, bring everybody around it. Everybody's career is equally important. I'm passionate about that and I'm happy to do that. And in my way, people come in. I want to make sure we grow together and and make them successful.
Kovid Batra: Yeah, I think it's, uh, it's because of your humble background and the hardships that you've seen in the early of your, uh, childhood and while growing up, uh, you, you share that passion and, uh, you want to help other folks to grow and evolve in their journeys. But, uh, the biggest problem, uh, like when, when I see, uh, with people today is they, they lack that empathy and they lack that motivation to help people. Why do you think it's there and how one can really overcome this? Because in my foundation, uh, in my fundamental beliefs, we, as humans are here to give back to the community, give back to this world, and that's the best feeling, uh, that I have also experienced in my life, uh, over the last few years. I am not sure how to instill that in people who are lacking that motivation to do so. In your experience, how do you, how do you see, how do you want to inspire people to inspire others?
Venkat Rangasamy: Yeah. No, it's, it's, it's like, um, It goes both ways, right? When you try to bring people and make them better is where you can grow yourself. And it becomes like, like last five to 10 years, the whole industry's become like really mechanics, like the expectation went so much, the breathing space. We do not have a breathing space. Hey, I want to chase my next, chase my next, chasing the next one. We leave the bottom food chain, like, hey, bring the food chain entirely with you until you see the taste of it in one product building. Bringing entire food chain to the ecosystem to bring them success is what makes your team at the end of the day. If we start seeing the value for that, people start spending more time on growing other people where they will make you successful. It's important. And that food chain, if it breaks, if it broke, or you, you kind of keep the food chain outside of your progression or growth, that's not actual growth because at one point of time, you get the roadblocks, right? At that point of time, your complete food chain is broken, right? Similar way, your career, the whole team, food chain is, it's completely broken. It's hard to bring them back, get the product launched at the time what you want to do. It's, it's, it's about building a trust, bring them up to speed, make them part of you, is what you have to do make yourself successful. Once you start seeing that in building a products, that will be the model. I think the people will follow that.
The part is you rightly pointed out empathy, right? Have some empathy, right? Career can, it can be, can, can, it can go its own progress, but don't, don't squeeze too much to make it like I want to be like, it won't happen like in a timely manner like every six months and a year. No, it takes its own course of action. Go with this and make it happen, right? There are ups and downs in careers. Don't make, don't think like every, every quarter and every year, my career should be successful. No, that's not how it works. Then, then there is no way you see failure in your career, right? That's not the way equilibrium is. If that happened, everybody becomes evil. That's not a point, right? Every, everything in the context of how do you bring, uplift people is equally important. And I think people should start focusing more on the empathy and other stuff than just bringing as an IC contributor. Then you want to be successful in your own role, be an IC contributor, then don't be a professional manager bringing your whole.. There's a chain under you who trust you and build their career on top of your growth, right? That's important. When you have that responsibility, be meaningful, how do you bring them and uplift them is equally important.
Kovid Batra: Cool. I think, uh, thanks a lot, uh, for this sweet and, uh, real intro about yourself. Uh, we got to, uh, know you a little more now. And with that, I, I'm sorry, but I was talking to you on LinkedIn, uh, from some time and I see that you have been passionately working with different startups and companies also, right, uh, in the space of AI. So, uh, With this note, I think let's move on to our main section, um, where you would, uh, be, where we would be interested in knowing, uh, what kind of, uh, ideas and thoughts, uh, are, uh, encompassing this AI landscape now, where engineering is changing on a day-in and day-out basis. So let's move on to our main section, uh, how AI is impacting or changing the engineering landscape. So, starting with your, uh, uh, advisories and your startups that you're working with, what are the latest things that are going on in the market you are associated with and how, how is technology getting impacted there?
Venkat Rangasamy: Here is, here is what the.. Git analogy, I just want to give some history background about how AI is getting mainstream and people are not quite realizing what's happening around us, right? The part is I think 2010, when we started presenting cloud computing to folks, um, in the banking industry, I used to work for a banking customer. People really laughed at it. Hey, my data will be with me. I don't think it will move any time closer to cloud or anything. It will be with, with and on from, it is not going to change, right? But, you know, over a period of time, cloud made it easy. And, and any startups that build an application don't need to set up any infrastructure or anything, because it gives an easy way to do it. Just put your card, your infrastructure is up and running in a couple of hours, right? That revolutionized a lot the way we deploy and manage our applications.
The second pivotal moment in our history is mobile apps, right? After that, you see the application dominance was with enterprise most of the time. Over a period of time, when mobile got introduced, the distribution channels became easier to reach out to end users, right? Then a lot of billion-dollar unicorns like Uber and Spotify, everything got built out. That's the second big revolution happening. After mobile, I would say there were foundations happening like big data and data analytics. There is some part of ML, it, over a period of time it happened. But revolutionizing the whole aspect of the software, like how cloud and mobile had an impact on the industry, I see AI become the next one. The reason is, um, as of now, the software are built in a way, it's traditional SDLC practice, practice set up a long time ago. What, what's happening around now is that practice is getting questioned and changed a bit in the context of how are we going to develop a software, make them cheaper, more productive and quality deliverables. We used to do it in the 90s. If you've worked during that time, right, COBOL and other things, we used to do something called extreme programming. Peer programming and extreme programming is you, you have an assistant, you sit together, write together a bunch of instructions, right? That's how you start coding and COBOL and other things to validate your procedures. The extreme programming went away. And we started doing code based, IDE based suggestions and other things for developers. But now what's happening is it's coming 360, and everything is how Generative AI is influencing the whole aspect of software industry is, is, is it's going to be impactful for each and every life cycle of the software industry.
And it's just at the initial stage, people are figuring out what to do. From my, my interaction and what I do in my free time with NJ, Generative AI to Change this SDLC process in a meaningful way, I see there will be a profound impact on what we do in a software as software developers. From gathering requirements until deploying, deploying that software into customers and post support into a lifecycle will have a meaningful impact, impact. What does that mean? It'll have cheaper product development, quality deliverables. and having good customer service. What does it bring in over a period of time? It'll be a trade off, but that's where I think it's heading at this point of time. Some folks have started realizing, injecting their SDLC process into generative AI in some shape and form to make them better.
We can go in detail of like how each phases will look like, but that's, that's what I see from industry point of view, how folks are approaching generative AI. There is, there is, it's very conservative. I understand because that's how we started with cloud and other areas, but it's going to be mainstream, but it's going to be like, each and every aspect of it will be relooked and the chain management point of view in a couple of years, the way we see an SDLC will be quite different than what we have today. That's my, my, my belief and what I see in the industry. That's how it's getting there. Yep. Especially the software development itself. It's like eating your own dog food, right? It happened for a long time. This is the first time we do a software development, that whole development itself, it's going to be disturbed in a way. It'll be, it'll be, it'll be more, uh, profound impact on the whole product development. And it'll be cheaper. The product, go to market will be much cheaper. Like how mobile revolutionized, the next evolution will be on using, um, generative AI-like capability to make your product cheaper and go to market in a short term. That's, that's, that's going to happen eventually.
Kovid Batra: Right. I think, uh, this, this is bound to happen. Even I believe so. It is, it is already there. I mean, it's not like, uh, you're talking about real future, future. It's almost there. It's happening right now. But what do you think on the point where this technology, which is right now, uh, not hosted locally, right? Uh, we are talking about inventing, uh, LLMs locally into your servers, into your systems. How do you see that piece evolving? Because lately I have been seeing a lot of concerns from a lot of companies and leaders around the security aspect, around the IP aspect where you are putting all your code into a third-party server to generate new code, right? You can't stop developers from doing that because they've already started doing it. Earlier, the method was going to stack overflow, taking up some code from there, going to GitHub repositories or GitLab repositories, taking up some code. But now this is happening from a single point of source, which is cloud hosted and you have to share your code with third parties. That has started becoming a concern. So though the whole landscape is going to change, as you said, but I think there is a specific direction in which things are moving, right? Very soon people realized that there is an aspect of security and IP that comes along with using such tools in the system. So how do you see that piece progressing in the market right now? And what are the things, what are the products, what are the services that are coming up, impacting this landscape?
Venkat Rangasamy: It's a good question, actually. We, after a couple of years, right, what the realization even I came up with now, the services which are hosted on a cloud, like, uh, like, uh, public LLMs, right, which, you can use an LLM to generate some of these aspects. From a POC point of view, it looks great. You can see it, what is coming your way. But when it comes to the real product, making product in a production environment is not, um, well-defined because as I said, right, security audit complaints, code IP, right? And, and your compliance team, it's about who owned the IP part of it, right? It's those aspects as well as having the code, your IP goes to some trained public LLM. And it's, it's kind of a compromise where there is, there is, there is some concern around that area and people have started and enterprises have started looking upon something to make it within their workspace. End of the day, from a developer point of view, the experience what developer has, it has to be within that IDE itself, right? That's where it becomes successful. And keeping outside of that IDE is not fully baked-in or it's not fully baked-in part of the developer life cycle, which means the tool set, it has to be as if like it's running in local, right? If you ask me, like, is it doable? For sure. Yes. If you’d asked me an year back, I'd have said no. Um, running your own LLM within a laptop, like another IDE, like how do you run an IDE? It's going to be really challenging if you’d asked me an year back. But today, I was doing some recent experiment on this, um, similar challenges, right? Where corporates and other folks, then the, the, the, any, any big enterprises, right? Any security or any talk to a startup founders, the major, the major roadblock is I didn't want to share my IPR code outside of my workspace. Then bringing that experience into your workspace is equally important.
With that context, I was doing some research with one of the POC project with, uh, bringing your Code Llama. Code Llama is one of the LLMs, public LLM, uh, trained by Meta for different languages, right? It's just the end of the day, the smaller the LLMs, the better on these kinds of tasks, right? You don't need to have 700 billion, 70 billion, those, those parameters are, is, it's irrelevant at this point of coding because coding is all about a bunch of instructions which need to be trained, right? And on top of it, your custom coding and templates, just a coding example. Now, how to solve this problem, set up your own local LLM. Um, I've tested and benchmarked in both Mac and PC. Mac is phenomenally well, I won't see any difference. You should be able to set up your LLM. There is a product called Ollama. Ollama is, uh, where you can use, set up your LLM within your workspace as if it's running, like running in your laptop. There's nothing going out of your laptop. Set up that and go to your IDE, create a simple plugin. I created a VC plugin, visual source plugin, connected to your local LLM, because Ollama will give you like a REST API, just connect it. Now, now, within your IDE, whatever code is there, that is going to talk to your LLM, which means every developer can have their own LLM. And as long as you have a right trained data set for basic language, Java, Python, and other thing, it works phenomenally well, because it's already trained for it. If you want to have a custom coding and custom templating, you just need to train that aspect of it, of your coding standards.
Once you train, keep it in your local, just run like part of an IDE. It's a whole integrated experience, which runs within developer workspaces, is what? Scalable and long run. It, if anything, if it goes out of that, which we, we, we have seen that many times, right, past couple of years. Even though we say our LLMs are good enough to do larger tasks in the coding side, if it's, if you want to analyze the complete file, if you send it to a public LLM, with some services available, uh, through some coding and other testing services, what we have, the challenges, number of the size of the tokens what you can send back, right? There is a limit in the number of tokens, which means if you want to analyze the entire project repository what you have, it's not possible with the way it's, these are set up now in a public site, right? Which means you need to have your own LLM within the workspace, which can work and in, in, it's like a, it's part of your workspace, that's what I would say. Like, how do you run your database? Run it part of your workspace, just make it happen. That is possible. And that's going to be the future. I don't think going any public LLM or setting up is, is, is not a viable option, but having the pipeline set up, it's like a patching or giving a database to your developers, it runs in local. Have that set up where everybody can use it within the local workspace itself. It's going to be the future and the tools and tool sets around that is really happening. And it's, it's at the phase where in an year's time from here, you won't even see that's a big thing. It's just like part of your skill. Just set up and connect your editor, whatever source code editor you have, just connect it to LLM, just run with it. I see that's a feature for the coding part of you. Other SDLCs have different nuance to it, but coding, I think it should be pretty straightforward in a year time frame. That's going to be the normal practice.
Kovid Batra: So I think, uh, from what I understand of your opinion is that the, most of the market would be shifting towards their Local LLM models, right? Yeah. Uh, that that's going to be the future, but I'm not sure if I'm having the right analogy here, but let's talk about, uh, something like GitHub, which is, uh, cloud-sourced and one, which is in-house, right? Uh, the teams, the companies always had that option of having it locally, right? But today, um, I'm not sure of the percentage, uh, how many teams are using a cloud-based GitHub on a locally, uh, operated GitHub. But in that situation, they are hosting their code on a third party, right? The code is there.
Venkat Rangasamy: Yup.
Kovid Batra: The market didn't shape that way if we look at it from that perspective of code security and IP and everything. Uh, why do you think that this would happen for, uh, local LLMs? Like wouldn't the market be fragmented? Like large-scale organizations who have grown beyond a size have that mindset now, “Let's have something in-house.” and they would put it out for the local LLMs. Whereas the small companies who are establishing themselves and then, I mean, can it not be the similar path that happened for how you manage your code?
Venkat Rangasamy: I think it is very well possible. The only difference between GitHub and LLM is, um, the artifact, the, GitHub is more like an artifact management, right? When you have your IP, you're just keeping it's kind of first repository to keep everything safe, right? It just with the versioning, branching and other stuff.
Kovid Batra: Right.
Venkat Rangasamy: Um, the only problem there related to security is who's, um, is there any vulnerability within your code? Or it's that your repository is secure, right? That is kind of a compliance or everything needs to be there. As long as that's satisfied, we're good for that. But from an LLM lifecycle point of view, the, the IP, what we call so far in a software is a code, what you write as a code. Um, and the business logic associated to that code and the customizations happenening around that is what your IP is all about. Now, as of now, those IPs are patent, which means, hey, this is what my patent is all about. This is my IP all about. Now you have started giving your IP data to a public LLM, it'll be challenging because end of the day, any data goes through, it can be trained on its own. Using the data set, what user is going through, any LLM can be trained using the dataset. If you ask me, like, every application is critical where you cannot share an IP, not really. Building simple web pages or having REST services is okay because those things, I don't think any IP is bound to have. Where you have the core business of running your own workflows or your own calculations and that is where it's going to be more tough to use any public LLM.
And another challenge is, what I see in a community is, the small startups, right, they won't do much customization on the frameworks. Like they take Java means Java, right, Node means Node, they take React, just plain vanilla, just run through end-to-end, right? Their, their goal is to get the product up to market quicker, right, in the initial stage of when we have 510 developers. But when it grows, the team grows, what happens is, we, the, every enterprise it's bound to happen, I, I've gone through a couple of cycles of that, you start putting together a framework around the whole standardization of coding, the, the scaffolding, the creating your test cases, the whole life cycle will have enforced your own standard on top of it, because to make it consistent across different developers, and because the team became 5 to 1000, 1000 to 10,000, it's hard to manage if you don't have standards around it, right? That's where you have challenges using public LLM because you will have challenges of having your own code with your own standards, which is not trained by LLM, even though it's a simple application. Even simple application will have a challenge at those points of time. But from a basic point of view, still you can use it. But again, you will have a challenge of how big a file you can analyze using public LLM. It's the one challenge you might have. But the answer to your question, yes, it will be hybrid. It won't be 100 percent saying everybody needs to have their own LLM trained and set up. Initial stages, it's totally fine to use it because that's how it's going to grow, because startup companies don't have much resources to put together to build their own frameworks. But once they get in a shape where they want to have the standardized practices, like how they build their own frameworks and other things. Similar way, one point of time, they'd want to bring it up on their own setup and run with it. For large enterprise, for sure, they are going to have their own developer productivity suite, like what they did with their frameworks and other platforms. But for a small startup, start with, they might use public, but long run, eventually over a point of, over a period of time, that might get changed.
And the benefit of getting hybrid is where you will, you'll make your product quick to market, right? Because end of the day, that's important for startups. It's not about getting everything the way they want to set up. It's important, but at the same time, you need to go to market, the amount of money what you have, where you want to prioritize your money. If I take it that way, still code generation and the whole LLM will play a crucial role on a, on the development. But how do you use and what third-party they can use? Of course, there will be some choices where I think in the future, what this, what I see is even these LLMs will be set up and trained for your own data in a, in a more of a hybrid cloud instead of a public cloud, which means your LLM, what you trained in a, in a hybrid cloud has visibility only to your code. It's not going, it's not a public LLM, it's more of a private LLM trained and deployed on, on a cloud can be used by your team. That'll, that'll, that'll be the hybrid approach in the long run. It's going to scale.
Kovid Batra: Got it. Great. Uh, with that, I think, uh, just to put out some actionable advice, uh, for all the engineering leaders out there who are going through this phase of the AI transformation, uh, anything as an actionable advice for those leaders from your end, like what should they focus on right now, how they should make that transition? And I'm talking about, uh, companies where these engineering leaders are working, which are, uh, Series B, Series A, Series C kind of a bracket. I know this is a huge bracket, but what kind of advice you would give to these companies? Because they're in the growing phase of the, of the whole cycle of a company, right? So what, what should they focus on right now at this stage?
Venkat Rangasamy: Here, here is where some start. I was talking to some couple of, uh, uh, ventures, uh, recently about similar topic, how the landscape is going to change as for software development, right? One thing came up in that call frequently was cheaper to develop a product, go to market faster, and the expectation around software development has become changing quite a while, right? In the sense, the expectation around software development and the cost associated to that software development is where it's going to, it's going to be changing drastically. Same time, be clear about your strategy. It's not like we can change 50 percent of productivity overnight now. But at the same time, keep it realistic, right? Hey, this is what I want to make. Here is my charter to go through, from start from ideations to go to market. Here are the meaningful places where I can introduce something which can help the developers and other roles like PMs. Could be even post support, right? Have a meaningful strategy. Just don't go blank with the traditional way what you have, because your investors and advisors are going to start ask questions because they're going to see a similar pattern from others, right? Because that's how others have started looking into it. I would say proactively start going through that landscape and map your process to see where we can inject some of the meaningful, uh, area where it can have impacts, right?
And, and have, be practical about it. Don't think, don't give a commitment. Hey, I make 50 percent cheaper on my development and overnight you might burn because that's not reality, but just.. In my unit test cases and areas where I can build quality products within this money and I can guarantee that can be an industry benchmark. I can do that with introducing some of these practices like test cases, post customer support, writing code in some aspects, right? Um, that is what you need to set up, uh, when you started, uh, going for a venture fund. And have a relook of your SDLC process. That's important. And see how do you inject, and in the long term, that'll help you. And it'll be iterative, but at the end of the day, see, we've gone from waterfall to agile. Agile to many, many other paradigms within agile over a period of time. But, uh, the one thing what we're good at doing is in a software as an industry adapting to a new trend, right? This could be another trend. Keep an eye on it. Make it something where you can make it, make some meaningful impact on your products. I would, I would say, before your investor comes and talked about hey, can you do optimization here? I see another, my portfolio company does this, does this, does this. That's, it's, it's better to start yourself. Be collaborative and see if we can make something meaningful and learn across, share it in the community where other founders can leverage something from you. It will be great. That's my advice to any startup founders who can make a difference. Yep.
Kovid Batra: Perfect. Perfect. Thank you, Venkat. Thank you so much for this insightful, uh, uh, information about how to navigate the situation of changing landscape due to AI. So, uh, it was really interesting. Uh, we would love to have you one another time on this show. I am sure, uh, you have more than these insights to share with us, but I think in the interest of time, we'll have to close it for today, and, uh, we'll see you soon again.
Venkat Rangasamy: See you. Bye.
Implement DORA metrics, gain dev visibility & drive success in your teams.
Try Live Demo
.png)
