This is a transcript of the podcast Malcolm deMayo, VP of Financial Services, Nvidia between GRIP Senior Report Carmen Cracknell, Malcom deMayo, who leads global financial services at Nvidia.
[INTRO]
Carmen Cracknell: Welcome to the GRIP Podcast. Today I am joined by Malcolm deMayo, VP of Financial Services at NVIDIA, a company that needs no introduction. According to the latest stats, it is the company with the sixth largest market cap worldwide, a trillion dollar company, and a market leader in AI. Malcolm, I’d love to just hear initially a bit more about you and how you got involved at NVIDIA and your role there.
Malcom deMayo: Thanks Carmen, and it’s great to be here. Thank you for this opportunity. And that was a very kind introduction. So I’ve been working for the better part of 30 years with the C-suite in financial services firms to help them leverage technology and innovation to better manage risk, better manage cost at brands like Oracle, EMC, Teradata, and NCR. And now I’m responsible for the global financial services industry at NVIDIA, which really means we have a unique model, a very different approach. Really we want to marinate in understanding the biggest problems our customers have in financial services and partner with our ecosystem and build that ecosystem. So my job is to create strategy and go to market with our partners. NVIDIA is 100% advisory. No one at NVIDIA makes commission. We’re here to help.
And what we’re trying to do is help customers understand the power of accelerated compute and how they can leverage high performance capabilities and also AI to better improve the delivery of financial services.
Carmen Cracknell: And how recent is this focus that NVIDIA has on AI?
Malcom deMayo: Well it’s an interesting question. If you look at our stock market results, you would think it’s in the last four months, but we’ve actually been working with financial services firms for over 15 years. And our commitment to artificial intelligence started in 2012. Jensen made a very serious bet. Jensen Huang our CEO and founder, made a very significant bet in 2012 when AI had its Big Bang moment at the University of Toronto with the birth of AlexNet, which was really the first application of AI to do a classification of images in the industry we call computer vision, but being able to understand the difference between a cat and a dog, which was powered by an NVIDIA GPU in 2012. So we have been at this for over 10 years. Last year we invested over $5 billion in R&D and a very substantial investment in NVIDIA Labs where we continue to create new compelling capabilities to help our customers adopt AI.
Carmen Cracknell: And why has NVIDIA gone in this direction now? Is it kind of the availability of partners? It’s the right time in terms of AI taking off. What is it that’s triggered this?
Malcom deMayo: Well so that’s a really good question. So what’s going on in the industry is generative AI is standing on the shoulders of a number of different waves of innovation. You had the mobile wave, the internet wave, the cloud wave, and each of these waves generated a lot more data, made a lot more data available, but really most enterprises haven’t been able to use the data because 80% of it’s unstructured. A good friend of mine, Sarah Hinkfuss over at Bain Venture Capitals, I think said this the best.
If these other waves of AI were waves, generative AI is a tsunami.
And so it’s so much more innovative and so much more potential for disruption. What’s going on is you have data centers today are packed to the gills with traditional CPU-based servers. And this sprawl of server compute is consuming almost 2% or more of the world’s available power. And unchecked in a few years, this is going to expand beyond 5%. So it’s completely unsustainable. And AI, with all of this data that’s necessary, not just needs the data, but also needs the compute and horsepower in order to process the data. So it’s really the convergence of data availability, the need to harvest insights from mountains of data, and the need to have the compute horsepower to drive that. That has created this moment in time for NVIDIA because of the investments we’ve made over the last 10 years. Let me give you an example, Carmen. Jensen put up a use case in Computex a couple of weeks ago in Taiwan. A large language model use case running on a CPU farm, 960-ish servers, with a cap X of $10 million and a power draw of over 11 gigawatt hours.
The same workload running on a GPU accelerated server farm required two servers with a $400,000 cap X and a 0.013 gigawatt hour draw. So a 85X reduction in power consumption and a 25X reduction in cap X. So in order to drive large language models, which is what generative AI is, you need the horsepower, but you don’t want to do this in a way that is going to drive more server sprawl, more power consumption, more and more. And the industry sort of guidelines something called Moore’s Law.
Moore’s Law is at its simplest form as CPU power doubles every year with cost and power consumption remaining constant. Well, the law has ended. The physics no longer work.
And so what you’re seeing is what you’re seeing us at is the beginning of companies leveraging generative AI. So shifting from machine learning and machine learning to generative AI or large language models, a shift in that model architecture, which is driving the need to shift from CPU only servers to GPU accelerated compute. So that’s what’s going on.
Carmen Cracknell: Right. I’ll try and wrap my head around all that. I’m not a technical person. That’s so interesting. For a consumer using, say, Internet banking, what is the benefit of having embedded generative AI in financial services? If you could kind of explain it to a layperson.
Malcom deMayo: I mean, it starts with doing doing more with less being able to do banks being able to do more with less. So the benefit on the financial services side is being able to deliver services at a more more efficiently. You know, for example, maybe not a great example for your Internet banking, but most banks have to most banks run a credit, a credit forecast every day to determine what their reserves need to be. And they report that to their their jurisdiction regulators. So in the U.S., that’s the Federal Reserve in Germany. That’s the BaFin in the UK. That’s the F.C.A. or Bank of England. And so they run this report and they run this. And it’s it’s a simulation that consumes massive amounts of compute. In some cases, can take up to 20 hours to run every day.
And this accelerated compute will take less than two hours. Across the span of a year you’ll save 20 million dollars, one workload, one thing that a bank has to do. And you start extrapolating that across all the things they have to do. And you see the possibilities for savings and doing more with less. What does that mean to the customer? It means that the cost of providing financial services is is going to shrink. Banks will be able to do this more efficiently. And that’s always a good thing because because there is competition, that’s always a good thing for the consumer.
Today, a customer calls a call center on a Saturday morning to to get help, to ask a question about a product, to reset their password to whatever. Some simple thing. And they end up waiting 30 minutes in a queue for an agent, for a bank agent to take the call and understand what it is they need and to help them. AI has the potential to do this immediately. AI can be trained. Each bank is required to transcribe or record every one of those calls. And they have thousands of these calls that instruct them on how to take a customer from. I’m really unhappy about something to thank you very much for the great service. You can train an AI model on those transcribed calls and it can become a helper to the call center agent or to the wealth management banker or to the. And so as a customer, when I contact the bank, instead of going through that, recording that gives me five options that aren’t very helpful and I push zero a million times to get to an agent and I finally get to somebody after 30 minutes, AI can do this immediately for me and can do it much more cost effectively and it can do it in any language.
So you start to think of the possibilities from a customer perspective. Banks have thousands of products. No agent could be trained on all of those, but could be an expert on answering questions of all those products. AI can. So so there’s an opportunity to make your call center people, to make your wealth managers, to make your bankers more capable of answering questions.
Carmen Cracknell: Yeah, I know you were at Money 2020 and some of the speakers there talked about freeing up human creativity by getting AI to do these kind of speedy, essential jobs. What do you do you think about that? What what’s the role of humans going forward, do you think?
Malcom deMayo: Well, I think, you know, if you look at the studies and you pay attention to them, which generally is a good idea, they say that people like to deal with people and they want to build a trusted relationship. So I think, you know, thinking about AI as a way to make your people more productive.
You know, if you’re running a bank, you’ve got thousands of people working in a call center. And, you know, one of the things, one of the one of the interesting things that happened during covid was more and more transactions became electronic or digital. And there’s more and more activity in the call centers. There’s a lot and a lot less activity in branches.
So at the end of the day, you want to make your people more productive so that you don’t have to continue to onboard thousands of people to do those to do that work. And AI will help help make them, as I just explained, be more productive. But I think you still want to have people interacting with people, especially when you start talking about advice, which is a regulated topic.
Carmen Cracknell:Definitely. And you mentioned different regulators. How do you think AI and financial services will impact regulation and how will regulators adapt, do you think?
Malcom deMayo: I mean, it’s so common just just being I don’t have a crystal ball. I don’t have a magic wand. But banks have been doing this for decades. Banks, and financial services firms and fintechs. They are all about leveraging innovation to provide financial services in a smarter way. And so they have been working with regulators for decades. They know how to do this. They built governance. They built lines of defense.
And so I think, you know, if you go back a couple hundred years, when locomotives first came out, humanity freaked out. Humanity thought, my God, these trains we’re not going to be able to breathe at this speed. People actually thought that they actually thought the vibrations on a train would somehow separate your brain from its its moorings.
So there was so we have this resistance to change in humanity.
But but the reality is much different. Trains are extremely helpful. Right. And so the same thing is the same thing is going to happen in AI. We’re going to want to find its level. Banks have been have been incorporating technology for years. They’ve been working with regulators for years and and understand how to bring technology responsibly to market. Doesn’t mean there won’t be issues. Doesn’t mean that people won’t make mistakes. Humans make mistakes. But but at the end of the day, this is going to this is going to sort itself out. And and I think the regulators and banks are quickly recognizing just how how an important back to Sarah’s quote. This is a tsunami. This isn’t just a small incremental opportunity. This is an opportunity to deliver bank services at a much lower cost, much more efficiently and to better manage risk while doing it.
Carmen Cracknell: Sure. And I really wanted to ask you about Crypto and Nvidia’s plans or kind of views on that in terms of financial services. Where are you at with that? Do you think?
Malcom deMayo: I mean, crypto is it’s a use case that requires a lot of compute. And so as long as and so we’re going to continue to work, we’ll continue to work with and we’ll continue to supply our technology to those. We we we’re always looking for we’re wired by our founder to look for the biggest problems and use cases that require accelerated compute and to help the ecosystem. That means the ISVs, the service providers and the ultimate providers of service to help them deliver those capabilities more efficiently. Using the technology we’re building and also to learn what new capabilities are required.
So we’re you know, we view crypto as a as an important workload that we’ll continue to support.
Carmen Cracknell: And I know, like I said, you were at Money 2020 Europe. I heard you speak there. What kind of trends did you notice from speaking to other FinTech leaders there and about the direction Fintech is going and how AI will be a part of this?
Malcom deMayo: I just so I think today today we’re six months into the ChatGPT iPhone moment. And a lot of this is still very aspirational. But the but the six months ago, it was completely aspirational. Six months later, plans are plans are in place. Company FinTechs and banks are experimenting with generative AI and looking, looking for ideating and looking for those use cases that are going to move the needle for them. And we’re we’re very involved. Last week or two weeks ago, we were working with one of the large money center banks. We workshopped an idea. And the idea was in a volatile market.
You can have collateral requirements change, for example, from having securities as collateral to moving to cash. The decision, the decision and understanding of that agreement is manual today.
So it takes time to go figure out, do we need do we need to approach our trading partner and require a collateral change? This can all be automated, which, you know, as as more and more of these functions get automated, you reduce counterparty risk, you reduce credit risk, you reduce market risk. And so we’re managing risk better. That’s going to make the regulators more comfortable. It’s going to make the markets more stable. And so the trend, the trends are to look for ways to leverage technology like we always have to do things smarter, better for less. And we’re at the center of that. NVidia is the is the engine of AI. Anyone that’s doing anything serious is working with us. We’ve created an inception program and a launch pad program for startups so that they have a place to go to to work with this technology at no cost to to get expert advice so that they can start figuring out how to refactor their software to run on to run an accelerated compute.
Carmen Cracknell: And Nvidia announced its partnership with Deutsche Bank, who you said you’ve just been chatting to in December last year that was announced. What’s going to be the biggest result, do you think, from this partnership? And can you say anything more right now about any other potential upcoming partnerships?
Malcom deMayo: You have to wait on the upcoming partnerships. But, you know, we’re we’re we’re always looking for opportunities to help to help a large enterprise, an important brand, think through how they’re going to incorporate and adopt a technology like generative AI, how they’re going to use accelerated compute.
And so working with working with Deutsche Bank is a great opportunity. We can’t we’re not we don’t have the scale to do this with hundreds of companies. So we we can only do it with a few and hopefully learn how to make make the capabilities that we build better. So with the Deutsche Bank, looking at ways to accelerate their data pipeline, when you think about A.I. and all the data required, you have lots of data sources in banks. There’s a lot of silos creating the ability to curate that data and accelerate it and accelerate that process so that your expensive data science resource isn’t spending 80 percent of their time doing that. Is an example of one of the things that we’re we’re looking at, looking at their trading, their trading risk management trading model and looking to see if there’s an opportunity for us to accelerate things like price discovery and risk valuation and back model back trading strategy, back testing. All of this is really important in doing a better job of managing risks. They’re hired by their customers to manage risk. And and if we can help them do that smarter, they can turn around and offer more value to their clients.
The leveraging generative AI in that customer service experience that I described a little bit ago. All right. Leveraging potentially digital humans, potentially, you know, experimenting with digital humans called avatars.
And but looking at the step before that, just creating a chatbot that can become an assistant to a wealth manager or an assistant to a private bank or an assistant to a call center person. Right. Experimenting with that. We had Deutsche Bank at our GTC show, which is a show for a conference, a conference for developers.
Jensen’s keynote draws 25 million viewers at every one of these GTC shows, GTC conferences that we do in March every year. Deutsche Bank partnered with us in one of the events. We have a number of our customers deliver it basically sessions, very, very developer centric sessions on how they’re leveraging technology, our technology and incorporating it into interesting use cases in their business. And Deutsche Bank is working very closely with us in the large language model space. So they put on one of the most highly attended sessions at our GTC conference in March. So lots of different places, you know, it’s it’s it’s constantly evolving.
But but really, it’s about helping them understand the power of of high performance compute and also AI and and working through where the needle moving place is to deploy it.
Carmen Cracknell: Awesome. Well, I’ll definitely be staying tuned to hear more about upcoming partnerships and everything going forward. We’ve absolutely whizzed through my questions. Is there anything I’ve missed that you’d like to add? Or I’m sure there’s something. Feel free to.
Malcom deMayo: I think the I think the thing that I would say is that one of the one of the questions I’m often asked is, you know, what are your concerns about the AI, the super powerful AI infinitesimal? Yeah. Yeah, I was actually going to ask, you know, that pretty much more broadly, the power and impact of AI ethically on employment, jobs, the future of work and what your take is on that. And I think this is my take. Not…So my take on that is this will create more opportunity like like other waves of technology. People need to change and adapt, but it’ll create more opportunity.
I think going back to what Sarah said, this is a tsunami. This is a major leap. And so one of the one of the things that everyone needs to be concerned about is that generative AI, previous waves of innovation were in the hands of a percent or two of the entire world’s population.
This is in everyone’s hands. Generative AI is available to everyone. ChatGPT, Bert and Ernie in China. These solutions are available to seven billion people. So almost everyone on the planet. What does that mean? It means that innovation is going to be the Jensen likes to say there’ll be a Cambrian explosion of innovation and accompanying that is disruption.
So this is a this is a train you don’t want to miss and you don’t want to be too cautious about getting on board. So when I think about the it’s it’s really about our enterprises moving fast enough to embrace AI, because the gap between those that are moving at lightning speed and those that are kind of moving at the traditional. This is how we do it. Speed. That gap is growing. And I worry that you’re going to see a lot of of if you if you look back at the Fortune 500 in 2000, if you look at the leading brands across the globe and you look at that list today, half of those companies are gone.
And that was with a wave of innovation. You think of a tsunami, the opportunity for disruption of brands is massive. And they need to get on their horse and giddy up. Absolutely.
Carmen Cracknell: Well, that’s a very pertinent place to leave this. Awesome. And yeah, thank you so much for your time. Really interesting discussion. Got some great material there. We’ve addressed everything.
Malcom deMayo: Great to meet you, Carmen. The opportunity to replace human toil with intelligence, to use data more interestingly.
And, you know, the one thing I would just say is we would invite we are headquartered in Santa Clara. We invite all enterprises. Come visit us.
Carmen Cracknell: I’d love to.
Malcom deMayo: We’d love to show you what we’re doing. We’d love to help you understand how powerful these capabilities are and share how some other industries are using them.
Carmen Cracknell: Awesome. I’ll take you up on that invitation one day. Definitely. Thank you very much for your time.
Malcom deMayo: All right. You take care.
Carmen Cracknell: Thanks very much. Take care.
