What Is Generative AI (GenAI)? How Does It Work?

Generative artificial intelligence, unlike its predecessors, can create new content by extrapolating from its training data. Its extraordinary ability to produce human-like writing, images, audio, and video have captured the world’s imagination since the first generative AI consumer chatbot was released to the public in the fall of 2022. GenAI now powers a range of consumer and professional applications and services that help save time, money, and effort.

But every action has an equal and opposite reaction. So, along with its remarkable productivity prospects, generative AI brings new potential business risks—such as inaccuracy, privacy violations, and intellectual property exposure—as well as the capacity for large-scale economic and societal disruption. For example, generative AI’s productivity benefits are unlikely to be realized without substantial worker retraining efforts and, even so, will undoubtedly dislocate many from their current jobs. Consequently, government policymakers around the world, and even some technology industry executives, are advocating for rapid adoption of AI regulations.

What Is Generative AI (GenAI)?

Generative AI (GAI) is the name given to a subset of AI machine learning technologies that have recently developed the ability to rapidly create content in response to text prompts, which can range from short and simple to very long and complex. Different generative AI tools can produce new audio, image, and video content, but it is text-oriented conversational AI that has fired imaginations. In effect, people can converse with, and learn from, text-trained generative AI models in pretty much the same way they do with humans.

Generative AI took the world by storm in the months after ChatGPT, a chatbot based on OpenAI’s GPT-3.5 neural network model, was released on November 30, 2022. GPT stands for generative pretrained transformer, words that mainly describe the model’s underlying neural network architecture.

There are many earlier instances of conversational chatbots, starting with the Massachusetts Institute of Technology’s ELIZA in the mid-1960s. But most previous chatbots, including ELIZA, were entirely or largely rule-based, so they lacked contextual understanding. Their responses were limited to a set of predefined rules and templates. In contrast, the generative AI models emerging now have no such predefined rules or templates. Metaphorically speaking, they’re primitive, blank brains (neural networks) that are exposed to the world via training on real-world data. They then independently develop intelligence—a representative model of how that world works—that they use to generate novel content in response to prompts. Even AI experts don’t know precisely how they do this as the algorithms are self-developed and tuned as the system is trained.

Businesses large and small should be excited about generative AI’s potential to bring the benefits of technology automation to knowledge work, which until now has largely resisted automation. Generative AI tools change the calculus of knowledge work automation; their ability to produce human-like writing, images, audio, or video in response to plain-English text prompts means that they can collaborate with human partners to generate content that represents practical work.

“The Oracle Cloud trains dozens of AI models and embeds hundreds of AI agents in cloud applications,” Larry Ellison, chairman and chief technology officer of Oracle, said during the company’s December 2024 earnings call.

“Oracle's AI agents automate drug design, image and genomic analysis for cancer diagnostics, audio updates to electronic health records for patient care, satellite image analysis to predict and improve agricultural output, fraud and money laundering detection, dual factor biometric computer logins, and real-time video weapons detection in schools.”

Generative AI vs. AI

Artificial intelligence is a vast area of computer science, of which generative AI is a small piece, at least at present. Naturally, generative AI shares many attributes in common with traditional AI. But there are also some stark distinctions.Common attributes: Both depend on large amounts of data for training and decision-making (though the training data for generative AI can be orders of magnitude larger). Both learn patterns from the data and use that “knowledge” to make predictions and adapt their own behavior. Optionally, both can be improved over time by adjusting their parameters based on feedback or new information.
Differences: Traditional AI systems are usually designed to perform a specific task better or at lower cost than a human, such as detecting credit card fraud, determining driving directions, or—likely coming soon—driving the car. Generative AI is broader; it creates new and original content that resembles, but can’t be found in, its training data. Also, traditional AI systems, such as machine learning systems are trained primarily on data specific to their intended function, while generative AI models are trained on large, diverse data sets (and then, sometimes, fine-tuned on far smaller data volumes tied to a specific function). Finally, traditional AI is almost always trained on labeled/categorized data using supervised learning techniques, whereas generative AI must always be trained, at least initially, using unsupervised learning (where data is unlabeled, and the AI software is given no explicit guidance).

Another difference worth noting is that the training of foundational models for generative AI is “obscenely expensive,” to quote one AI researcher. Say, $100 million just for the hardware needed to get started as well as the equivalent cloud services costs, since that’s where most AI development is done. Then there’s the cost of the monumentally large data volumes required.

Key TakeawaysGenerative AI became a viral sensation in November 2022 and is expected to soon add trillions of dollars to the global economy—annually.
AI is a form of neural network–based machine learning trained on vast data sets that can create novel text, image, video, or audio content in response to users’ natural language prompts.
Market researchers predict that the technology will deliver an economic boost by dramatically accelerating productivity growth for knowledge workers, whose tasks have resisted automation before now.
Generative AI comes with risks and limitations enterprises must mitigate, such as “hallucinating” incorrect or false information and inadvertently violating copyrights.
It is also expected to cause significant changes in the nature of work, including possible job losses and role restructuring.

Generative AI Explained

For businesses large and small, the seemingly magical promise of generative AI is that it can bring the benefits of technology automation to knowledge work. Or, as a McKinsey report put it, “activities involving decision making and collaboration, which previously had the lowest potential for automation.”

Historically, technology has been most effective at automating routine or repetitive tasks for which decisions were already known or could be determined with a high level of confidence based on specific, well-understood rules. Think manufacturing, with its precise assembly line repetition, or accounting, with its regulated principles set by industry associations. But generative AI has the potential to do far more sophisticated cognitive work. To suggest an admittedly extreme example, generative AI might assist an organization’s strategy formation by responding to prompts requesting alternative ideas and scenarios from the managers of a business in the midst of an industry disruption.

In its report, McKinsey evaluated 63 use cases across 16 business functions, concluding that 75% of the trillions of dollars of potential value that could be realized from generative AI will come from a subset of use cases in only four of those functions: customer operations, marketing and sales, software engineering, and research and development. Revenue-raising prospects across industries were more evenly distributed, though there were standouts: High tech topped the list in terms of the possible boost as a percentage of industry revenue, followed by banking, pharmaceuticals and medical products, education, telecommunications, and healthcare.

Separately, a Gartner analysis correlated with McKinsey’s predictions: For example, that more than 30% of new drugs and materials will be discovered using generative AI techniques by 2025, up from zero today, and that 30% of outbound marketing messages from large organizations will, likewise, be synthetically generated in 2025, up from 2% in 2022. And in an online survey, Gartner found that customer experience and retention was the top response (at 38%) of 2,500 executives who were asked about where their organizations were investing in generative AI.

What makes it possible for all this to happen so fast is that, unlike traditional AI, which has been quietly automating and adding value to commercial processes for decades, generative AI exploded into the world’s consciousness thanks to ChatGPT’s human-like conversational talent. That has also shed light on, and drawn people to, generative AI technology that focuses on other modalities; everyone seems to be experimenting with writing text, or making music, pictures, and videos using one or more of the various models that specialize in each area. So, with many organizations already experimenting with generative AI, its impact on business and society is likely to be colossal—and will happen stupendously fast.

The obvious downside is that knowledge work will change. Individual roles will change, sometimes significantly, so workers will need to learn new skills. Some jobs will be lost. Historically, however, big technology changes, such as generative AI, have always added more (and higher-value) jobs to the economy than they eliminate. But this is of little comfort to those whose jobs are eliminated.

How Does Generative AI Work?

There are two answers to the question of how generative AI models work. Empirically, we know how they work in detail because humans designed their various neural network implementations to do exactly what they do, iterating those designs over decades to make them better and better. AI developers know exactly how the neurons are connected; they engineered each model’s training process. Yet, in practice, no one knows exactly how generative AI models do what they do—that’s the embarrassing truth.

“We don’t know how they do the actual creative task because what goes on inside the neural network layers is way too complex for us to decipher, at least today,” said Dean Thompson, a former chief technology officer of multiple AI startups that have been acquired over the years by companies, including LinkedIn and Yelp, where he remains as a senior software engineer working on large language models (LLMs). Generative AI’s ability to produce new original content appears to be an emergent property of what is known, that is, their structure and training. So, while there is plenty to explain vis-a-vis what we know, what a model such as GPT-3.5 is actually doing internally—what it’s thinking, if you will—has yet to be figured out. Some AI researchers are confident that this will become known in the next 5 to 10 years; others are unsure it will ever be fully understood.

Here’s an overview of what we do know about how generative AI works:

Start with the brain. A good place to start in understanding generative AI models is with the human brain, says Jeff Hawkins in his 2004 book, “On Intelligence.” Hawkins, a computer scientist, brain scientist, and entrepreneur, presented his work in a 2005 session at PC Forum, which was an annual conference of leading technology executives led by tech investor Esther Dyson. Hawkins hypothesized that, at the neuron level, the brain works by continuously predicting what’s going to happen next and then learning from the differences between its predictions and subsequent reality. To improve its predictive ability, the brain builds an internal representation of the world. In his theory, human intelligence emerges from that process. Whether influenced by Hawkins or not, generative AI works exactly this way. And, startlingly, it acts as if it is intelligent.

Build an artificial neural network. All generative AI models begin with an artificial neural network encoded in software. Thompson says a good visual metaphor for a neural network is to imagine the familiar spreadsheet, but in three dimensions because the artificial neurons are stacked in layers, similar to how real neurons are stacked in the brain. AI researchers even call each neuron a “cell,” Thompson notes, and each cell contains a formula relating it to other cells in the network—mimicking the way that the connections between brain neurons have different strengths.

Each layer may have tens, hundreds, or thousands of artificial neurons, but the number of neurons is not what AI researchers focus on. Instead, they measure models by the number of connections between neurons. The strengths of these connections vary based on their cell equations’ coefficients, which are more generally called “weights” or “parameters.” These connection-defining coefficients are what’s being referred to when you read, for example, that the GPT-3 model has 175 billion parameters. The latest version, GPT-4, is rumored to have trillions of parameters, though that is unconfirmed. There are a handful of neural network architectures with differing characteristics that lend themselves to producing content in a particular modality; the transformer architecture appears to be best for large language models, for example.

Indian Scientist Awards 2025!

Visit: https://indianscientist.in/

Nominate Now: https://indianscientist.in/award-nomination/?ecategory=Awards&rcategory=Awardee

Registration page : https://indianscientist.in/award-registration/

Contact us : indian@indianscientist.in