In 2022, generative AI burst onto the public scene, and by 2023, it began to take its place in the business world. Now, in 2024, AI stands at a crucial crossroads as both researchers and businesses explore how this groundbreaking technology can be seamlessly woven into our daily lives.
The year 2023 witnessed a surge of increasingly efficient open-source foundation models. Meta’s LLaMa series led the charge, followed by others like StableLM, Falcon, Mistral, and Llama 2. Models such as DeepFloyd and Stable Diffusion have nearly caught up with proprietary leaders. Enhanced by community-driven fine-tuning techniques and datasets, many of these open models now rival the top closed-source ones, even with fewer parameters.
Here are some key AI trends to watch in the coming year:
- Grounded expectations: More realistic goals and outcomes
- Multimodal AI: Integrating multiple types of data inputs
- Smaller, smarter models: Continued open-source breakthroughs
- Hardware challenges: GPU shortages and rising cloud costs
- Enhanced model optimization: Making performance more accessible
- Customized solutions: Tailored models and data pipelines
- Advanced virtual agents: Beyond basic chatbots
- Regulation and ethics: Navigating legal and moral landscapes
- Shadow AI: Managing unsanctioned AI use in the workplace
A Dose of Reality: Adjusting Expectations
When generative AI first caught the public’s attention, business leaders often had a limited understanding, usually based on flashy marketing or sensational news. Many only dabbled with tools like ChatGPT or DALL-E. Now, with some experience under their belts, the business community has a more nuanced understanding of AI’s potential and limitations.
Generative AI is currently at the “Peak of Inflated Expectations” in Gartner’s Hype Cycle, teetering on the edge of a potential slide into the “Trough of Disillusionment.” Meanwhile, Deloitte’s early 2024 report suggested that many executives expect transformative impacts in the near future. The reality will likely be somewhere in the middle—AI presents unique opportunities, but it’s not a cure-all.
Multimodal AI and Video: Expanding Horizons
Generative AI’s ambitions are growing. The next wave of advancements will focus not only on improving performance in specific areas but also on creating multimodal models that can handle multiple data types. Though multimodal models like CLIP and Wave2Vec have been around for a while, they typically excelled in one direction and were limited to specific tasks.
The new generation of models, including proprietary ones like GPT-4V and open-source versions like LLaVa, will blur the lines between natural language processing and computer vision. Video is also joining the mix, with models like Google’s Lumiere capable of text-to-video tasks.
Smaller Models, Bigger Impact: The Open-Source Surge
In the realm of large language models (LLMs), we’ve likely hit the point where adding more parameters yields diminishing returns. Sam Altman, CEO of OpenAI, has hinted that the era of ever-larger models might be ending, with future improvements focusing elsewhere.
While massive models have propelled AI’s recent golden age, they come with drawbacks. Only the largest companies can afford the resources needed to train and maintain these energy-hungry giants. For instance, training a model like GPT-3 can consume as much electricity as over 1,000 households in a year.
The rise of smaller models brings three key advantages:
- Democratization: Smaller models can be run on less expensive hardware, making AI more accessible to individuals and institutions.
- Local Deployment: Running models locally on devices like smartphones sidesteps many privacy and security concerns.
- Explainability: Smaller models are easier to analyze and understand, making AI more transparent and trustworthy.
Hardware Challenges and Cloud Costs
The trend toward smaller models isn’t just about efficiency—it’s also driven by necessity. As demand for AI hardware, particularly GPUs, increases, supply constraints and rising cloud costs will push businesses to seek more efficient solutions.
The ongoing “run on GPUs” is creating pressure not only for more hardware production but also for innovation in cheaper, more accessible computing solutions. Many organizations currently rely on cloud providers for their AI infrastructure, but as hardware shortages worsen, this could drive up costs and complicate on-premise server setups.
Making Optimization Accessible
The push to maximize the performance of compact models is well-served by recent open-source advancements. Key innovations include:
- Low Rank Adaptation (LoRA): This technique reduces the number of parameters that need updating during fine-tuning, speeding up the process and reducing memory requirements.
- Quantization: Reducing the precision of model data points (e.g., from 16-bit to 8-bit) lowers memory usage and speeds up inference.
- Direct Preference Optimization (DPO): This simpler alternative to reinforcement learning for human feedback (RLHF) aligns models with human preferences more efficiently.
These innovations are democratizing AI by allowing smaller players to compete with the big names, offering sophisticated capabilities that were previously out of reach.
Custom AI Solutions: The Power of Tailoring
In 2024, businesses can achieve differentiation through customized AI models rather than relying on repackaged services from tech giants. Open-source AI models provide the flexibility to develop powerful, tailor-made solutions quickly and cost-effectively, especially in specialized fields like legal, healthcare, and finance.
These fields, which often require highly specialized language and concepts, benefit from smaller models that can be run locally on modest hardware. This approach not only enhances privacy and security but also allows for real-time, context-specific AI solutions.
Beyond Chatbots: Evolving Virtual Agents
With more advanced and efficient tools at their disposal, businesses are ready to expand the role of virtual agents beyond basic customer service tasks. As AI systems evolve, they will enable not just communication but also task automation.
Multimodal AI opens new possibilities for virtual agents, allowing them to interact with users in more dynamic ways. For example, rather than just answering text queries, a virtual assistant could use a smartphone camera to provide real-time instructions based on what it “sees.”
Regulation, Copyright, and Ethical AI: Navigating the Minefield
As AI capabilities grow, so do the risks. Issues like deepfakes, privacy concerns, and bias in AI models are becoming more prominent. Regulatory environments are trying to catch up, with mixed results.
In December 2023, the EU reached a provisional agreement on the Artificial Intelligence Act, which includes measures to curb practices like indiscriminate scraping of images for facial recognition databases and using AI for social manipulation. However, in the U.S., meaningful legislation may be slow to arrive, especially in an election year.
China has already implemented some AI restrictions, such as banning price discrimination by recommendation algorithms on social media. But the global landscape remains fragmented, with different regions pursuing varying approaches to AI regulation.
Shadow AI: Managing the Unseen Risks
As generative AI becomes more popular and accessible, businesses face the growing challenge of “shadow AI”—the unsanctioned use of AI tools by employees. This can expose companies to significant risks, including security breaches, privacy violations, and legal liabilities.
Employees might unknowingly feed sensitive information into public AI models or use copyrighted material without permission, leading to serious consequences for their employers. As with shadow IT, companies will need to establish clear policies, provide training, and monitor usage to mitigate these risks.
In the end, the coming year will see generative AI continue to evolve, with a focus on making it more practical, ethical, and accessible for everyone. The winners will be those who can navigate this complex landscape, harnessing AI’s power while avoiding its pitfalls.
