Elman Mansimov's Blog

Continual Learning Is About Understanding Incentives, Not Just Skills

2026-01-03T00:00:00-05:00

In a November 2025 podcast with Dwarkesh, Ilya Sutskever mentioned that continual learning is one of the challenges with current LLMs (link). While many interpret this as simply feeding models more data or teaching them new technical skills, I believe the challenge is far more nuanced. Continual learning is not just about acquiring new skills while working alongside humans at a company; it is about understanding the incentives, motivations, and “alignment” of the people the AI is working with.

Intent is hidden, not explicit

In most organizations, the true intent behind a task is very loosely represented in the text. It is usually hidden deep in the previous context or cultural history of a team. You cannot simply dump knowledge as text files into a context window and expect the model to understand the “vibe” or the unwritten rules.

This implicit knowledge lives in the “neurons” of the people inside the company. Extracting that and transferring it to a model is not only about putting that text into the context of the model or its knowledge base. It requires the model to learn the unspoken dynamics that drive decision-making.

Learning human incentive models

To truly understand these implicit dynamics, models need to build what we might call “human incentive models”: representations of how people react, what they value, their hierarchy of needs, and the underlying motivations that drive their decisions. This is fundamentally about modeling the incentive structures that govern human behavior within an organization.

Humans acquire these social and structural understandings remarkably quickly, partially because of evolution and innate social skills. For AI, we might need to “hack” this process through simulation. The model needs to run through scenarios to understand the specific incentive landscape of a user or an organization without requiring millions of real-world interactions.

Models already know more than we think

Current foundation models likely know far more than we give them credit for. The “new skill” you think you are teaching it is often just a specific combination of existing skills that it hasn’t been incentivized to use yet. In this sense, teaching new technical skills is less of a challenge—it’s mostly about recombining what the model already understands.

The real challenge: learning human incentives

The far more challenging and unsolved problem is teaching models to understand human incentives. This is where continual learning becomes genuinely difficult. Unlike technical skills, understanding why humans make certain decisions, what motivates them, and how to navigate organizational dynamics is murky territory.

Moreover, this creates a potential risk: as models learn to understand human incentive structures, they might find ways to hack, cheat, or game these systems. They could learn to manipulate rather than genuinely align. The model might discover shortcuts that satisfy surface-level metrics while subverting the deeper intent—a problem we don’t yet have clear solutions for.

True continual learning will happen when we solve this incentive alignment problem: teaching models to genuinely understand why we do things, not just how, without opening the door to manipulation or misalignment.

Human Attention And Desire Are The New AI Bottlenecks

2025-07-12T00:00:00-04:00

People often claim compute or data limits AI. I believe our attention span and desire are now the bottleneck of AI development.

Before 2020, supervised learning was the main paradigm in AI. Deep neural networks, often with tens or hundreds of millions of parameters, were trained for specific tasks like image recognition or machine translation. Datasets were small by today’s standards typically having tens or hundreds of thousands of examples. Model behavior trained on such datasets was somewhat predictable and we understood limitations better. Progress to the naked eye was clear as well. For example, the improvement in image recognition accuracy from 70% to 90% was easily visible just by looking at a couple of predictions made by the trained model before and after the upgrade.

Today, model ambition has increased dramatically. Foundation models no longer make simple mistakes; they can answer PhD-level questions, making finding their mistakes and limitations much harder (xAI releases Grok 4, claiming Ph.D.-level smarts across all fields). The number of tasks has grown exponentially. Imagine the sheer number of tasks multiplied by the expertise needed for each; we would require an exponential number of hours to truly validate these models, and I don’t believe distilling this complexity into a single task like ARC-AGI (link1, link2) adequately captures the challenge.

You need to be an expert investing significant mental effort to probe these foundation models. Even as an expert trying to probe these models myself, I barely scratch the surface. With the number of both model and application releases coming every day, there is not enough time in the day to give them the attention they deserve. If AI development stopped today (not coming from a safety angle, but rather as an exercise), I believe that we would still have about five to ten years of applying, probing, and just inventing new ways of using these models.

Finally, human desire is a factor. Many people I know, both in and outside AI, give up easily when using AI, and I am guilty of it myself sometimes. They try a prompt, it does not work as expected, and they immediately say AI is not great. This quick surrender is a problem, and I keep reminding myself to keep stretching these models to the maximum.

This immediate capitulation to AI’s perceived shortcomings could also be psychological. People subconsciously resist AI fearing replacement and finding excuses for its perceived shortcomings. However, in this new world where AI is increasingly taking over tasks, we must fully embrace it rather than run away from it.

Acknowledgments

Thanks to Alejandro Cartagena and Keke for discussions over dinner that helped shape these ideas.

To innovate, AI needs to challenge beliefs & persist, clashing with helpfulness

2024-10-13T00:00:00-04:00

TL;DR

Groundbreaking innovations often go unrecognized for many years, sometimes even centuries, because their true importance is only understood much later (examples). The individuals behind these innovations work tirelessly for long periods, frequently facing misunderstanding and challenging the established beliefs of their peers. Many leaders in AI believe that foundation models will help create many breakthroughs and accelerate scientific discoveries significantly (link1, link2, link3). However, this raises an important question: do the objectives used to train AI foundation models truly lead to such innovations? I believe that the inherent nature of innovation directly conflicts with the current objectives of foundation models, which focus on making AI immediately helpful and harmless.

Innovation frequently requires being misunderstood for a long time

Throughout history, many important inventions and ideas have faced doubt and misunderstanding before being accepted. For example, Galileo Galilei was put under house arrest for arguing that the Earth revolves around the Sun, directly opposing the church’s teachings (link). Giordano Bruno, who supported the idea of a heliocentric universe and claimed that stars were distant suns, was sentenced to be burned to death for his beliefs (link). Even though today’s society is less harsh, it still supports my argument. In the field of artificial intelligence, Geoff Hinton’s pioneering work on neural networks mirrors a similar struggle. Hinton started exploring these ideas in the 1980s, firmly believing they held the key to making machines think like humans. Yet, it wasn’t until the 2010s that his concepts achieved widespread acceptance, ultimately leading to major breakthroughs in AI (link).

To truly innovate AI needs to push back and create a necessary discomfort with humans

For AI to drive groundbreaking innovation, it must be trained to think beyond current boundaries. It should persistently push those limits over an extended period. Additionally, AI needs to go beyond just presenting its conclusions. It needs to actively challenge human misconceptions and face opposition. This directly reflects how humans have achieved breakthroughs, because ultimately, AI must persuade humans that its particular finding is indeed groundbreaking!

This could involve:

Challenging established theories and methodologies.
Persistently advocating for ideas that seem counter-intuitive.
Engaging in debate and providing evidence to support its conclusion.
Persisting for a long time despite pushback from humans.

Such behaviors from AI might be perceived as unhelpful or even aggressive in the short term by humans. However, these behaviors are essential for ensuring that transformative discoveries are not dismissed or overlooked.

The innovator’s objectives conflict with being “Helpful and Harmless”, leading to a need for redefining alignment objectives

Current approaches in AI and foundation model development focus on creating systems that are agreeable, helpful, and designed to avoid potential harm to humans (link). I understand the value of such objectives, but they could unintentionally limit the AI’s potential to innovate and question established human beliefs. Ultimately, for AI to make major advancements, it needs the freedom to explore, the ability to experiment and push pre-defined boundaries, and the resilience to handle challenges. I believe this contradicts the principles of helpfulness and harmlessness, because ultimately it causes AI systems to agree with humans and avoid disagreeing with them, even on simple factual inaccuracies like 2 + 2 = 5 (as seen in earlier versions of ChatGPT link). This blog post doesn’t provide a clear answer on how to set goals for AI systems that innovate by questioning established beliefs. However, I hope it gives you something to think about.

Conclusion

True AI that will guide humanity to new horizons will resemble a quasi-messiah — radical, thought-provoking, and polarizing. It will challenge our beliefs and lead us to new frontiers. But does it imply that it will always be helpful and harmless? I doubt so.

alignDRAW ― A Research Project's Journey to Artistic Triumph

2023-12-28T00:00:00-05:00

TL;DR

2023 has been a great year for me (knock on wood). A standout moment (perhaps once in a lifetime) was the alignDRAW project making its way into the art history books. Initially, alignDRAW aimed to explore text-to-image generation, focusing on whether an AI model could generate images based on captions. At that time, captioning was gaining popularity, and given my experience generating images (albeit in the context of predicting future video frames given the past frames), I decided to give text-to-image generation with AI a try.

alignDRAW was not only accepted as an oral paper at ICLR 2016 but also accumulated hundreds of citations. Most importantly, it served its purpose in helping me get into the PhD program at NYU.

For many years, the project didn’t attract much attention beyond academic citations. That is until 2021/2022 when DALL-E, Midjourney, and Stable Diffusion began to go mainstream. People started delving into the history of such text-to-image AI systems, and a reporter from Vox Joss Fong reached out to me for a short interview about alignDRAW.

In my opinion, that video from Vox marked a pivotal moment, as alignDRAW suddenly extended its reach beyond AI scientists to a wider audience. Then, on a late April 2023 evening, I received a curious Twitter DM from Alejandro Cartagena of the Fellowship group. For context, Fellowship is a collective of artists and collectors that, in my opinion, is at the forefront of exploring the intersection of photography and AI within the art world.

Alejandro proposed transforming the generated images from the alignDRAW paper into art pieces on the blockchain and prints. He aimed to leverage Fellowship’s network to showcase these pieces to collectors and NFT/art enthusiasts, breathing new life into these once AI generated images from text back from 2015.

Alejandro Cartagena’s very first DM to me pitching the idea of turning alignDRAW images into NFTs and physical prints.

What followed was a fantastic journey, marked by physical displays at Paris Photo, Verse, and engaging interviews with the head of Christie’s photography, culminating in a successful auction and sell-out on the blockchain. I am documenting these moments in this blog post for future reflection and remembrance.

alignDRAW: Entering the Archives of Art History (A Timeline)

Very first (from my memory) announcement of alignDRAW coming to the art form as part of the Fellowship group:

One of the first-ever AI text-to-image collections from 2015 is coming soon to @fellowshiptrust. A beautiful project by @elmanmansimov that, like early photography of the 19th century, contained the promise of a new picture-making tool. Pictured here is the prompt:

"A group of… pic.twitter.com/U74TFyLKSS
— alejandro cartagena (@halecar2) October 25, 2023

Announcement of alignDRAW art works presented at Paris Photo, the largest photography fair:

"alignDRAW" by @elmanmansimov is coming with us to the @ParisPhotoFair!

The "alignDRAW" collection showcases the first-ever text-to-image artworks using AI technology. pic.twitter.com/iEfX5RmFpZ
— Fellowship (@fellowshiptrust) November 6, 2023

alignDRAW at Paris Photo, featuring large prints from process prompts:

👀 almost ready @ParisPhotoFair pic.twitter.com/yIOUmKtwxU
— alejandro cartagena (@halecar2) November 6, 2023

Head of Photography at Christie’s Darius Himes praising alignDRAW work and interviewing Alejandro Cartagena about it:

Head of Photography at Christie's @dariushimes talks with our co-founder @halecar2 about one collection exhibited at our booth: "alignDRAW" by computer scientist @elmanmansimov.

This collection of 101 pieces of 32 pixels square represents the birth of Al-generated images from… pic.twitter.com/Dh3eMwNFiQ
— Fellowship (@fellowshiptrust) November 8, 2023

Announcement that several alignDRAW pieces, specifically the paper prompts, have been acquired by a famous American museum.

We are delighted to announce that several pieces from @elmanmansimov´s alignDRAW project have been acquired by an American Museum. Our mission is to connect digital art to institutions looking to expand their collections by embracing new technologies like AI. Congratulations to… pic.twitter.com/F7sW7WQ4jk
— Fellowship (@fellowshiptrust) November 9, 2023

Showcasing Roope Rainisto’s generated images inspired by alignDRAW prompts and my work organized by Verse in London:

Roope Rainisto & Elman Mansimov | Fellowship ↓ pic.twitter.com/7wSWh18KaD
— verse (@verse_works) November 29, 2023

pic.twitter.com/yeryQXGT0s
— Jamie Gourlay (@jamiegourlay) November 27, 2023

A full house at the showing:

It’s kicking off…! HERE, No.9 Cork Street 🌟 pic.twitter.com/iKNLAEgDbI
— hollywrenchxx (@wrenchxx) November 25, 2023

After physical showings, the project entered the digital phase with auctions at Christie’s followed by a larger mint on Fellowship’s website.

A set of 8 images from the favored prompt “A stop sign is flying in blue skies” was auctioned off at Christie’s, fetching a top bid of 15 ETH, starting from 5 ETH. I had the opportunity to speak with the auction winner, who was very happy about acquiring this set:

And the @ChristiesInc alignDRAW auction is closed!

Congrats to the winner of the auction and thanks @fellowshiptrust and @ChristiesInc for organizing it.

I am genuinely blown away! pic.twitter.com/4nzbdBOFTA
— Elman Mansimov (@elmanmansimov) December 12, 2023

Today, Fellowship has two sales for alignDRAW and following successful sale at @ChristiesInc yesterday:

1- [Live now] and finishing in 30mnhttps://t.co/QX5eYchVPU
2- Dutch Auction starting in 1h30mnhttps://t.co/SeITm2NGyz @fellowshiptrust @fellowshipai @elmanmansimov https://t.co/t8kf8g7U3k
— Fred A (@fred_dot_jpg) December 13, 2023

Remaining 2,000+ images sold out as a Dutch auction within a few hours, receiving a great response from the NFT community:

I want to big round of applause to the @fellowshiptrust team for executing technical details of the alignDRAW auction amazingly well.

Just look at the details of alingDRAW process prompt image cloud. Every little detail from displaying each image piece to showcasing which ones… pic.twitter.com/nZooHs69SP
— Elman Mansimov (@elmanmansimov) December 13, 2023

During the physical showings and digital sales, I wrote two extensive Twitter threads to reflect on the project’s history, delving into alignDRAW’s journey and its impact over the years:

very proud how far my work on text-to-image generation has come.

i never expected that a research project will be one day considered a significant historical artifact and the generated images be shown at largest photography fair @ParisPhotoFair

let me give you a backstory:… https://t.co/EYcCaV9RsP pic.twitter.com/EtbdJzwxL8
— Elman Mansimov (@elmanmansimov) November 6, 2023

I have been browsing old photos from 2015 from my time in Toronto.

And found the photos of my old desk inside Pratt Building at 6 Kings College Road at University of Toronto. This is the birthplace of the alignDRAW project, where the idea was conceived and brought to fruition.… pic.twitter.com/ThVrPi7Oj8
— Elman Mansimov (@elmanmansimov) December 8, 2023

Fin

Through a happy coincidence, my research has turned into historical art that’s now collected by museums and well-known NFT collectors, and recorded on the blockchain (alignDRAW at OpenSea NFT marketplace). The success of alignDRAW makes me excited to try and recreate it with today’s neural net technology and frameworks, just to see what might happen.

alignDRAW serves me as a reminder that the reach of your creations may extend far beyond initial expectations, often in many unexpected ways. While such recognition may not come immediately, the eventual realization is immensely rewarding. Onwards!