Researchers from the Meaning Alignment Institute have proposed a new approach, Moral Graph Elicitation (MGE), to align AI systems with human values.
As AI becomes more advanced and integrated into our daily lives, ensuring it serves and represents everyone fairly is paramount.
This study further argues that aligning AI with users’ goals is insufficient to ensure positive outcomes.
The researchers argue that aligning AI systems solely with operator intent is insufficient for achieving good AI outcomes.
They state, “AI systems will be deployed in contexts where blind adherence to operator intent can cause harm as a byproduct. This can be seen most clearly in environments with competitive dynamics, like political campaigns or managing financial assets.”
“What are human values, and how do we align to them?”
Very excited to release our new paper on values alignment, co-authored with @ryan_t_lowe and funded by @openai.
📝: https://t.co/iioFKmrDZA pic.twitter.com/NSJa8dbcrM
— Joe Edelman (@edelwax) March 29, 2024
To address this issue, the researchers propose aligning AI with a deeper understanding of human values.
The MGE method has two key components: value cards and the moral graph. These form an alignment target for training machine learning models.
- Values cards capture what is important to a person in a specific situation. They consist of “constitutive attentional policies” (CAPs), which are the things that a person pays attention to when making a meaningful choice. For instance, when advising a friend, one might focus on understanding their emotions, suggesting helpful resources, or considering the potential outcomes of different choices.
- The moral graph visually represents the relationships between value cards, indicating which values are more insightful or applicable in a given context. To construct the moral graph, participants compare different value cards, discerning which ones they believe offer wiser guidance for a specific situation. This harnesses the collective wisdom of the participants to identify the strongest and most widely recognized values for each context.
To test the MGE method, the researchers conducted a study with 500 Americans who used the process to explore three controversial topics: abortion, parenting, and the weapons used in the January 6th Capitol riot.
The results were promising, with 89.1% of participants feeling well-represented by the process and 89% thinking the final moral graph was fair, even if their value wasn’t voted as the wisest.
The study also outlines six criteria that an alignment target must possess to shape model behavior following human values: it should be fine-grained, generalizable, scalable, robust, legitimate, and auditable. The researchers argue that the moral graph produced by MGE performs well on these criteria.
This study proposes a similar approach to Anthropic’s Collective Constitiutal AI, which also crowdsources values for AI alignment.
However, study author Joe Edelman said on X, “Our approach, MGE, outperforms alternatives like CCAI by @anthropic on legitimacy in a case study, and offers robustness against ideological rhetoric. 89% even agree the winning values were fair, even if their own value didn’t win!”
Our approach, MGE, outperforms alternatives like CCAI by @anthropic on legitimacy in a case study, and offers robustness against ideological rhetoric.
89% even agree the winning values were fair, even if their own value didn’t win! pic.twitter.com/sGgLCUtwzN
— Joe Edelman (@edelwax) March 29, 2024
Limitations
There are limitations to AI alignment approaches that crowdsource values from the public.
For example, dissenting views have been integral to societal decision-making for centuries, and history has shown that the majority can often adopt the minority’s divergent viewpoints. Examples include Darwin’s theory of evolution and the struggles to abolish slavery and grant women the right to vote.
While direct public input is democratic, it may lead to populism, where the majority could override minority rights or disregard expert advice.
Ensuring that the voices of marginalized groups are heard and acted upon is a critical challenge.
Moreover, involving the public in decision-making is crucial, but there is a risk of oversimplifying complex issues. Striking a balance between global principles and localized nuances is another challenge, as a widely accepted principle in one culture or region might be controversial in another.
AI constitutions could also reinforce Western values, potentially eroding the views and ideas of those on the periphery.
While the study acknowledges limitations and the need for further development, it provides another strategy for creating AI systems that align with human values. Every attempt counts.
