{"id":23806,"date":"2023-10-18T17:21:19","date_gmt":"2023-10-18T17:21:19","guid":{"rendered":"https:\/\/nftandcrypto-news.com\/crypto\/anthropic-built-a-democratic-ai-chatbot-by-letting-users-vote-for-its-values\/"},"modified":"2023-10-18T17:21:21","modified_gmt":"2023-10-18T17:21:21","slug":"anthropic-built-a-democratic-ai-chatbot-by-letting-users-vote-for-its-values","status":"publish","type":"post","link":"https:\/\/nftandcrypto-news.com\/crypto\/anthropic-built-a-democratic-ai-chatbot-by-letting-users-vote-for-its-values\/","title":{"rendered":"Anthropic built a democratic AI chatbot by letting users vote for its values"},"content":{"rendered":"
\n

In what may be a first of its kind study, artificial intelligence (AI) firm Anthropic has developed a large language model (LLM) that\u2019s been fine-tuned for value judgments by its user community.<\/p>\n

\n

What does it mean for AI development to be more democratic? To find out, we partnered with @collect_intel<\/a> to use @usepolis<\/a> to curate an AI constitution based on the opinions of ~1000 Americans. Then we trained a model against it using Constitutional AI. pic.twitter.com\/ZKaXw5K9sU<\/a><\/p>\n

\u2014 Anthropic (@AnthropicAI) October 17, 2023<\/a><\/p><\/blockquote>\n

Many public-facing LLMs have been developed with guardrails \u2014 encoded instructions dictating specific behavior \u2014 in place in an attempt to limit unwanted outputs. Anthropic\u2019s Claude and OpenAI\u2019s ChatGPT, for example, typically give users a canned safety response to output requests related to violent or controversial topics. <\/p>\n

However, as innumerable pundits have pointed out, guardrails and other interventional techniques can serve to rob users of their agency. What\u2019s considered acceptable isn\u2019t always useful, and what\u2019s considered useful isn\u2019t always acceptable. And definitions for morality or value-based judgments can vary between cultures, populaces, and periods of time. <\/p>\n

Related: <\/em><\/strong>UK to target potential AI threats at planned November summit<\/em><\/strong><\/p>\n

One possible remedy to this is to allow users to dictate value alignment for AI models. Anthropic\u2019s \u201cCollective Constitutional AI\u201d experiment is a stab at this \u201cmessy challenge.\u201d <\/p>\n

Anthropic, in collaboration with Polis and Collective Intelligence Project, tapped 1,000 users across diverse demographics and asked them to answer a series of questions via polling. <\/p>\n

Source, Anthropic<\/figcaption><\/figure>\n

The challenge centers around allowing users the agency to determine what\u2019s appropriate without exposing them to inappropriate outputs. This involved soliciting user values and then implementing those ideas into a model that\u2019s already been trained.<\/p>\n

Anthropic uses a method called \u201cConstitutional AI\u201d to direct its efforts at tuning LLMs for safety and usefulness. Essentially, this involves giving the model a list of rules it must abide by and then training it to implement those rules throughout its process, much like a constitution serves as the core document for governance in many nations. <\/p>\n

In the Collective Constitutional AI experiment, Anthropic attempted to integrate group-based feedback into the model\u2019s constitution. The results, according to a blog post from Anthropic, appear to have been a scientific success in that it illuminated further challenges towards achieving the goal of allowing the users of an LLM product to determine their collective values.<\/p>\n

One of the difficulties the team had to overcome was coming up with a novel method for the benchmarking process. As this experiment appears to be the first of its kind, and it relies on Anthropic\u2019s Constitutional AI methodology, there isn\u2019t an established test for comparing base models to those tuned with crowd-sourced values. <\/p>\n

Ultimately, it appears as though the model that implemented data resulting from user polling feedback outperformed the base model \u201cslightly\u201d in the area of biased outputs. <\/p>\n

Per the blog post: <\/p>\n

\u201cMore than the resulting model, we\u2019re excited about the process. We believe that this may be one of the first instances in which members of the public have, as a group, intentionally directed the behavior of a large language model. We hope that communities around the world will build on techniques like this to train culturally- and context-specific models that serve their needs.\u201d<\/p><\/blockquote>\n