Tournesol - Secure Collaborative Governance of Large AI Models

Background

Tournesol is a research project focusing on developing a collaborative algorithmic governance for large scale AI systems. The research project is sustained by an association founded in 2021 and builds upon multiple years of research in the Ethics and Safety of AI systems [1, 2, 3].

The main focus of our research is to collect a reliable and secure database of human judgements that can serve to align large AI models such as LLM or large scale recommender systems, thus informing and shaping the development of aligned AI models. As part of this endeavor, Tournesol contributes to the ongoing research in ML security in order to securely aggregate these judgments.

Additionally, the collaborative algorithmic governance system that we develop and the data it allows to collect would support regulators by providing them with the algorithmic tools required to assess the alignment of often inscrutable large scale AI models.

You can view our latest research outputs on our publication page. Our website, code and research are entirely open source and accessible to everyone.

Research agenda

Liquid democracy

Two of the important challenges that we have been encountering are the imbalance of contributors’ activities and the difficulty to assess contributors’ expertises. As part of our project we plan to design and implement liquid democracy. Namely, data contributors will be able to select several other contributors that they would be willing to transfer part of their voting right to, either only on content that they themselves did not rate, or on content where they consider that their expertise is not sufficient. Liquid democracy seems critical to allow us to securely value expertise.

Algorithmic representative

An important part of the large-scale collaborative governance mechanisms developed by Tournesol, will be the generalization of each contributor’s values by Algorithmic Representatives [4] Algorithmic representatives are AI agents who vote on behalf of contributors in collective decision-making processes. The algorithmic representatives not only enhance the efficiency of the process but also promote a democratic and inclusive environment that can align with the values of the community.

Certification of expertise

Large scale collective decision systems benefit greatly from being able to identify experts who exhibit more reliable and useful judgements. We plan to support certification of such experts by relying on a secure certification mechanism and to allow transparency and accountability of the judgment of experts. We expect that communities of experts will provide more impactful data for robustly aligning AI models with human values.

Funding target

Tournesol has so far been mostly funded by its founder for about 150’000$. We target to receive about 50’000$ / year of funding to allow us to continue hiring the main developer of the Tournesol platform. A stretch goal for our association is to receive up to 200’000$ of funding which we will use mainly to hire a team of researchers to contribute to our research agenda.

Resources

1. Hoang, L.N. and El Mhamdi, E.M., 2019. Le fabuleux chantier: Rendre l’intelligence artificielle robustement bénéfique (No. BOOK). EDP Sciences.
2. Hoang, L.N., Faucon, L., Jungo, A., Volodin, S., Papuc, D., Liossatos, O., Crulis, B., Tighanimine, M., Constantin, I., Kucherenko, A. and Maurer, A., 2021. Tournesol: A quest for a large, secure and trustworthy database of reliable human judgments. arXiv preprint arXiv:2107.07334.
3. Find our publications at: https://tournesol.app/#publications
4. Lee, M.K., Kusbit, D., Kahng, A., Kim, J.T., Yuan, X., Chan, A., See, D., Noothigattu, R., Lee, S., Psomas, A. and Procaccia, A.D., 2019. WeBuildAI: Participatory framework for algorithmic governance. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), pp.1-35.