The PINNACLE research project (Public Participation in the Evaluation of Artificial Intelligence for Education and System Benchmarking) was successfully completed, with the participation of the Open University of Cyprus.
The Cyprus Center for Trustworthy AI (CyCAT) at the Open University of Cyprus has successfully concluded its PINNACLE project (Public Engagement in AI Evaluation for Education and System Benchmarking), carried out with the generous support of the Cyprus Research and Innovation Foundation (BRIDGE2HORIZON/0823E/0203). Over 18 months, the funding (150.000 Euro) enabled our team to conduct initial scientific research and to coordinate a proposal to extend the work through Horizon Europe.
With the continued emergence of Generative AI and Foundation Models, new AI-enabled products and services enter the market every day. Google's AI-generated answers to our search queries, or the recommendation systems that quietly decide what appears next in our Netflix and Spotify feeds, are just some of the ways in which we now interact with AI in "everyday" contexts, often without even realising it. Yet as these systems become ever more embedded in daily life, most of us have little means of judging whether they treat us fairly, safeguard our privacy, or behave as they should. PINNACLE set out to address precisely this gap by placing the people who use AI at the centre of evaluating it.
Throughout the project, our team developed innovative mechanisms by which members of the public can participate actively in discovering and evaluating AI-enabled products and services, in the spirit of the EU's vision for Trustworthy AI. According to this vision, which underpins the EU AI Act, to be trustworthy, AI systems must respect a particular set of principles, such as being technically robust, fair, non-discriminatory, and respectful of users' privacy. The PINNACLE methodology enables members of the public to have their say in evaluating the extent to which the AI-enabled systems they encounter are truly "trustworthy," in a systematic way that can be documented.
In this spirit, we offered the online course "AI in Everyday Life" twice, using it to test two complementary approaches to public AI evaluation. In the first run, participants adopted a bottom-up approach, reporting each week on an AI application they had genuinely used and assessing its adherence to key ethical principles. In the second, they took a top-down approach, each selecting a single chatbot (ChatGPT, Gemini, or Copilot) to use throughout the course and following guided weekly tasks to evaluate it. Across the two runs, more than 500 members of the Cypriot and Greek communities took part, and the project successfully collected user-sourced evaluations of AI across both groups.
Beyond the course, PINNACLE laid down a conceptual framework for user-centred AI evaluation, produced nine peer-reviewed scientific publications, and coordinated a Horizon Europe proposal that scored well above threshold and will be revised for a future resubmission. Together, these outcomes have strengthened the team's participation in Horizon Europe and expanded its network of collaborators across Europe.