The TrustChain project, part of the European Commission's Next Generation Internet (NGI) initiative, is dedicated to creating a more secure and trustworthy digital environment. Through a series of open calls, TrustChain seeks to fund and support innovative projects that address the critical issues of trust, security, and privacy in digital communication and data sharing. These open calls attract cutting-edge proposals from across Europe, aiming to advance technologies that can ensure robust and reliable digital interactions.

In its second open call (OC2), TrustChain focused on projects that could significantly enhance the privacy and utility of digital data. After a rigorous evaluation process, Clearbox AI was selected to contribute its expertise in synthetic data generation and privacy-preserving technologies to the TrustChain project.

The challenge: Evaluating the privacy and utility balance of synthetic data

Synthetic data generation has emerged as a key technology in mitigating privacy risks associated with AI. Synthetic data mimics the statistical properties of real datasets without exposing individual data points, making it a powerful tool for data sharing and analysis in compliance with privacy regulations like the General Data Protection Regulation (GDPR). However, the challenge lies in ensuring that this synthetic data retains its utility for machine learning and analytics tasks while providing strong privacy guarantees.

Clearbox AI’s approach: The SURE Library

Clearbox AI’s solution to this challenge is the development of the SURE (Synthetic Data: Utility, Regulatory compliance, and Ethical privacy) library. The SURE library introduces an integrated evaluation framework that assesses both privacy and utility of synthetic datasets. Unlike traditional methods that treat these aspects separately, SURE provides a holistic evaluation, enabling users to understand the trade-offs and optimize their synthetic data generation processes accordingly.

Key Innovations of the SURE Library

• Integrated Evaluation Framework: SURE offers a unified interface for evaluating both privacy and utility of synthetic data. This integrated approach ensures that privacy and utility are not considered in isolation, but as interconnected aspects of data management.
• Scalability: Leveraging technologies like Polars and Cython, the SURE library is designed to handle large datasets efficiently, making it suitable for enterprise-level applications.
• Advanced Privacy and Utility Metrics: SURE employs a range of metrics, including statistical similarity, machine learning utility, and distance-based metrics, to provide a comprehensive assessment of synthetic data. These metrics help in quantifying the privacy risks and utility losses, offering deeper insights into the data’s effectiveness.
• Dynamic Risk Evaluation: The library includes mechanisms for continuous testing against evolving privacy threats, ensuring that the synthetic data remains secure over time.
• User-Centric Design: Designed with a focus on user needs, SURE provides an intuitive interface and automated reporting tools. These features make it accessible to a broad range of users, from data scientists and AI developers to compliance officers with limited technical expertise.

Strategic implementation and impact of the SURE project

The implementation strategy of the SURE project is meticulously planned, involving phases of research, development, user feedback, and continuous refinement. This iterative approach ensures that the library is not only technically robust, but also aligned with the practical needs and challenges faced by organizations in managing data privacy and utility.

By offering the SURE library openly, Clearbox AI fosters innovation and collaboration within the AI community, enabling a wider adoption of privacy-preserving technologies.

Conclusion and next steps

Clearbox AI’s contribution to the TrustChain project, through the development of the SURE library, represents a significant advancement in the field of synthetic data. By providing a comprehensive framework for evaluating privacy and utility, Clearbox AI empowers organizations to harness the benefits of synthetic data while safeguarding individual privacy. The SURE library not only addresses current challenges but also sets the stage for future innovations in data privacy and AI.

In an era where data is a valuable resource and privacy concerns are paramount, Clearbox AI’s work in the TrustChain project is a crucial step towards building a trustworthy and efficient AI ecosystem. Through their innovative solutions, they are ensuring that the potential of AI can be realized in a manner that is ethical, compliant, and respectful of individual privacy.

Seeking beta testers for the SURE library

As we move forward, Clearbox AI has begun testing the SURE library and is actively seeking beta testers between July and September 2024. This is an exciting opportunity for data scientists as well as legal professionals in organizations to be at the forefront of cutting-edge technology that balances data utility with privacy, contributing to a more secure and trustworthy digital future.