Protecting Data in the Age of AI: A Year in Review and What Lies Ahead

By Kevin Shepherdson, CEO of Straits Interactive.

The techscape in 2023 has been nothing short of a sprint on the AI racetrack. With Sam Altman crowned Time’s CEO of the Year and Google launching Gemini to rival OpenAI’s GPT-4, the AI domain has been bustling with innovation and competition. But beneath the algorithmic fanfare, an undercurrent surges - the challenge of data protection which continues to persist and evolve with every groundbreaking AI development. 

It is a landscape that I’ve surveyed at the start of the year, predicting five key trends that would reshape the way organisations and professionals navigate data privacy. From the rise of new cyberthreats to the gaining momentum of data governance, I will now revisit those predictions and reflect on how they have materialised in an AI-charged climate.

1) Ongoing Digital Transformation Will Create Increased Privacy and Security Threats

The acceleration of digital transformation has continued to reshape workplaces globally. What began as a scramble in the thick of Covid-19, is now driven by generative AI. With ChatGPT hitting 100 million users within two months, the impact on jobs has been profound, enhancing productivity and transforming AI into a business collaborator.

However, this positive transformation has brought along risks and constraints. The accessibility of generative AI apps, in particular, has led to increased privacy risks that users may be blind to. 

In a study conducted by our research arm, "Clone Apps" leveraging OpenAI's GPT APIs on the Google Play Store were found to have significant discrepancies between declared data safety practices and the actual behaviour of these apps. 46% of the sampled apps asserted that they did not collect any Personally Identifiable Information (PII) but a closer look at the app permissions reveals that chat histories, classified under PII, were accessible. Popular generative AI desktop applications also fell short of the General Data Protection Regulation (GDPR) and AI transparency standards. Among them, only 48% of the apps that fell under the GDPR’s purview were compliant with the regulations. Fewer than 10% of the apps transparently disclosed AI use or model sources, and 64% remained ambiguous about how their AI models make decisions about the data subjects.

While these studies illuminate the pervasive privacy risks associated with generative AI apps, the full scope of risks are not limited to those of data protection. Ethical fault lines concerning misinformation, plagiarism and livelihoods have also arisen, bringing us to the next point.

2) Continued Increase in Privacy Breaches and Enforcements Beyond Data Security

The rise of AI has introduced new dimensions of risks beyond traditional data security concerns. Misuse and fake content, inaccurate or biassed content, leakage of corporate data, ownership and IP issues, and concerns about consent and ethical use have become prominent, corroborated by various enforcements reported in the media. 

Earlier this year, China reported its first arrest over fake news generated from ChatGPT by a man who conjured a non-existent train crash. Similarly, inaccurate or biassed AI-generated content has had real-world consequences - lawyers in New York faced a USD5000 fine for including six fake judicial decisions generated by ChatGPT in a 10-page legal brief while representing a plaintiff. The failure of the attorney to verify cases cited by the chatbot underscores the need for professionals to critically evaluate AI-generated content, especially when accuracy and reliability are paramount.

Confidentiality has been a major source of anxiety for companies whose employees are using external generative AI apps. Case in point, three separate instances of unintentional leaks of corporate information to ChatGPT were reported by Samsung employees who were using the platform for work. This included sharing confidential source code, code optimisation requests, and a recording of a meeting. While Samsung took immediate action to limit the ChatGPT upload capacity, the case brought to public awareness that sensitive information should never be uploaded onto public generative AI tools as they can be used to further train the model.

On the IP front, there is increasing pressure to hold AI developers accountable for ensuring that their training datasets comply with copyright laws. Comedian and author Sarah Silverman, as well as two other authors, are suing OpenAI and Meta over dual claims of copyright infringement. Alleging that ChatGPT and LLaMA were trained on illegally-acquired datasets containing their works from “shadow library” websites like Bibliotik, Library Genesis and Z-Library.

Evidently, we are currently facing completely new areas of ethical risks associated with generative AI. All these contribute to a growing demand for data protection-related expertise, especially those who are capable of enacting AI Governance in their own companies. 

3) Transition from Data Protection to Data Governance as Demand for Data Protection Related Expertise Grows

As the demand for data protection-related expertise grows, there has been a notable shift from data protection to data governance. In Singapore, a job search study conducted in 2022 found a 125% year-on-year increase in data protection-related job postings. It also found a 608% increase in hiring for Data Governance roles in various sectors since 2021, of which Data Governance-specific roles saw a 272% increase in 2022. 

Calls for AI Governance have also surfaced this year. Largely anchored in the principles of data governance, the dawn of AI Governance has been met with a new AI development life cycle (using the ISO/IEC 38505 Governance of Data as a reference), expanded from the conventional CUDS (Collection, Use, Disposal, Storage) data life cycle under the Personal Data Protection Act (PDPA). These intersect with the AI ethical principles that form the pillars of AI Governance.

To meet the new demands of data governance, we have developed the industry’s first AI DPO Assistant, bringing a new way of enacting AI-driven data governance. The DPO Assistant taps into our Capabara Knowledge System and serves as a self-help tool in the form of conversational AI, providing suggestions for operational queries, information on regulatory mandates, generating policies and SOPs for data governance processes. Leveraging Straits Interactive's exhaustive bank of research on data protection, the AI assistant aims to simplify the time-consuming task of information gathering within the organisation.  

4) More Regulatory Actions Expected Against Improper & Unfair Use of Social Media, Surveillance and Children’s Data

While regulations like the EU's Digital Services Act and the UK's Online Safety Bill target harmful content and unfair data practices on social media, stricter actions against surveillance and children's data are still evolving. However, increased public scrutiny and growing data protection awareness are driving stricter measures in these areas. 

Recently, the Italian Data Protection Authority (DPA) temporarily blocked ChatGPT from processing Italian users’ data, citing privacy concerns and raising questions about children's safety following a data breach. And over in Ireland, Tiktok was fined 345 million euros ($370 million) for breaching privacy laws regarding the processing of children's personal data in the European Union. All these reflect the growing scrutiny over AI applications and the need for regulatory measures to ensure user privacy, especially concerning children.

5) Increased Focus on AI Governance and Ethics as EU Passes New AI Governance Law

The European Parliament's provisional agreement on the Artificial Intelligence Act marks a significant step towards ensuring safe and ethical AI use. Key areas covered include safeguards for general-purpose AI, limitations on the use of biometric identification by law enforcement, bans on social scoring and AI exploitation of user vulnerabilities. Additionally, the World Economic Forum has launched the AI Governance Alliance to champion the responsible global design and release of transparent and inclusive AI systems. All this sets the EU on course to become the 1st global power to regulate AI. 

In response to the expanded equities at issue in AI Governance, the International Association of Privacy Professionals (IAPP) has also rolled out the Artificial Intelligence Governance Professional (AIGP) certification and training for the emerging AI Governance profession. Cementing AI Governance as an essential capability of the now and the future. 

While the EU is pushing for globally harmonised rules aligned with its stringent framework, Southeast Asian countries are adopting a business-friendly approach to AI regulation. The drafted ASEAN “guide to AI ethics and governance” emphasises cultural diversity and serves as voluntary guidance for domestic regulations. Thailand has approved a draft National AI Strategy and Action Plan (2022–2027) to promote AI technology development, while Malaysia's government is exploring the regulation of AI applications and labelling AI-generated content for transparency. In contrast, Singapore's IMDA has introduced the Model AI Governance Framework, offering practical guidance for private sector organisations to address ethical and governance concerns when deploying AI solutions. 

Looking Ahead to 2024

As we approach the end of 2023, it's clear that the intersection of AI and data protection continues to be a defining battleground for how we will interact with the digital world. To stay ahead, companies should remain proactive, update their regulations in response to evolving trends, and establish robust AI governance frameworks. Naturally, this also requires empowering employees to become AI Business Professionals who understand the value, risks and constraints of generative AI in using it responsibly and ethically. The Advanced Certification in Generative AI, Ethics and Data Protection is up for the taking under the Data Protection Excellence (DPEX) Network to expand your tool belt as an informed user.

The demand for data protection expertise, particularly in managing data governance, is on the rise. In the coming year, we will unveil the top five trends for 2024, offering insights into the evolving landscape of AI and data protection. Till then, we must continue to take a steadfast approach to navigating the volatile techscape. 

Capabara will be available by the start of 2024. Stay tuned to our latest announcements on its development by following our CAPABARA Linkedin page or heading over to capabara.com to find out more about how it can empower your organisation. 

One last thing…the DPEX Network is the biggest collective of privacy professionals in ASEAN. If you or your team is embarking on your data protection journey, visit dpexnetwork.org for more community resources or straitsinteractive.com to see how we can support you.