Generative AI and GDPR: A Practical Guide for DPOs, CIOs and Lawyers

Does the GDPR apply to generative AI, and can a model escape it?

The GDPR applies as soon as an AI system collects or uses personal data. The CNIL puts it this way: like any processing, the collection and use of data through an AI system must comply with the GDPR and with individuals' rights [2]. This applies to training data as well as to data that users enter. An unofficial web source expresses the same point, as an illustration [21].

What "generative AI" covers

The CNIL uses "generative AI" to refer to systems capable of creating content, for example text [7]. It broadly distinguishes three categories of tasks, one of which is generating content from a general instruction [3].

Can a model escape the GDPR?

Yes, in some cases, but this must be demonstrated, not presumed. The CNIL identifies two situations:

• Anonymous model: some AI models are anonymous and do not have to apply the GDPR [1].
• Model that memorises: others, such as large language models (LLMs), may fall under the GDPR [1]. This is the case if they memorise personal data from their training [6].

A model's status is therefore not settled in advance. It requires a case-by-case analysis, which the CNIL details in a dedicated practical guide [6]. It also recommends anonymising training data or making sure the model is anonymous [10].

Consequence for the controller

A model considered anonymous does not exempt anyone from justifying that position. Every controller must adequately document the compliance of its processing, for example in a data protection impact assessment (DPIA) [9].

This text explains the CNIL's recommendations. It does not replace the assessment of compliance, which falls to the company and its advisers.

Which legal basis and which purpose should be chosen for each use?

Any AI system that processes personal data must rest on a legal basis. According to the CNIL, it "may only be implemented if it corresponds to a justification provided for by law", and the GDPR provides for six [5]:

• consent;
• compliance with a legal obligation;
• performance of a contract;
• performance of a task carried out in the public interest;
• protection of vital interests;
• pursuit of a legitimate interest.

Purpose first, legal basis second

The purpose is the "why" of the processing, and it is what allows the appropriate legal basis to be chosen. The CNIL confirms that AI is compatible with the GDPR, provided certain conditions are met and certain red lines are not crossed. The purpose limitation principle is one of these conditions [11]. A purpose and a legal basis must therefore be defined for each use, not a single one for "AI" in general. Summarising content, helping draft customer replies and screening job applications do not have the same purposes or the same risks.

Example: a client's name or a contract in a prompt

Pasting a client's name, an employee's file or a contract into a prompt amounts to transmitting that data to a provider. This is processing, and it requires a purpose and a legal basis [30]. This example comes from a training website, an unofficial source: it illustrates the reasoning but is not a CNIL text. It nonetheless fits the approach of identifying processing at each stage: prompts, documents, training and outputs [15].

Oupi is designed, developed and hosted in France (learn more). This does not choose the legal basis on the organisation's behalf: the assessment of compliance remains the company's and its advisers'.

Which data can be sent to a generative AI tool?

The principle of data minimisation sets the rule: only send a generative AI tool the personal data strictly useful for the purpose pursued. The GDPR requires that collected data be "adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed" [23]. In practice, use is neither prohibited nor unrestricted: it is possible provided you do not supply more than necessary and know where the data goes [24]. This formulation comes from a commercial actor, but it restates the minimisation principle.

The flows to map before any use

You must identify the data processed at each stage of the cycle [15][18]:

• prompts: what users enter;
• documents: files attached to or connected to the tool;
• training: any reuse of data to improve the model;
• outputs: false information about a person can also be personal data [15].

This mapping also helps to assign responsibilities and the safeguards to be put in place [18].

What the CNIL recommends

In its recommendations on the development of AI systems, the CNIL advocates, where possible, general rules for detecting and pseudonymising personal data rather than a blacklist [10]. It also recommends anonymising training data, or ensuring that the model is anonymous [10]. These recommendations concern the development of AI systems. Applying them to users' prompts and documents is a precautionary extension, not a requirement stated by the CNIL.

Choosing a tool designed, developed and hosted in France, such as Oupi, does not exempt you from this approach: minimisation remains for the company to assess, with its advisers.

What should be checked with the provider, particularly in the case of transfers outside the European Union?

The CNIL places the duty to verify the provider on the user or integrator. According to its questions and answers on generative AI, it is for whoever deploys a generative AI system to ask the provider whether that system complies with the GDPR, since most have been configured using personal data [3]. This step falls under the accountability principle: a provider's claim to be compliant does not relieve the company of the duty to verify.

A provider established outside the European Union implies a data transfer that must be framed. When an employee submits personal data to a tool whose provider is established outside the EU, that transfer must meet the strict conditions set out in Articles 44 et seq. of the GDPR [20]. This source is a law firm, not an authority: it is offered here as a reference point, and the text of the regulation remains the authority to consult.

The points to have the provider confirm in writing follow directly from this framework:

• the country where the provider is established and that of any subprocessors;
• the location where the data entered (prompts, documents) is hosted;
• whether there is a transfer outside the EU and, if so, the legal mechanism that governs it;
• the provider's answers on whether its system complies with the GDPR [3].

Where the tool is designed and hosted is one of the elements to examine. Oupi is designed, developed and hosted in France. For an organisation carrying out this verification, this can simplify the analysis of the transfer question, since the location of hosting is a criterion to assess. This does not amount to automatic compliance: the assessment remains the company's and its advisers', in light of its own uses, purposes and the contractual guarantees obtained.

Is a data protection impact assessment (DPIA) required, and how should the processing be documented?

Every controller must document the compliance of its processing, and the DPIA (data protection impact assessment) is one possible medium. The CNIL sets this requirement under the accountability principle: documentation must be "adequate", "for example in a data protection impact assessment" [9]. This wording appears in its guidance on analysing the status of an AI model. The DPIA can therefore also record that analysis: does the model memorise personal data from its training, and does it fall under the GDPR [6][9]?

When is a DPIA required?

A commercial source claims that the CNIL applies a practical rule: processing meeting at least two criteria would be presumed to require a DPIA [31]. This cumulative-criteria rule is not established here by a primary source. The lists of criteria and the threshold for the obligation must be checked in the texts of the CNIL and of the European Data Protection Board (EDPB), which are not in our file. We therefore do not settle whether a DPIA is required for any given use.

Documenting the processing in the register

Whether or not a DPIA is required, the processing must appear in the register. According to commercial sources, the purpose and legal basis must be recorded there [28], and using generative AI systems constitutes processing of personal data as soon as such data is processed [33].

For a given use, you can record:

• the purpose and the legal basis chosen;
• the data processed, for example what is entered in prompts;
• the analysis of the model's status, where it has been carried out [9].

The assessment of compliance rests with the company and its advisers.

How do the GDPR, the AI Act and new uses such as agentic AI fit together?

The GDPR alone is not sufficient to govern AI, because it is a general law on data, and the AI Act complements it. According to the Agence nationale de la cohésion des territoires, artificial intelligence raises questions that the GDPR alone does not sufficiently cover [4].

Two texts with different objects

For a DPO, a CIO or a lawyer, reasoning must therefore operate on two levels:

• The GDPR applies as soon as personal data is processed. The CNIL reminds us that the collection and use of data through an AI system must comply with the GDPR and with individuals' rights [2].
• The AI Act addresses issues specific to AI that the GDPR does not sufficiently cover [4].

The obligations of one do not replace those of the other. The source file does not allow us to confirm an application timetable for the AI Act based on an official text. We therefore cite none: check the deadlines directly in the text of the regulation.

Agentic AI, a subject to watch

Agentic AI raises new questions for data protection. The CNIL and the CIANum (the French Council for AI and Digital Affairs) have conducted joint work on the issues it raises [8]. This publication is worth following if you plan to entrust agents with tasks involving personal data. At this stage, it does not replace the analysis of each processing operation: purpose, legal basis and documentation still need to be established.

What this means for hosting in France

Oupi is designed, developed and hosted in France. This clarifies the framework within which data is processed, but it does not create automatic compliance: compliance must still be demonstrated, use case by use case.

This article explains public texts and recommendations. It does not constitute legal advice. The assessment of compliance rests with the company, as controller of its processing, and with its advisers.