TCF v2 & Google Errors 1.1/2.1a
Whose fault is it?...
Google now participates in the IAB Europe's Transparency & Consent Framework v2.0, the solution to support compliance with ePrivacy and GDPR, and announces compliance with its Policies and Specifications.
We are glad since Google is one of our trusted partners and we ourselves have been part of the TCF since its start. Sirdata has been registered as Vendor (ID 53) and as CMP (ID 92) since the launch of the standard.
Google has been providing and transmitting the TC String for every ad request since mid-August 2020 and has produced interoperability guidelines to meet the current requirements of Google's policies - in particular those of Google's EU User Consent Policy and its policies prohibiting the retrieval of fingerprints for identification purposes (for example, those contained in its Requirements for third-party ad serving).
In order to give publishers time to manage errors and misconfigurations related to the launch of IAB Europe's Transparency & Consent Framework v2.0, Google produces detected TCF errors reports and allows a 90-day grace period to resolve these errors.
It is in this context that, to our knowledge, many publishers, using a CMP compatible with TCF v2, have received alerts from Google in their AdSense or Ad Manager console.
Despite the quality of our solutions, our partners have not been spared. In most cases, these are the 1.1 and 2.1a alerts that we study below:
According to Google's documentation, class 1 errors always disallow ads broadcasting in response to requests and do not provide a grace period.
These errors always take precedence over other types of errors, even if a given query has multiple errors.
Here is the definition of error 1.1 and related errors:
"Error" 1.1 occurs when Google's ad tags are called up without (before) the user giving his/her consent for Google as a Vendor (ID 755 in the TCF).
This may result in:
- a non-presentation of Google as a partner of the publisher in the CMP (Vendor)
- a user's choice to reject the Google Vendor
When a site loads an AdSense script (adsbygoogle.js) without consent for Google (Vendor ID 755 in the TCF) it contravenes the Google Policy on EU User Consent, which require the collection of consent for cookies and for the processing of personal data by Google and its partners to serve personalized or non-personalized advertising.
However, this is not an error in the integration of the CMP but a problem in the conditioning of Google's advertising tags.
Google acknowledges, about CMPs, that "These solutions can be configured in different ways..." and "It's important to note that such configurations won't control the data collection or cookies on your site automatically."
The TCF is architected on the principle of Privacy by Design and allows conditioning at all stages by each actor involved: a Vendor has the ability to request the CMP API to determine its consent status and must act accordingly. Tag loading conditioning is a good method, but conditioning the operation of these same tags by themselves is much more secure.
This is why TCF does not require a site editor (Publisher) to condition the tag calls, but rather to make a signal available via its CMP to each Vendor to allow him to control what he can or cannot do and act accordingly.
Indeed, the TCF Policies specify as follows:
TCF requires publishers to allow each Vendor to control its status via the TC String ("tcString") provided by the CMP API or via an upstream Vendor who has obtained it from the CMP, and so on.
In the same manner, thanks to this TC String, a Vendor will be able to check the status of another Vendor downstream to determine if he has consent for the trackers and an established legal basis for the purpose before transmitting data to him.
This is what Google does when it decodes the TC String to determine to which partners it will transmit a bid request: for example, it will not send any data to a partner who does not have consent for cookies in France.
We were interested in the functioning of Google's scripts... In order to activate AdSense on his site an editor must load the following script:
To incorporate Google's consent request, we and our partners have followed the ad code examples in the Google Help section of Google's European Union User Consent Policy:
We have implemented these recommendations and automatically run the following command during CMP initialization, to pause AdSense before finishing loading the CMP and interact with the user if necessary:
Then, once the CMP is loaded and the user's choices are established, we launch the following command to allow Google to retrieve the consent signal according to the TCF specifications:
Despite this hold and without any consent information given by the CMP, the adsbygoogle.js script will load other scripts that allow it to access its advertising cookies:
This last script interacts precisely with the CMP in order to retrieve the user's choice status and verify that the user has given his consent at least to Google (TCF Vendor ID 755) and at least to the placement of cookies (TCF Purpose 1).
Here is the function with which Google uses the information retrieved from the CMP:
and where the function Gy is defined as:
Meaning:
- If tcData does not exist (consent management outside TCF) advertising is served
- If GDPR does not apply (e.g. user not hosted in Europe) advertising is served
- If the TC String is equal to "tcunavailable" the ad is served
- If the user has given his consent to Vendor 755 AND Purpose 1 (OR if Purpose 1 is not mandatory according to the publisher and the publisher is in Germany) advertising is served
Then the URL calling the banner is built and contains different variables including potentially gdpr (1 or 0 depending on whether the user is subject to the GDPR or not), gdpr_consent (the TC String, if gdpr=1), and tcfe (the Google error code) if any:
If there is no error and the consent test for Google and cookies is positive then this script triggers the ad call via an url of this type:
The answer then includes the banner code.
In this bid request, you will find:
- The gdpr parameter which must be equal to 1 for a user located in Europe
- The gdpr_consent parameter which is the encoded form of the choices made by the user. The value is called the TC String and can be decoded at https://iabtcf.com/#/decode
If there is no error and the consent test for Google and cookies is negative then this script triggers the ad call via a url of this type:
This call does not get a banner code in response.
The difference between the two calls is in the TC String that Google receives in the parameter gdpr_consent and which, once decoded, will tell this vendor, in the first case that it has consent for the cookies, but not in the second.
In the first case, there will be an answer:
But not in the second one:
It should be noted that, in both cases, the banner call that is initiated by the show_ads_impl_fy2019.js script gives Google access to its cookies:
As such, Google carries out this processing without justifying prior consent for cookies in the TCF, and cannot rely on Special Purpose 1 (Ensure security, prevent fraud, and debug) as a legal basis.
The adsbygoogle.js script integrated by the publisher will therefore trigger the insertion of a second script that will allow the controls to be carried out within the TCF: if the test on his ID (755) or the test on the Purpose 1 fail, this Google script detects the unavailability of consent... And still triggers a third tag call that contains trackers and personal data.
Although the answer is empty, the call itself is troublesome: even if Google shelters behind the obligation it imposes on the publisher to obtain consent before integrating the first script (adsbygoogle.js), it cannot ignore the fact that the user did not give his consent when the second one was executed.
By initiating requests in this way to at least log and subsequently report class 1 errors, we believe that Google has not properly implemented the necessary Accountability measures.
By deploying a TCF v2-compatible CMP and working with vendors who are committed to signal compliance, publishers are making an effort in conditioning and compliance.
The fact that Google's script ignores the client-side test result and triggers the last call despite the lack of consent signal received, is not in compliance with TCF Policies in our opinion. Even though Google has control on this type of conditioning as it implements it for its own Google Consent Mode program that it launched on September 3, 2020. Pure calendar coincidence...
From a contractual point of view, errors 1.1 are imputable to the publisher who does not condition its tags and calls Google without consent, regardless of the TCF and the implementation of our CMP.
In theory, such a conditioning is enough to make error 1.1 disappear... In practice, there are unfortunately situations that the editor does not control and that disturb adsbygoogle.js or gpt.js tags conditioning: if a third party script only injects one of these two tags on the page, it will trigger the advertising calls in an uncontrolled way despite the publisher's effort.
For all of these reasons, we reasonably believe that Google is not complying with the TCF Policies by triggering its banner call tags even though it is advised by the CMP that it does not have the necessary consent.
If Google wants to log the implementations of its tags without consent, we suggest that it does not process any personal data at the same time and that it uses tags for this purpose on a distinct domain for its advertising activities and which does not carry cookies.
We are happy customers and satisfied users of many services provided by Google galaxy (search engine, Google Pro suite, Google Cloud, Marketing Suite...). In all these areas and especially in digital advertising, Google offers one of the best technologies in the world and seems to place efficiency above all.
In our own developments, we prioritize the interest of the user. This gives us a certain skill in understanding and implementing the interactions between technology and privacy. We would have appreciated being informed of these test protocols, which we felt were poorly expressed when we prepared Google's arrival during the TCF v2 transition.
We would be happy to help Google clarify relevant documentation and error messages communicated to publishers to provide greater transparency about who is responsible for what.
Indeed, despite Google's editorial effort and the density of information produced on consent, we believe that publishers are not fully aware of what Google expects.
Nor do they all seem to perceive the scope of the regulatory changes and the articulation of the different laws (Informatique et Libertés for example), Directives (ePrivacy...) and Regulations (GDPR...).
For lack of understanding, the reaction is generally to imitate what Adtech leaders do on their own assets and this is why they can be inspired by Google's search engine which displays a results page of this type to a new Internet user:
In addition to the results of his search, the Internet user is displayed:
- an information message explaining that Google uses cookies and invites him/her to accept them.
- advertisements ("ads").
A quick console consultation shows that cookies were placed when the page was loaded without the User having clicked on the "I accept" button, and in particular the NID and ANID cookies on the google.com domain:
These cookies are used, according to Google's Privacy Policy and Terms of Use, to record users' searches on Google's website and their visits to an advertiser's website in order to display personalized ads on Google:
Other cookies are placed, one of which is named "CONSENT" and takes, at the first loading of the page (before Google's consent messages) in our test, the following value:
Then once the consent is given its value changes to "YES":
Cookies that can be used for advertising purposes are placed without (before) the user's giving consent to Google, which in our opinion corresponds to the definition of error 1.1.
We are confident that Google's methods and tools are respectful of privacy and the regulatory framework. But we understand that publishers who are asked by Google not to do so and who are blamed for these "mistakes" are confused or even lost.
Google should not, in our opinion, simply ask its publisher partners for prior consent to use its ad tags. It would be more judicious to fully implement TCF, and its methods, and thus condition the final banner calls according to the Internet users choices, even if the initial scripts are injected systematically, by mistake or by mimetism...
II- Error 2.1a
According to the Google documentation class 2 errors are related to the grace period 0 which applies as follows: During the first 30 days of the grace period, publishers will be able to resolve misconfiguration issues without consequences on monetization.
During the remaining 60 days of the grace period, non-personalized ads will be served, regardless of the settings of existing personalized and non-personalized ads.
Once the grace period has elapsed, no ads will be posted in answer to ad requests.
To understand this error code, it is necessary to know how the TCF works and in particular the possible CMP statuses:
During a correct loading sequence the CMP status (cmpStatus) should therefore start at "stub" when it is initiated, go to "loading" when it loads, and finally to "loaded" once it is loaded.
If for some reason it experiences a problem when loading or afterwards, then cmpStatus will be set to "error".
So with error 2.1a, Google informs editors of the number of times:
- Where the CMP has been in error
- Where the CMP remained blocked at initialization
- Where the CMP remained blocked during loading
For some publishers, however, the number of errors was very high, much higher than the error or crash rates measured in our tests.
We have therefore further investigated and isolated two fundamental problems: the first is related to Google's tags and the second to third party tags.
1) TCF implementation by Google vendor
Google's tags have been adapted as explained above to decode TCF signals (priority over other Google signals unless more permissive).
It appears from our investigations, that Google seemed to prefer efficiency to user's choice respect by setting a timeout of 500 milliseconds for the CMP loading among all the scripts on the page: at the end of this timeout, the script mentioned in the first point (adsbygoogle.js) stops the process and returns the error tcfe=1 and the value "tcunavailable" for the parameter gdpr_consent.
This timeout is not the time allowed to the CMP alone but to all scripts running in the allocated 500 milliseconds time frame, which is too narrow, and each interrupted CMP load is counted as a 2.1a error by Google.
We believe that timeout is at the root of most of the errors that constitute this class and that our CMP is not in error in most reported cases.
The problem is that the Google servers that receive this call will not read the consent signals but the information "tcunavailable", the response may include an advertisement.
During our tests, we have seen banners that use cookies and personal data:
A click on the information icon leaves no room for doubt: the ads are personalized.
This is in accordance with the explanation provided by Google: "during the grace period despite this error 2.1a personalized ads will be displayed for 30 days, then non-customized ads will be displayed for another 60 days".
From the publisher's point of view and for Google, the effort is laudable: everyone's income is maintained at least to a large extent. But from the Internet user's point of view, it's more problematic: during our tests, all these personalized ads were displayed even though we had previously refused cookies and all Vendors including Google.
Google's Ad Technology Provider partners may also have received and processed personal data involuntarily.
The Google script does not wait for the loading completion and the eventStatus "tcloaded" because of the timeout. For this reason, the script does not systematically receive the CMP signal or the corresponding TC String but still uses its trackers and personal data. So potentially without consent or even in spite of a refusal as in our tests.
Taking into account the usual statistics, it can be estimated that 10 to 15% of the 2.1a errors globally counted, masked refusals and wrongly triggered targeted advertising. Considering Google's weight in online advertising (2 million publishers are members), even if all of them have not yet implemented a TCF v2 compatible CMP, this probably represents hundreds of millions or even billions of impressions!
We therefore believe that, as it stands, this timeout can represent a very important danger for the respect of privacy.
We also believe that by interrupting the process before the CMP is loaded and thus not receiving consent status, Google is in breach of the TCF Policies and in particular these articles:
2) Third party tags
The second problem is related to third party tags on the pages (neither a tag from the publisher, nor the CMP, nor Google) that disrupt or even corrupt the TCF process. Many slight errors will trigger latencies, but in some cases it is more serious: the CMP API is overwritten by a fictitious API that comes to modify or generate consent signals.
III- Business impacts and evolution
Suggestion to editors
Many publishers are discussing with each other and with us about privacy rules. It is not that Google did not inform publishers that it is necessary to obtain consent for cookies and Google prior to any invocation of their ad tags, but rather that the way in which the information was written and made available to publishers did not allow them to properly understand their obligations.
The market is currently in a particular temporality: many publishers who have, wrongly or unintentionally, invoked personalized and cookie-based ads for 100% of their traffic in recent years are discovering that they need to equip themselves with compliance tools or measures and that the economic impact is significant.
Revenue drops of 10% to 50% are reported to us.
As a result, many publishers who do not feel they must implement these systems are tempted to continue to systematically call on targeted advertising tags, or even to go backwards after deploying a CMP for example.
This would be a serious miscalculation, and we suggest that publishers who are thinking about it should forget this idea and accept to lose part of their revenues today rather than all of it tomorrow.
Indeed, with the forthcoming publication by the CNIL of its new Trackers Recommendation, we are entering a new era of controls. The placement of cookies without consent is visible in the browser console without any additional software required and particularly exposes the actors involved, i.e. those in charge of processing.
In the relationship between Google and publishers, Google seems to be most of the time Data (Joint) Controller, which means that in case of a CNIL control, the publisher who allows Google to place cookies will be concerned, but so will Google.
We believe that obviously Google cannot take such a risk and will put in place measures to avoid such a scenario. In the information that Google delivers to Internet users a mention appears besides:
Google announces that it will unilaterally close hundreds of thousands of accounts that, among other things, misuse the personal information of Internet users.
It would be very easy to detect which publishers are not complying with the rules, and the most exposed in our opinion are those who use both advertising and analytics tools.