*35c3 preroll music*
Herald: This talk will be held by Damon McCoy. He will be explaining online U.S.
political advertising and he has been working with researching like how
different online communities basically behave around many different topics. But
this is what he's going to talk about today so please give him a great round of
applause. *Applause*
Damon McCoy: Thank you everyione for coming. I'm up here speaking and I'm the only one that
wanted to fly to Germany over Christmas and New Year's. However there were three
real people that were really key in helping out with this research and before
we get started, I just want to credit them. One is my grad student Laura
Adelson. She did a lot of the analysis that you're going to be seeing generated
all the graphs. One of the undergraduate students that's from our NYU Shanghai
campus secure did a lot of the work to collect all the data that you're going to
see here. And then Raytown who is a professor at NYU and the Shanghai campus
also helped out with kind of our initial efforts of collecting some of this data.
And so before we get started I guess I'll give a little bit of an introduction about
myself. I'm a professor at NYU tenant school of engineering. As was mentioned
before I do a lot of stuff kind of looking at how technology kind of impacts the
security and privacy of you know society groups of people and things like them. So
this was really kind of an opportunistic project that kind of captured the impact
of online advertising in the political sphere of U.S. campaigns. And also quick
plug. So everything that I'm going to be showing you most of the data scripts and
things like that we've put in a github that's accessible by anyone that wants to
analyze the data or look at our scripts and improve them or things like that.
*Applause* Thank you. This is the first time that I've given this talk outside of
the U.S. so let me just start with some quick explanation as to how U.S. elections
work for those of you that might not know about this. So every two years in the U.S.
we hold federal elections. These are elections - right - that impact all of the
states within the U.S. And so every four years we have an election for president,
2018 were our last elections. This was not a presidential election year so the
elections were for the Senate and the House seats at the federal level. And then
we also had elections for state and local positions as well with in here. And some
of them that are captured in our data especially our Facebook data not so much
in our Twitter and Google add transparency data that we have here. So this will be
focused this talk on the 2008 elections that happened on November 7th. Election
day is always the first Tuesday in November in the U.S. every two years. So
to begin with in the background right probably some of you know about this, some
of you might not know about this, but in the 2016 elections which were a
presidential election year there was right this election interference that happened
here. And so Facebook has released these ads. These ads were paid for by a Russian
company the Internet Research Agency that ran these ads. And Facebook released these
to - right - the Senate and then the Senate re-released these publicly to
people. And so this is an ad basically trying to disenfranchise people from
voting in the elections. And you can see right it's targeted at people in the U.S.
of a certain age range and interests like Martin Luther King, African-American
culture, African-American civil rights. Facebook doesn't actually allow you to
directly target to people based on like their ethnicity. So this is a pretty good
proxy though. If you target these kinds of interests for this, so this would probably
be fairly effective at targeting African- American people within the U.S. for this
ad to try and disenfranchise them from voting in the elections. There were other
ads , right, that tried to do misinformation distance information kinds
of campaigns. So this was an ad that was again paid for by the Russian agency that
was trying to perpetuate this rumor basically unsubstantiated rumor that Bill
Clinton has this ill legitimate child within here. And again, right, the
targeting information is targeted at African-Americans within the U.S. and
African-Americans are kind of a key voting block for a kind of more liberal,
democratic people within the U.S. oftentimes. That's probably the other
thing that should have explained about the U.S. election system especially at the
federal level is that we effectively have two parties that are you know when any
meaningful amount of elections within the U.S. and one of them is the Democratic
party that tends to skew more liberal. So they're more kind of right for bigger
government, more social services and then we have the Republicans which skew kind of
more conservative wanting kind of a smaller government providing less services
and kind of less regulation around things as well. And so right these are two
examples but there were a whole bunch of these ads that were shown on Facebook
within a year. So right. Pretty much all of these that they're tried to
disenfranchise people or tried to kind of create chaos, kind of polarize people
around the election oftentimes with kind of disinformation sorts of that things.
And so in 2017 our Office of the Director of National Intelligence put out a report
- sorry for the big block of text, this will be the only big box of text in here.
But I thought it was kind of important to show this because they they pretty much
unequivocally state that Russia tried to interfere in the US elections and that
Vladimir Putin was somehow involved within this interference. And so this is - this
is pretty much as far as the National Security Agency, the CIA, NSA pretty much
solid evidence that they have - that this occurred within here. And so the other
thing that broke was right the Cambridge Analytica scandal as well broke within
Facebook where there was this straight third party advertising agency that
collected you know a whole bunch of data on 80 million profiles within Facebook and
then tried to create psychological profiles for targeting and messaging and
things like that around here. And so these two particular scandals broke within here
and the first result of this is we have Mark Zuckerberg in a suit a real suit not
a hoodie suit. Testifying right in front of our Senate within a year. And so he,
right, he testified before House and Senate committees about the abuses
occurring within Facebook. And he did this on April 10th and 11th 2018 within here,
and right. So this is right. In here, he did admit that Facebook had made mistakes
and that they they need to improve things moving forward within their platform. The
most kind of tangible outcome from these testimonies were these transparency
archives that began to appear. And here is a view of what Facebook's add transparency
archive looks at. When it originally deployed you needed a Facebook account to
interact with it. Now Facebook has dropped the requirement so anyone with Internet
access, unless you're censored somehow, can go to this archive and access these
ads. So the user facing portal for these add transparency archives you type in
keywords and then it basically does is write a pattern matching on the ad text
and other parts of the ad and then returns the ads that matches that within here. And
so then you can see all the political ads that match that particular term within
here. Facebook began archiving these ads kind of at a large scale starting on May
7th 2018 by Election Day November 7th 2018. There were 1.6 million ads paid for
by over 85 thousand advertisers within Facebook's platform. Facebook is actually
fairly broad as to what they included within their political archive. They
included any ads related to US elections, either federal state or local elections.
They also included these very important kind of issue ads as we saw when we looked
at the Russian interference a lot of times the ads didn't mention actual political
candidates, they mentioned kind of polarizing issues within the U.S. So
Facebook also included these ads of political national importance. They had a
list of I think about 13 different criteria is the last of them being values.
And so it was a fairly encompassing set of ads they tried to include within their
archive. Along with the text and images or videos of the ad, they also included
basically ranges of geographic impressions and demographic impressions. So right.
State level impression information in some kind of ranges and demographic by gender
and by age kind of bucketed within here. And they did this again for impressions
and then for they also included some spend information again kind of in ranges so
they gave ranges of 0 to 99 dollars, a hundred dollars to say like 500 dollars,
501 dollars, 2000 dollars and so on. And so forth within these buckets one of the
key pieces of information that they did not release was the targeting information.
So like I showed you before of those ads they - right - they have that targeting
information, Facebook does not release that within their transparency archive.
They have this right. They have right- thay had that user portal where you could
do the keyword search from within there. However right. I'm I like to do large
scale data analysis and so I wanted to basically try and collect all of the ads
within this web portal. And so initially all they had was this keyword search
portal within here. And so what we did is we compiled kind of a large list of what
we thought were reasonables of keywords, names of prominent politicians, names of
states issues within here. And so we tried to compile this long list of keyword
searches and we began scraping the reporter within here and I'll tell you the
story of how our scraping efforts went. Now currently they are so off for a API
it's still keyword based their API and it's restricted by an NDA so I'll kind of
flesh out the story of how this goes. So at the beginning they, they released this
kind of towards the end of May the user archive and I played with it and I
realized that this didn't lend itself well to kind of large scale analysis of these
ads and so on. I went to my students Secur and Laura and Secur worked kind
of furiously night and day and within three days he had a workable scraper that
was able to put in our keywords and then we were able to scrape all the results
from our keyword within here. And so we ran the scraper for about 2 months and
then we released a report. Just kind of a very general statistical report and we
released the data in our github archive at that. After that about 2 weeks later
Facebook began anti-scraping measures within here. And so, right, this kind of
hampered our efforts to scrape Facebook's archive. At this point. I'm - I don't want
to attribute any malice. I don't believe that Facebook was targeting just our
scraping efforts they were targeting everyones scraping efforts. The
transparency whether it's wrong or right to block people from collecting data on a
transparency archive I might kind of quibble with them on that and say they
might once provide better access to the data within their transparency archive.
But this was the choice that Facebook made to kind of clamp down on the scraping
within here. So we tried to fight with them a little bit to - right kind of a can
mouse game. You know we make some changes to our scraper to avoid their anti
scraping. They do some things on their end to block our scraper and probably other
people scrapers that are doing similar things to us as well within here. And so
this persisted for probably about 2 weeks, and then Facebook basically deployed their
API within here. However they said right, their API is very limited and still in
beta at this point. So these were part of the terms and conditions from here. One of
the ones that I found kind of the most unease (?) is that it limited it only to
U.S. people so we could essentially only very closely work with U.S. people within
here and at least it did kind of - it limited the types of people that we could
work with in here. And so right unfortunately this kind of ruled us out
from working closely with journalists from you know really good news organizations
like the Guardian and so like that just happened to have the misfortune of being
located somewhere outside of the U.S. within here. Maybe the good fortune, yes.
And then the list of restrictions continue. They also placed the data
retention on it so we could only retain the data for one year. Again placing data
retention. So Facebook's data retention on their archive is 7 years within here but
they're placing a 1 year data retention on the data that we collect from their NDA.
I'd like to say that - right - we - right - I got this NDA and I lit it on fire, I
tore it up and we continued to scrape the archive. Within a year. No unfortunately
it was a hard call to make but right you know there's basically two students and we
basically had to make a call whether we wanted the data to analyze or whether we
wanted to spend all of our time kind of faded - fighting with Facebook's anti-
scraping efforts. And so in the end we did - I did in fact agree to their NDA within
a year. So the initial data we scraped, we release were still scraping a small amount
of data that we do release as well from here. But unfortunately at this point any
of the data that we collected from the NDA we cannot release within here. If anyone
doesn't want to fight with Facebook and resurrect the crawler within here I would
be more than happy for that to happen within a year. Unfortunately given our
engineering constraints it just simply wasn't feasible for us to do that within a
year. And so the story is a little bit different with Google. So Google's archive
they began archiving ads on May 31, 2018. By election day they had 45.000 and from
600 advertisers. Their criteria for introducing advertising was much more
narrow than Facebook's, so they only released ads related to U.S. federal
candidates and federal office holders within here. So it is a much more limited
set of data that Google released within a year. None of the issue ads that Facebook
released. They didn't release any of the geographic or demographic data by
impression, they did release ranges of impressions and ranges of spend data, and
they did release some limited targeting data from here so they released geographic
and demographic targeting information which Facebook hadn't released in their
ads. And their data is available through a similar keyword based portal. But they
also make it available through just a database, if you want to within here. So
this is what their portal looks like within here. And this is - right - their
big table, sorry, their big query database that they released from here. And so they
updated every week within here and you can download it and analyze the data
relatively easily within here. So the last one to kind of implement their archive was
Twitter. Twitter began archiving ads on June 27, 2018. The scale of ads and
Twitter is very small compared to the rest of them. The scale of their ad network in
general is much smaller than Google and Facebook's. And what they included, it was
similar to what Google included in terms of - right - only federal candidates
within here. Kind of closer to the election, they also said that they were
going to release political issue ads. However, the mechanism of enforcement
doesn't appear to exist within Twitter's system. There doesn't appear to be anyones
job it is to actually enforce transparency of ads from here. So we've been kind of
manually finding accounts and reporting them to Twitter within here and then when
we manually report them to Twitter, Twitter then includes them and future
transparency kinds of efforts within here. But it appears like we're basically the
ones *[Damon McCoy] short laughter* it's become our job to monitor the Twitter
accounts and then notify Twitter and then they'll manually kind of deal with it.
Unfortunately, they still don't appear to have a person that actually manages this
process internal to Twitter at this point. Twitter does however release the most
information. So they release exact data not the range data on impressions and
spend information, also by geographic and demographics and they also include all of
the targeted information as far as we can tell and their data is available through
without an account. Basically through their portal and we've been scraping them
and there's been no problems they haven't blocked us at this point. So we just
simply scraped their data and then we republish it to github at that point. And
we've had no problems with Twitter in this way in the scale, their data is so small
that it's been relatively easy to keep pace with it at this point. And here's
just a picture of the Twitter transparency archive and again this have a list of all
the Twitter accounts that they include in their transparency archive. So we can
monitor this and then we can monitor other people that we know that are politically
active when we see them doing paid advertising then we can notify Twitter and
then Twitter will include them in their transparency archive normally within like
a week, or so of here. And so this is, this is kind of the background that you
need to understand the transparency archives. So now we have a data set that
we can begin to analyze within here. For Facebook since it was the keyword driven
thing at the beginning and it still is, we were able to collect about 80% of the ads
in Twitter's database from there. The other problem with the API is that it is
severely rate-limited at this point. I'm talking about 3 to 4 queries per minute
that we can get through Facebook's API at this point. And so we kind of did our best
effort to collect as much data as we could from Facebook. About two weeks before the
election, Facebook began releasing a transparency archive that included
basically an aggregated list of all the advertisers and how many ads they have and
how much spent and this is how we can tell that we got about 80% of the ads from
Facebook's archive based on this within here. And the nice thing about the
transparency report is that we could go back and now that we know we're missing we
could readjust our usage of the API and so now we have virtually 100% coverage of
Facebook going forward within here. Twitter - right - we could collect 100% of
their data. And again we've republished the SOL (?) in an easier to process kind
of form. Google again - right - we have a 100% of their ads because they're all in
the big query database. However when we started analyzing the data we noticed that
for a lot of the ads we're missing the actual content, the images and text of the
ad. It turns out that for Google's ad network if the ad was originally purchased
through a third party advertiser and then run on one of Google's properties the
content of the ad won't be archived within your system. This is unfortunately a big
loophole. So - right - if you're if you're running a kind of malicious misinformation
thing, you can easily unfortunately circumvent Google's archive at least from
archiving your content by simply just paying for it by a third party within
here. It's unclear whether this is a policy limitation or whether this is the
technical limitation on Google's part, but the outcome is that we only have the
content for about 70% of Google's ads that were paid for directly on Google's
platform and within here. So one of the first things that we want to do is kind of
add some semantic meaning to these ads a kind of large scale. And so we played
around with a few techniques, some fancy kinds of natural language processing and
things like that. But we found that there's actually a really fairly simple
and effective way of categorizing kind of the intent of the ad, and that's that most
of these ads have a URL of some kind and a lot easier or else just point back to
like third party services like if you're holding some kind of event you're going to
coordinate it with like Everbright or something that if you're seeking
donations, if you're a Democrat you're going to use this third party Paron
processor they're called Act Blue, if you're Republican there's like two or
three payment processors that you're going to use for this. So we could simply just
look at these really prominent URLs that occur a lot of times and just kind of
manually tag what is the purpose for this. And by doing this we can tag ads as either
just purely informational that they wanted just kind of get some kind of message
about the candidate either positive or negative out their connection ads that are
seeking contact information like people's e-mail addresses, phone numbers, names and
things like that. Presumably so they can - you know - either get them to volunteer or
donate money in the future for the campaign. There's move ads that are either
they're trying to get people to vote or to attend some kind of rally or to volunteer
or something like that and then right there's donation ads. And then finally
there's kind of commercial ads. These are things either they are selling products
that are kind of directly critical nature like a bobble head of some candidate or
they might be like solar panels which have tax credits in the U.S. and things like
that. So there's some kind of commercial good that's linked somehow to some
political messaging within here. So we use this method and we were able to categorize
about 70% of the ads, we took a random sample of them, we manually checked what
we were doing and we found it was pretty accurate. About 96% accuracy we got using
this methods. The other thing that we did is for the top advertisers, so for
Facebook the top 75% of the advertisers, for Google the top 80% of the advertisers,
in terms of the money spent by the advertiser. We went in and we manually
categorized what was this type of organization. Was it a political
candidate, was it what's called a political action committee, so these are
the PACs within the U.S., was it a union, was that a for profit operation,
was it a non-profit operation. So on and so forth and so we wrote like some regular
expressions that got us most of the way there. Most of them have fairly uniform
naming conventions and for the ones that we couldn't kind of automatically classify
we just did it manually, within a year. And then since Twitter had so few
advertisers, we just did these all, manually, within here. Now, right, we can
start to do some analysis. So the first analysis that we did, the easiest
analysis, was we looked at the size of the ads. And the thing that pops out is that
the majority of ads on all the platforms are between $0 and $100 dollars. So these
are what are normally called the micro targeted ads, that are typically seen by
less than a 1000 people within a year. So these are very short lived, narrowly
targeted ads that are kind of honing in on a specific demographic within here. So
these are these micro targeted ads within here. And it appears, right, that the
majority of ads, especially on Facebook's platform 82% of them, are of this micro
targeted kind of ilk within here. So it's kind of confirms the reporting that people
had of this kind of trend of microtargeting within political
advertising. The other thing, based on our categorization we can look at how the
different platforms were used from within here. The problem with these numbers is
that there was different inclusion criteria within each of these databases.
And then right. Finally, we can kind of look at the different types of advertisers
on these kind of platforms. And again it's hard to read too much into these numbers
because again, right, Facebook included much more of the commercial stuff. So
we're going to see a lot more of the commercial stuff within here. And the the
final analysis of the entire data set that we did was looking at right kind of
basically the ramp up to the election. We cut this off in late October. This
analysis was done for a paper. So the due date of the paper was ironically November
6 within here. So we cut it off a few weeks later and we haven't regenerated the
contents since then. The one thing that you can see is at the top there is that
green Spike. That's kind of the move ads. So right, closer into the election the
campaigns were kind of doing sophisticated get out the vote kinds of ads, within
here. So there were really sophisticated kind of microtargeted ads that get out the
vote. Where like, it was almost kind of spooky where like they knew where the
person lived that they were targeting and so they gave them like directions on how
to get from where they live to their nearest polling place within here. So
there are these really sophisticated kind of get out the vote efforts that were
being run online, within here, towards the end of the campaign. To kind of give you
more of a kind of apples to apples comparison of these different ad
platforms, we also did some analysis kind of narrowing each of the different
advertiser types to the ones that were made transparent by all three platforms,
which were the federal candidates only. And so this can give you some idea of kind
of a scale of these things. And we can see that when we narrow it here we can still
see that Facebook has a lot more advertisers and a lot more ads compared to
Google. However the spending numbers are kind of comparable here. For Facebook
impressions and spends are ranges, that's all that Facebook releases. For Google the
impression data is ranges, however we can get exact spend data, because Google
basically released a weekly report of exact spend numbers, aggregated by the
different advertisers, with here. So we can use that, to get an exact number of
the spend. And again, right, Twitter's numbers are much smaller in terms of
everything, within here. And we redid some of our analysis to just see whether our
effects were simply a distortion based on what was included in the archives. So
right we redid our ad size analysis and even when we limit it to federal
candidates we can see this still holds, that a lot of the ads on Facebook are so
these micro targeted ads. And they are still micro targeted ads on the other
platforms, as well, within here. And right this microtargeting of course varies
depending on the advertiser. So you take someone like President Trump and he does a
lot of microtargeting. So almost all of his ads probably about 90%, 95% of his ads
are micro targeted, within here. You look at other candidates and they do much less
microtargeting, within here. So this is definitely different strategies are used
by different advertisers, within here. But when we look at it in aggregate, it still
appears that microtargeting is a very popular strategy across advertisers. We
can also, right, look at some of the spend type by ad type and this kind of shows you
a little bit how the different platforms are used, within here. So Facebook's
platform looks like it's a little bit more kind of informationally, it's still used a
lot for donations, whereas Google's platform is used a lot more for donations
and a lot less for a kind of informational ads and to connect within here. It's
really kind of hard to read anything into Twitter's data because it's such a small
set of data. But from the data that we do have it looks like there's a lot more kind
of collection of e-mails and things like that, within here. The other analysis that
we did on the federal candidate ads was to look at, that for Facebook in particular
right, we have the geographic impression data from here. So we can effectively look
at how many states were targeted by each ad with a Facebook advertiser. And the
interesting thing here is that right. There was no presidential election. So
basically all these campaigns were operating in one state. So their
constituents for all these elections were essentially in one state, within here. And
so if you look at the inform ads, right, most of those shown a very small number of
states. So the inform ads are mostly being shown to the constituents that are
actually voting for that candidate. However, if we look at that bottom line,
the kind of gold line, those are the donation ads. And we can see that they
were fundraising in many more states outside of their constituency, within
here. So FiveThirtyEight did an interesting analysis of one particualar
candidate, Beto O'Rourke. He was a candidate for Senate in Texas, Texas is a
very conservative state in the U.S., and he did surprisingly well, within here. And
he kind of embraced online advertising and online donations seeking, were kind of
cornerstones of his election, within here. And so FiveThirtyEight did an analysis of
his donation records in the U.S., at the federal level. All donations to candidates
have to be reported to the Federal Election Committee. So this is all in a
database for the Federal Election Committee the FiveThirtyEight people do
analysis And they kind of confirmed what we saw on the donation ads, that he was
getting about 52% of his donations from Texas and 48% from other states, primarily
kind of from coastal states that tended to lean more liberal, like New York,
California, Washington and places like that, was where he was donations seeking.
So this appears to be a very effective way of getting small dollar donations kind of
throughout the U.S. within here, through this online advertising. The last thing that
I'm going to talk about is the ad targeting. Facebook didn't directly
release the ad targeting. However, we were lucky enough and Pro Publica made a
browser plugin, that people can install in their browser, and that's browser plugin
would identify what it thought was political ads, based on a machine learning
algorithm. And for the political ads it would upload these to their server along
with the targeting information. So, for those of you with a facebook account, if
you're seeing ads you can actually click on that ad kind of in the upper corner of
the ad and you can see why is this ad targeting me, within here. And Facebook
will tell you a little bit, not all of why you were targeted for this particular ad.
They will essentially show you the two broadest categories of why you were
targeted for this particular ad, through this feature they've added to their
platform. And this is this is actually kind of interesting, this is something
that if you're a user of Facebook, I highly recommend that you do. Because I
started doing it, and it was kind of eye opening, as to the level of targeting that
was being done in terms of advertising. That's kind of one thing, that we've
definitely learned from this is that when you're seeing an ad, oftentimes there's a
very specific reason as to why you're seeing that particular ad, within here.
And so we felt that it was very important to, as much as we could, understand this
targeting that was going on within Facebook's platform. So Pro Publica had
this browser plug-in and they had this data set that anyone can analyze, with
here. So if you do have Facebook and you're located within the US I would
highly recommend that you install this plug-in, because it helps us to kind of
understand the political advertising in terms of the targeting, within here. So we
took ProPublica's data set and we effectively joined it with Facebook's add
transparency archive, within here. This required us to scrape Facebook's ad
archive, because we needed the ad ID and this is something that they don't expose
to their API, currently, within here. However, they do expose it through their
user portal, within here. So we scraped their user portal to join the specific ads
that were in the ProPublica data set to the archive dataset, within here. And we were able
to join about 75% of the ads from here. There were a lot of ads that were
collected by the ProPublica data set, that just simply weren't archived by Facebook's
transparency archive. It misses things, within here. It's imperfect as to how it
does things. And this would be another interesting analysis to do, to understand
what is Facebook missing in their ad transparency archive and this ProPublica
data set can allow you to somewhat do this, although through bias of who
installs the Pro Publica plug-in in the first place. So we we join these few data
sets, again with the caveat that the ProPublica data set is, right, it's
obviously biased by the set of people that installed it, which are probably not going
to be a normal representative set of Facebook users, within here. But
unfortunately, it's the best thing that we have in terms of a data set that releases
the targeting information, within here. And so we collapse into three different
categorizations of targeting, within here. I'll just quickly explain Facebook's ad
targeting platform for people that don't know about it. So one way to target ads
is, right, through interest or segments, right, age segments, gender segments or
interests like I showed you before, within here. So this is one way to target ads
within Facebook's platform. Another way to target ads is through uploading lists of
information. So you can upload lists of people's phone numbers, people's email
addresses or their names. And then when you upload this list Facebook will find
those profiles within their database, so they'll basically join those emails with
the emails that were entered by the users accounts, and then they'll target these
people. So they'll create what they call an audience of these people through this
personally identifiable information and then they'll target them, through this
method. The final kind of major form of targeting that Facebook offers is through
what they call these lookalike audiences. So this is where you can upload PII
information, like email addresses, phone numbers, names. Facebook will link them to
their accounts and then they'll look at kind of the interests and things that
these users and then they'll find you other users, not these users, but other
users, that have a similar kind of profile to these users within here. So these are
the lookalike audiences that Facebook offers within their platform. And so we
categorized it by this and again by advertiser type, within here. So the thing
that stands out is, right, is that the for profit companies are doing a lot of
targeting based on interests and segments. So they probably don't know who their
people that they want a message to are and they're doing it mostly by interests and
segment. Whereas when you look at the PACs and the political candidates they have
lists. So they have a lot of lists of people's you know email addresses, phone
numbers, names, of things like this. And they're plugging these into Facebook's
system. And this is how they're targeting a lot of people, within here, is through
these lists. And this was expected, but it's interesting to kind of quantify how
much of this is happening. And then the lookalike audiences are also being used, a
good deal by everyone within here. And this kind of makes sense, right? Because
if you have a list of people then you advertise to them but then right you have
this lookalike audience of people that are similar to them that are also perhaps good
people to advertise to, as well, within here. The other thing we can do is break
this down by the intent of the ad here, and this shows the difference even more
starkly, of the difference in behavior between the commercial people and the
noncommercial people. The commercial people are targeting mostly based on
interest, whereas the other people that are, say, looking to connect with people,
they're the ones that are using the most lookalike audiences. And this makes
perfect sense because right the connection ads are there to get people's e-mails,
addresses, phone numbers, names and things like that. So when you use the look like
audiences then you can, right, generate more lists of people they'll convert for
whatever you want and then you can retarget them with the direct lists
targeted ads, later on. So this all makes pretty good sense when you look at how
this is behaving, from here. But again it's interesting, right, kind of make this
transparent for people to understand how targeting is happening within the U.S.
political advertising sphere, within here. So these were pretty much the two major
analyses that we did in terms of targeting, within here. The final part and
the part that kind of makes the juiciest of stories is kind of the more dubious
advertisers that are advertising within these platforms in terms of political
advertising. So we kind of call these more politely kind of "new types of
advertising", within here. The first type is one that you would you would pretty
much expect, so this is this corporate astroturfing kind of stuff, that's going
on, within here. We see these ads for assistance for tobacco rights. And I
pretty much expected that you look up this group and it's probably going to be some
you know quasi nonprofit that's supported by some industry money from the tobacco
lobbyists, or something like that. That's pretty much what I expected to see when I
saw these ads. You go to this website and it's actually pretty honest as to what it
does. This is probably because right of all the lawsuits and regulations around
tobacco in the U.S. in advertising. But the website clearly states, right, that
it's operated by Philip Morris, the tobacco company, within here. And this
actually isn't a legal entity, this citizens for tobacco rights. Is just
simply a website that's been stood up, that's owned and operated by Philip
Morris, as far as we can tell, within here. And this gets to a big problem with
Facebook's transparency archive, which is that they don't actually vet that
disclaimer string of the sponsor, within here. So pretty much anyone can type
anything that they want within that disclaimer string and Facebook will allow
you to run it. We've tested it and as far as we can tell, you can't say that you're
from Facebook, Instagram or that you're Mark Zuckerberg, they'll block that. But
pretty much anything else that you type in there they'll allow that ad to run, within
here, with no vetting. So we discovered this, we politely, privately mentioned it
to Facebook. Some reporters kind of trolled Facebook within here and so there
was a reporter that trolled Facebook and opened up ads for all the senators, within
here, on Facebook. And of course Facebook approved them all, from within here, and
they they did some other things to troll Facebook where they insert some other
advertisements, within here. But the point is, that that disclaimer string is not
vetted within here. Google actually does that disclaimer string within there, so
they require either a tax ID number or a federal election committee I.D. number and
they actually do vet it and they publish that tax I.D. number or federal election
I.D. number along with the disclaimer string, within here. Which makes it really
easy to track down advertising on Google. On Facebook, because right they can
basically type in whatever they want in the disclaimer string, it makes it much
more difficult to actually link these advertisers. And sometimes just outright
impossible, if the disclaimer string is made up or just too mutilated in some way
or form, within here. So this is definitely a problem, where we have these
lobbyist organizations, or in this case not even lobbyist organizations, just
industry, that can effectively lie about who's paying for this ad in Facebook's
platform. The other thing we found were what is now kind of being called these
junk media outlets. So this is for profit outlets that are claiming that they're
doing kind of news operations. But right. It's not really traditional kind of
reporting journalistic things. It's more just kind of propaganda messaging, within
here. So there is this group called New American Media Group LLC. They also ran
the name of New Democracy, or sorry Democracy Now was their other name, within
here. And so they ran this, within here. We tracked down these LLCs and they were
just simply shell companies and that kind of led to nowhere, within here. We worked
with a journalist from The Atlantic that actually did a lot of digging into the
shell companies. And he was able to, through his basically investigation, link
these companies to the actual entity that created these shell companies and was
running these ads, within here. And so when we did our analysis of this, this
company basically this third party advertising company was creating these.
They're meant to look like kind of grassroots kind of organizations. There
were, a lot of them were kind of targeted at more conservatively leaning groups, but
then they would bombard them with liberal messaging, within here. So they would
create these fake communities that looked more conservative. And then once they
attract an audience they would bombard them with these liberal kinds of
messaging, within here. And so this particular company is based in Colorado.
It's called MOTIVE AI. Apparently, it's hoping to become the Cambridge Analytica
of the liberal side. I don't know if that's something to aspire to or not. Some
other journalists also did some digging, within here. There was some journalists
from ProPublica that did some digging, within here. They found more of this
astroturfing by political lobbyist groups and things like that. Big oil insurance
companies, again when they advertised on say Google's platform they would be honest
about their disclaimer string, and then when they advertised on Facebook's
platform they would often kind of obfuscate their disclaimers string, to
make it more difficult to link them together. And so they unmasked a whole
bunch of these other kinds of junk media operations, as well, that were kind of
spreading propaganda, within here. I'm picking on Facebook a lot. Again Google
does vet the tax I.D. number of these people, but you see something like, right,
this DIGICO LLC that paid for some ads. So you track this down, and this is again one
of these third party advertising agency. It's easy to track down because of the tax
I.D. number. But it still doesn't actually tell you who paid for the ad. It just
tells you the third party that, right, it presumably was paid on behalf of someone
else to run these ads, from here. So this is a big problem with these disclaimers
strings, that oftentimes they don't actually identify the person that's paying
for the ad. So to kind of wrap this up, within here, after our kind of experiences
looking at these transparency archives I would say they're fairly adequate to
understand good actors. So we could fairly well understand how good political
advertisers were behaving in Facebook's platform. However, right, for the bad
advertisers, we probably missed a lot of them because they could just simply type
in lots of different disclaimers strings and easily avoid our analysis, at this
point. None of these current archives have it just right yet. All of them have
issues, right. Facebook isn't providing good access to their data. They're not
releasing targeting information. Google is missing 30% the content because of third
parties using their advertising system. They're not releasing spend and impression
information based on demographics, within there. Twitter just simply hasn't hired
someone to enforce the policy of transparency, well, within here. And
unfortunately our experience throughout this process has been that these companies
are oftentimes reactive, instead of proactive, within here. Which means that,
right, we have to continuously put pressure on them, in order for them to
kind of improve these archives, within here. So this is unfortunately kind of the
state that we're in, within here. And I'm sure, one thing that I really want to give
a shoutout, is right there's people at these companies that are actually trying
to build these transparency archives. And I want to give them a lot of credit for
taking on this task, that's probably not well rewarded within their companies, of
building these transparency archives, within here. And so my hope is that by
applying pressure we can get them more support to kind of get more resources and
be able to make more transparent, within their companies, as well. Because I hope
that, right, this puts us in better shape to understand the 2018 elections, but 2020
is another presidential election and my hope is that we'll continue the improved
these archives, so that we'd be in a much better position to understand both the
good and the bad advertisers by 2020, with here. However this is going to take
probably regulatory pressure, legal pressure, pressure by technologists and
things like this to improve these archives, at this point. So with that,
again, I have my collaborators, that aren't here on the stage, but they
definitely did a lot of the heavy lifting to make this happen, within here. And
again all of our tools and most of our data except for the Facebook data, that's
under NDA, is available through our GitHub, there. And so with that I will
open it up to questions.
*applause*
Herald: Thank you so much Damon. I know
that there are a few questions among the audience. So, microphone 6 please.
Question: So [Name] on the IRC is asking "Have you looked at links between the
advertisers and do they use the same images or text for instance?".
Answer: This is a really good question. This is actually one of the analysis that
we're currently doing. So we're starting with the text, because that's obviously
the easiest. But we're also exploring some image clustering algorithms, as well. To
cluster the advertisers across platforms and also within platform because we're
finding a lot where, you know, they create multiple shell companies, where they just
lie about their disclaimers and so this is definitely something that we're focusing
on, is better clustering of the advertisers. Because like that group
MOTIVE AI, even though they created the different LLCs they were running the same
images and videos across their different LLC shell companies.
Herald: Great thank you. Please if you have any questions, queue up by the
microphones. Microphone number 1 please. Question: Hi, Oliver Moldenhauer? Thanks a
lot for the talk. Definitely one of the best I've seen here so far. Two questions.
A: Why do those transparency archives exist? Was there some law or political
process around that? And B: As we are nearing the European election next year,
what kind of data is available for Europe? Answer: That are both good questions.
Again I'm not intern in one of these companies, so I can just speculate as to
why these transparency archive exists. But my my guess is, right, that this was
reactionary. So Mark Zuckerberg and high ranking officials from Twitter and Google
were hauled in to testify in the House and Senate, and this is them trying to self
regulate instead of having regulation imposed on them by people. So that again,
this goes to the pressure part is that there was regulatory pressure put on them,
the threat of regulatory pressure and so that's what made them do these
transparency archives. In terms of what's available in Europe. I guess as long as
the UK is still in the EU, kind of teetering Facebook has started to make ads
transparent in the UK. They also make them transparent in Brazil and they're going to
make them transparent in India. And I think they have plans to make them
transparent in other places, in the EU as well. However, they haven't done that.
However, again this goes back to the pressure part. So there's no API for the
other countries, there's only an API for the US and that might be because we put
pressure on them by scraping them and publicly releasing their data. And, right,
there's no transparency reports for other countries, as well, there's only
transparency report for the US. And again that might have been because we applied
pressure and we were publishing numbers. Some of the numbers in terms of spend were
very low, because, right, they were just giving us ranges. So we might have been
making them look bad, when we took the bottom range their spend and they might
have wanted to correct that with their own transparency archive, as well. So again, a
lot of this unfortunately requires pressure to get them to improve their
transparency efforts. Herald: Great thank you. Microphone number
two please. Question: So you mentioned you mentioned
FiveThirtyEight and their work on the donations. Do you think it makes sense
to combine the data you gathered with what they have to look at election outcomes,
like, election results and turnout and stuff like that?
Answer: Yes. Actually this is the number one project on our road map, right now.
Is, actually Google has processed the FEC information and they've made this
information available via their big query database. So we've downloaded this, we've
manually linked the Facebook advertisers and the Google advertiser to the FEC data
and now we're doing the regression models, specifically focused on the donation ads
first. Because those are what are reported to the FEC, at this point. So we are
essentially trying to understand how effective these donation ads are at
actually driving donations, within here. Herald: Thank you. Microphone number 4
please. Question: Hi. First of all thank you Mr.
McCoy and your team for this very interesting research. I was wondering,
whether you know if there are any follow up research conducted by political
scientists, sociologists etc. analyzing the political repercussions of these ad
campaigns. Answer: Yes, so we're aware of a few
efforts. I don't want to out the teams that are doing them, in case they don't
want to be outed. There's there's nothing that's been published, publicly I believe
on this. But we're definitely trying to. That's one of the main goals of kind of
our overarching online political advertising transparency thing, is to try
and get as much data as we can in the hands of less technical people in an easy
way for them to analyze. And so this is basically the primary goal of our project,
in here. So we've been working as hard as we can to get political science to stay up
to speed on the data. And this is why it's really unfortunate that Facebook has its
NDA in place for their particular data, because this makes it very difficult for
us to share and collaborate in that particular data. Which puts pressure on us
unfortunately as being the only ones that can do some of this analysis right now. So
this is why I would I would love to apply enough pressure to Facebook, to get better
access to their particular data. Herald: Yes. And the question from the
Internet please. Signal Angel: So Nomad is asking "Why are
those advertisements considered political or election interference in the USA. Can't you just
see, that someone paid money to display that content and conclude its purpose is
to promote an agenda or manipulate them?". Answer: This is a good question. Right, a
lot of this goes to the tactics that they're using here. So again they're
creating these communities, that they're making look like their grass roots
communities and then they're kind of sucking people in with these ads, that up
until recently had no disclaimers string on them. So you had no idea who paid for
them. So they appear to be paid for by kind of these grassroots organizations. So
you felt like you were, kind of, part of a grassroots movement, enjoining these kinds
of communities. I think this is the really scary, kind of subtle things. And you
might not realize why you're being targeted for these particular ads or who
was behind these particular ads. So, I think it was really easy for people to
kind of get unwittingly, kind of, duped into joining what looked like these
grassroots campaigns. So that's why I think improving these disclaimers strings
and showing who is really behind these communities and these advertisements is
really important, to dispel this notion of these fake grassroots communities, that
are luring people in within here. So I think that's one of the big things that
can be gained by these transparency archives. But it requires improvement of
the transparency archives, to do that. Herald: Microphone number 3 please.
Question: Yes. So I'm curious about the efficacy of some of the advertisements
that are on Facebook and Twitter. And I'm wondering is any group like the ProPublica
web extension checking the engagement rate? Like the number of comments, the
number of views and the number of shares, to like kind of get an estimate of, OK
this big grassroots community is building up a number of followers and these
followers population sizes and whatnot. Answer: Yeah, this is again a really good
question. This is something that we are, I would certainly encourage other people to
potentially do as well. So the problem is that a lot of that information isn't
exposed by the transparency archives. This is more of what they call kind of the
organic information, the non paid for information, within here. And so this is
stuff that none of the platforms are releasing. And so it requires kind of a
scraping operation, essentially, to gather this information and collect it. And it's
something that we're definitely thinking about how to efficiently do, is how to
efficiently scrape and collect this information. Because this is very hard
because, right, you go against the anti scraping teams of these companies, that
are well resourced. And this requires accounts, and these accounts are going to
be shut down and detected. So this is something that we're trying to pilot to
understand. Our other idea of how to do this potentially is try and crowdsource
this information. This is similar to how ProPublica crowdsourced it for the browser
extension information. We could potentially crowdsource it, where you
know, when people interact with these communities or these ads the plug-in could
potentially crowdsource that information back to us. And then we would have to
figure out some strategy to sanitize that information in some way. Because at that
point you might have some sensitive information they are collecting. This is
something that we're thinking about. We're cautious, I think, rightly so because this
can start stepping on, again, more sensitive information that's available
from within here. But I think it's definitely key to understanding the
effectiveness of these ads. Something that we're going to have to do or we're going
to have to convince Facebook somehow to do on our behalf in order to really
understand the effectiveness of these ads. Herald: Thank you. Last question for
microphone number 1. Question: All right. At the beginning of
your talk you explained how Russia influenced the elections. I'm curious
about the attribution. Is there possibly any doubts at any instance that you
presented that it was not Russia or maybe some other country, China or Iran? How do
you know, and did you check the facts? Answer: I mean, that's a good question.
Unfortunately, right, the national security agencies don't release the
sources of their information. There's another investigation done by the
Department of Justice by Robert Mueller, that did release some more information
about this, within here. I've looked at that information and it looks, you know,
right, you can never a 100%, unequivocally state that it was Russia. It could have
been a false flag operation. But I think that pretty much the overwhelming
information that everyone has found when they've investigated this has pointed at
Russia and the organizations that were prosecuted by Mueller.
Herald: Damon McCoy, thank you very much. Please give them a great round of applause.
*Applause*
*35c3 postroll music*
subtitles created by c3subtitles.de in the year 2019. Join, and help us!