#1263 - Renée DiResta

Mar 12, 2019

Renée DiResta is the Director of Research at New Knowledge and a Mozilla Fellow in Media, Misinformation, and Trust.

Play Episode

ladies and gentlemen welcome to the show reason cuz not only can you use it to pay people back send and receive money very easily you can also buy and sell Bitcoin and you get something called the cash card and that's where things get funky the cash cards the most powerful debit card on the planet Earth because it's the only debit card that comes with something called boosts which the cash app invented so you can't get anywhere else you just select a boost in your cash app swipe your cash card and you save 10% or more at Whole Foods Shake Shack Poteau Taco Bell Chick-fil-A Domino's and coffee shops across the country coffee shops now the coffee shop is fucking phenomenal cuz it takes a dollar off every time you buy a car

people always complain wood coffee cause so much it cost a dollar less when you use the cash card that's including Duncan Starbucks Coffee Bean by 300 cups of coffee a year you save $300 with cash app it is beautiful and it is simple download the cash app from the App Store the Google Play Market and order your cash card today and when you download the cash app enter the referral code Joe Rogan all one word $5 will go to you that's five $3 people go spend it any way you want and $5 will also go to support our good friend Justin Brands fight for the Forgotten charity which is helping build Wells for the pygmies in the Congo to date through this promotion the cash app and you listeners have raised over $100,000 providing clean drinking water to More Than 6,400 People in Uganda and Tanzania and on top of all that some of the funds of help fight for the Forgotten

purchase land for the pot while pygmies in Southwest Uganda bringing six new water well to that region with the cash apps help positive image video the company that shoots all of my comedy specials will make a documentary about Justin's efforts so that everyone can experience his journey so download the cash app use the referral code Joe Rogan all one word and wonderful things will happen for everybody SimpliSafe SimpliSafe home security that kicks ass fantastic home security exceptional home security that is award-winning it's Warden 24/7 protection that protects your home through it all blizzards blackouts and burglars SimpliSafe has won awards from all the text the all the Tek experts why did I want to turn Tech in it

text Travis me awards from a shitload of tech experts The Verge calls it the best home security system it one Readers Choice from PC Magazine it's a two-time winner of the CNet editors Choice Award and a wire cutter top pick SimpliSafe has no contract no hidden fees and no gachas and they always keep their prices fair and honest thanks to SimpliSafe fear has no place in your home

try SimpliSafe with free shipping and free returns you get a 60-day risk-free trial as well so order now and have your home protected within a week go to simplisafe.com Rogan to get started today that simplisafe.com Rogan SimpliSafe simplisafe.com Rogan go there and they will know that you were sent by us get yourself some wonderful security and last but not least we're brought to you by my all-time favorite underwear I'm wearing right now God dammit me and he's meundies BuzzFeed has said this about me on these

they feel like actual Heaven against your skin we're going to assume Heaven is really soft in this context I don't know about all that I would assume in heaven you don't need underwear but let me tell you something about the real world motherfukers in the real world that the best goddamn underwear I've ever put on my body to the best they're made out of something called micro modal fabric fabric it is a full three times softer than cotton so fuck you cotton how about that

the best the best underwear you can get I'm not exaggerating I don't wear any other underwear anymore I used to be stubborn I had some underwear that I bought nope not anymore I've learned my lesson I talked about how I got it evacuated during the fires well when I did that I was only wearing one pair of meundies so what did I do I couldn't go back home as far as Raging all around my my home so I went out and bought some regular bulshit asunder where it made me appreciate me on these even more for the best yes and how about this

they give you a 100% satisfaction guarantee that's how confident they are that's how confident they are beautiful styles and colors and adventurous Prince. A cool shed go to the website check it out and you can get 15% off your first pair free shipping and a 100% satisfaction guarantee when you go to meundies.com Rogan that's meundies.com Rogan is Renee diresta I really heard her on Sam Harris's podcast it's called making sense and I was blown away this is she's done a shitload of research on Russian Bots and the effect that they have on a discourse on communication it's so much deeper and weirder than you could ever possibly imagined I don't want to say any more

I just want to welcome her please welcome Renee diresta

The Joe Rogan Experience

hello Renee hello thank you doing it's really appreciate it and I was utterly stunned I had to listen to it twice cuz I just couldn't do it let's get it let's get into this from the beginning how did this start out how do you start researching these online Russian trolls and Bots and in all this Jazz had my first baby in 2013 and I decided to do was I started looking at Auntie vaccine activity in California because I wanted to put on preschool this is going to fit with the parents basically needs and it started looking at the way that small groups were able to kind of disproportionately amplify messages on social channels and some of this was through very legitimate activity and then some of it was through really kind of coordinated deliberate attempts to kind of game is and I'll grab

for amplifying Content implying particular types of narratives and thought it was interesting and I started writing about it and I end up writing about ways in which hashtag gaming ways in which people were kind of using automation to just be in a hashtag all the time so it's kind of a way to really gain control of share voice and what that meant when very small groups of people could achieve this kind of phenomenal application and what the pros and cons of that work and then this is 2016 so that this sort of

awareness of social media challenges came came about was actually when I was working on this other people were looking at it from the same the same tactics but how they were being used by ISIS by the terrorist organization and they're also you had this very small group of people that manage to use Bots and implications really kind of own a narrative really push this this brand is this digital kalafut to kind of build it on all social platforms almost simultaneously and the ways in which information was hopping from one platform to another through kind of deliberate coordination and then also just ways in which information flows kind of contagion style end up working on thinking about how the government was going to respond to the challenge of terrorist organizations using American Social platforms to spread propaganda but we came to realize was that there was just this information ecosystem and it had involved in a certain way over. Of about eight years or so and the kind of unintended consequences of that and

Russia and turn into the conversation was around October 2015 when we were thinking about what what to do about Isis what to do about terrorism and interest you know kind of proliferation on social platforms this was right around when Adrian Chen has written the article the agency for the New York Times and that was one of the first big expose is of the internet research agency the first time in American journalist on over there and actually met the trolls been in St Petersburg and began to write about what was happening over there and the ways that they had pages that were targeting certain facets of American culture so while we were in DC talking about what to do about terrorists using these platforms to spread propaganda there were beginning to Rumblings that Russian intelligence and in a Russian entities were doing the same thing answer the question became can we think about ways in which the internet is vulnerable to this type of manipulation by anyone and then and then come up with ways to stop it so that was how the other Russian

began was actually around 2015 and a handful of people started looking for evidence of Russian Bots and trolls on social platforms 2015 if we think about social media and the birth of social media essentially it hit only been alive for I mean what was Twitter 2007 I believe so 8 years like eight years of social media and then all the sudden they figured out how to game the system and then they figured out how to use this to make people argue against each other so there was this that if you go back to like remember like geocities and maybe you know kind of thing was great but the internet like internet 1.0 we can call it right was this idea that everybody was giving a platform you could use your platform you could put up your blog you could say whatever you wanted you didn't

attention but you could say whatever you wanted and so there is this kind of consolidation as a social platforms kind of came into existence content creators were really excited about the fact that now they not only had this this access to write their own stuff but they also had access to this audience because as the network effects got more and more pronounced more and more people came to be on social platforms Facebook remember it was like to be another psych Friendster and Myspace and social networks kind of involved when I was in college Facebook was still limited to like you know how it feels like Ivy League schools and so I was even a little boy as you watch this consolidation happen if you start to have this information ecosystem really dominated by a handful of companies that grow very large because they're providing a service that people really want but there's a consolidation of audiences on to the tansel platforms just becomes really interesting for regular people who just want to find their friends rich people

message for propagandists and trolls and in this case terrorist organizations and state intelligence Services instead of reaching the entire internet and I have to concentrate their efforts on a handful platforms so that consolidation is one of the things that kind of kicks off some of the more than one of the reasons we had these problems today on your Facebook or Twitter and Instagram and a couple other minor platforms other than YouTube I mean anything that you can tell it's an actual person like YouTube that is a problem right because you can see he's an actual person if you'd have your narrating something you know if you were in front of the camera explaining things people you're going to know that you're an actual human being whereas there's so many these accounts that I'll go to like how I watch people get involved in these little online beasts with each other and then I'll go to some of these accounts with this doesn't seem like a real person and I'll go and it's like #maga there's a

American Eagle in front of a flag and then you read their stuff this is probably a Russian troll account and it's strange like you feel like you're not supposed to be seeing that so you seen the wiring under the Border something and then you'll go through the timeline and all they're doing is engaging people and arguing you know for Trump and against you know whatever the fuck they're angry about whatever whatever it is it's being discussed and they're they're basically just like some weird little argument mechanism 16 there's a lot of pain right there were there was so much that was written you know we can go back to the free speech thing we are kind of tied him up before they're so much it was written about harassment rolling and negativity these kind of hordes of accounts that would Brigade people and there's a lot of people who are just assholes on the internet

is as we began to do the investigation into the Russian operation in it started on Twitter in about 2014 actually so 2013-2014 internet research agency is targeting Russian people so they're treating in Russian at Russian and Ukrainian about folks people in their sphere of influence so they're already on their they're ready trying this out what they're doing is they're creating these these these accounts it's kind of wrong to call them box cuz they are real people are just not what they appear to be so I think the unfortunate term for it has become like cyborg like semi-automated you know sometimes it's automated sometimes it's a real person but a sock puppet is the other way that we can refer to it as a person pretending to be somebody else have you stopped puppets and they're out there and they're tweeting in 2014 about the Russian annexation of Crimea or about mh17 that plane that went down with Russia and of course I've no idea what happened and it wasn't their fault at all and gradually as they begin to

that's what I imagine they thought it was a success that's when you see some of these accounts pivot to Target Americans and sew in 20 late 2014 early 2015 you start to see the the strategy that for a long time had been very inwardly focused making their own people think a certain way or feel certain way or have a certain experience on the internet it begins to to spread out it begins to to look at words and so you start to see these accounts communicating with Americans and as we were going through the data set switch the Twitter dataset of public anyone can go and look at it at this point you do see some of the accounts that are kind of you know that we're that were somewhat notorious for being really virulent nasty trolls anti-semitic trolls going after journalists to know somebody's account

being revealed as actually being Russian trolls it doesn't kind of the actual American trolls that were very much real and active and part of this and expressing your opinion but you do see that they're mimicking Vester using that same style of tactic that harassment to to get a real people simply make another account the use of some sort of you know what is it a virtual virtual server what is that called trying to do that as long as they want they can continue to make new accounts and it probably also involved in the actual Mara controls because they're going to go out a little bit further than everybody else a little bit crazier and change the tone of discourse Within These communities that are arguing about a certain subjects get nastier and nastier because of the interference of the

trolls igate seems like they've been vaccinated managed to not just cause a lot of discourse but to change the way people are interacting with each other in to make it it just make it more more vicious so what they're doing is are operating in communities so one of the really common criticisms if you know people who live people think that this didn't have a huge impact did it snow in the election we have no idea but it targets is it can change that tone and that's where you see it's amazing everybody probably had this experience your part of a group and then a new person gets added to the group in the dynamic changes and so there's this opportunity to tip to be no kind of expand the bounds of Tolerance just that little bit more or try to normalize using found particular ways of

maybe a group wouldn't naturally gravitate to but then it does so there are there are definitely ways in which type of shift the language of the community shift the the culture just a little bit when did the agency do this and do we have someone who's ever left there or become a whistleblower who can give us some information about what the Mandate was and and how it was carried out a couple of whistleblowers and actually some investigative journalism in Russia they described the employees of these has the internet research agency so it's a little bit like a social media marketing agency plus tactics that we would not expect a social media marketing agency to use things that are a little more like what you would expect to see from an intelligence agency

so besides just making your pages in your blogs on your social posts are also in their kind of connecting with real people and real activists and pretending to be something that they're not to develop kind of a one-on-one relationship but most of the the whistleblowers who have come out there's a woman named Ludmila Salvage truck she wrote an expose I believe on this it's described as being much like you would expect if you were doing social media groundwork you have a certain number post per day you know you're driving trying to get a certain amount of Engagement you're trying you got a better quote has a most people are young Millennials the people that work there are well-versed in trolling cultures are well-versed in Internet culture you know they're up to speed on like popular memes and things like that and so you do see this thing that they do is they talk about in Mueller indictment you see some really interesting descriptions of like the stand-ups that they have stand-up is a thing to do tech companies

talk about your goals and responsibilities in blockers and things and any stand-up they would be sitting there saying things like if you're targeting black LGBT people make sure you don't use white people in your in your image and you're mean because that's going to be like trigger them so trying to get at the very Niche rules for Uno for communicating authentically in an American Community what does you know online and you sometimes they're very specific ways in which a community expects a member of that Community to communicate and so they are in their feelings by Mueller's team and by the eastern district of Virginia the the degree of granularity that they have to recognize that if you are running a black LGBT page and your mimas of white people you're going to cause some tension and consternation and assuming that that's not necessarily what you want to be doing you should go find a meme of black LGBT people to put in the

United to put as Your Meme for the day so there's a lot of there's a lot of sophistication there's a lot of understanding of American culture and then there's a lot of understanding of trolling culture and so these things combined to be a rather effective you know very effective social media agency overwhelming sort of narrative that they're trying to pursue the trying to push I just want to research for the Senate and the Senate data came from the platforms so what I had was the attribution was made by the platforms it wasn't like Renee deciding this was Ira it was the platforms giving it to your government and the

information in there what it showed was that across all platforms across Twitter across Facebook Instagram YouTube they were building up tribes so they were really working to create distinct communities of distinct types of Americans that would be for example there's an LGBT page that is very much about LGBT Pride there is any created it to read it and they created curated it has a another voice like a Persona a lot of the post on LGBT page or written by what sounded like a millennial lesbian was the voice so it was a lot of you know memes of LGBT actresses and they would brand it with a specific brand mark it was a rainbow heart LGBT United was the name of the page that had a matching Instagram account which would also expect to see from a media property right you would expect him to see in both places and this

a young woman talking about crushes on actresses and things actually you knows it was it was really besides this sometimes walking English virtually indistinguishable from what you've read on any kind of like young Millennials focused social page if it wasn't none of it was Radical or divisive it wasn't like the way that they got the division across was they built these tribes were there reinforcing in group dynamics so you'll be LGBT page you have numerous pages are in the black community that was where they spent most of their energy a lot of pages targeting far-right so both old far-right meaning people who are very concerned about what is the future of America look like and then young far right which was much more Angry much more like trolling alter so they recognize that there's a divided there that the kinds of memes are going to use to Target younger right-wing audiences are not the same time

means you're going to use to Target folder right by not answer so there's a tribe for older right-wing younger right-wing and the black community there's a Baptist tribe there's a black liberation tribe there's a black women tribe there's one for people who have incarcerated spouses there is a brown brown tolerably was the name of it page that was very much about Mexican and Chicano culture there was Native Americans United. All of these are fake this was not a short-term thing I started these pages and 2014-2015 timeframe most of them they started some other ones that were much more political later we can talk about the election if you want to but with this tribal thing you're building a tribe so you're saying like as black women in America this is yours posts about things that we care about his post about black hair Hurst post

child-rearing years posts about fashion and culture and then every now and then there would be a post that would reinforce like as black people we don't do this and so we're as LGBT people we don't like this and so you're building this reports with me and you were having a conversation and we're developing a relationship on this page overtime and then I say like as this kind of person we don't believe this so it's a way to subtly influence by appealing to an in-group Dynamic repealing to like as members of this tribe as LGBT people of course we hate Mike Pence as full of course we're not going to vote because you know we hate Hillary Clinton because we hate her husband as as people who are concerned about the future of America Texas secessionists you know everything is presented as members of this tribe we think this as members of this tribe we don't think this a lot of the procida

those were not even political they were just sort of affirming the standards of the tribe so they're kind of setting up this whole long game and then once they got everybody on board a how many followers are these do these Pages have Instagram accounts and I would say maybe

30 of the Facebook page has had over a thousand followers which is not very many and then maybe the top 10 had upwards of five hundred thousand followers so there's no campaign sometimes you've hid sometimes you flops with the flops as you would see them repurpose them so they would decide you know the same way if you're running a social media agency while we've got this audience this page isn't doing so well that's like rebranded a little bit change it up try to try to make it appeal to somebody else Instagram memes and 133000 of them and I was there is a cluster of images of Kermit the Frog is Kermit the Frog doing in here and so are you so the night goes out of the post and then I got a folder the images

turn order to like connect the dots I had to have the image up on one screen in the this something to the csb up on the other screen turned into a database that we could track things a little bit more easily across the platform spot so I have this cost of Kermit the Frog memes and then I realize that the third tributed to an account called Army Jesus that's all that's interesting. I never really raunchy Miss Piggy like to be crappy memes attached to Army of Jesus and what the hell is going on here but he's going through it hundreds of Kermit memes and I get to a post where they say like this page is owned by Homer Simpson out Kermit went to jail for being like I don't know that they made some like some joke was stupid and all of a sudden the data set turns into Homer Simpson memes

raunchy Homer Simpson culture like 900 posts and so the account at some point used to be called nuts news and then they nuts news was what they called it when it was the Kermit the Frog meme page and then it gets a repurposed when they realized, it's not doing it it's not getting the audience they want Homer Simpson's not getting the audience her engagement they want and then they pave it over to Jesus and then all of a sudden they start you know the lights and and things start pouring so what they're doing is there actually like either deliberately or they're just creating placeholders it's kind of a red flag when a brand new account that was created yesterday somebody starts talking about some highly politically divisive thing or whatever but if you lay the groundwork can you do it over. Of 2 years and somebody who goes and checks to see

what the account was where it came from how old it is is going to see something that was 2 years old so it's an opportunity to create almost like sleeper accounts where you create them now and then you activate them you politicize them you'd actually put them to use a couple of years in the future so we saw all kinds of over and over again that turned into an anonymous account at one point on they're pretending to be anonymous Uno the activist so they repurposed this black guns matter page which just had it was it was advocating it black people buy weapons and I'm carrying like a pro-second Amendment page but for the black community and they took that page when it wasn't getting I guess it kind of Engagement and it became was called

oh gosh I remember the exact name of the anonymous page and I don't want to say it was something that's legit but into an anonymous page and when they do that do they go back and repurpose the, the content of the earlier post to they change have more insight into a we could see you again you know you think if you started following an army of Jesus pays you know this raunchy Kermit shirt from like a year ago that would raise some Flags I would assume that they scrubbed and restart it but I don't know if your podcast with Sam changed how I look at a lot of the pages that I actually follow cuz I follow some pages like that have classic cars or something like that and then I'll see the most of it is just photographs of cars like beautiful old cars and it'll have a giant following and then all the sudden something of your political

and I'll look at it and go out wow like this is probably one of those weird accounts like they're getting people to get engaged with it because it represents something they're interested in like classic muscle cars and then they use it for activism they use it for 2 to get this narrative across I think

the challenges is like

you want people to be aware that the stuff exists but you don't want them to be paranoid that it's a sin to see things and I'm like what are the odds but and I try to let you know not feel like you don't know what it's like and sound like Tom Clancy novels

it's it's this balance between when you make people aware of it and I think people deserve to be aware but they deserve to understand how this plays out the flip side of that is you do wind up in these weird and you see it happen on social media now or click into a trump tweet and you'll see like you're rushing and so that's where you get at interesting conversations

you know in some ways getting caught this is this is one of the challenges with running disinformation campaign so I did it makes it really hard for people to know what's real after the fact that leaves you a little bit off balance right feel like gum you know when you feel like you can't quite tell what's real and that's part of the goal is to make you not have a not feel entirely balanced and in your information environment is as real as the snot and so in some ways there's not much downside to doing this because you do a few either knock it out of the park and you influence the election and influence people and you have this secret covert operation going on for years or you get caught and then there is a until there's some confidence in the ability of platforms to detect this stuff There's real concern among everybody that that you're you're in town or something fake

did the overwhelming narrative is that the Russians were very much invested in having Trump win

bright and if they were very much interested in having Trump win with was the reason why they focus so heavily on the African American Community because I have African American Community traditionally seems to vote Democrat so they were trying to do something to break that up or trying to do something to weekend the position of the that you know the incumbent worker Hillary Clinton and maybe put some emphasis on Jill Stein or some alternative candidates

when is the political campaign in the plug last actor that played out today stablished they started building his relationships in 2015 and the other during the poor part of this community and then when you start to see them do is really they're actually there was a tiny tiny cluster in the early primaries were they were supporting Rand Paul and then they pivot to Trump pretty quickly and probably Rand Paul just didn't pull well in there like there's no way to getting a lift your butt but maybe Trump was getting you know some actual left in the media and so move into supporting Trump and then for the remainder of the data set 4 from 2015 through the end which was mid 2017 or so when this thing ends it's it's adamantly Pro Trump on the right and on the right you see not only pro-trump but you see them really working till like a road support for mainstream or traditional Republicans traditional conservatives you see a lot of the memes about like

are you at the conservatives are the conservatives in there you know and so the cops are what is of course they're like they've got pictures of Lake Lindsey Graham and John McCain I hate John McCain John McCain shows up a million times and I exactly why Ted Cruz as is and I believe this is probably true they've really strongly dislike Hillary Clinton because there was concerned that she would you know things that she was saying about increasing freedoms in Russia were very threatening they thought the best bet to get sanctions removed was Trump I'm so they had specific outcomes that they were hoping for and that was one of the units there's always like a political motivation so there is this narrative around they just want to kind of like screw in American society create divisions and play divisions me look at the political content the clear and and sustained support for Trump and even more than that the clear disdain for Hillary Clinton there is not on Facebook and Instagram

there's not one single pro-hillary post there were so many Trump post because if you're running an LGBT page of course they're going to say negative things about Trump you know and and they're saying it so you should vote for Jill Stein there was early support until the left-leaning pages for Bernie Sanders but you actually see the support for Bernie Sanders come in more after it becomes clear that he's not going to win because then they're using Bernie Sanders as a way to say this was stolen from him biting Bill Clinton's or Jill Stein you know he was a true a true independent real liberal I wish to be voting for her if you want to support a woman's head of these feminism Pages really pushing this Narrative of Jill Stein so if the left-leaning pages total ENT Clinton and then you have the right leaning Pages staunchly Pro Trump and also strongly Auntie Cruz and Rubio Auntie Lindsey Graham basically every now it's called establishment Republican and there's this

kind of pushing a people to opposite opposite ends of the political Spectrum this is where you get at the conversation around facilitating polarisation so not just

it wasn't enough to just support Donald Trump it was also necessary to strongly disparage the kind of traditional conservative moderate center right in the course of amplifying the Trump candidacy does that make sense the election and I was trying to figure out is this because Trump is so bombastic and he's so outrageous and he's just a different person that the way I was describing on stage was that like finally the assholes have a king because they never had a king before I like everyone who was running for president was at least mostly dignified I mean basically it's really difficult to go back in time and find someone who isn't find someone who'd there's no one who insults people like he does me and he insult people's appearances he calls them losers he called Stormy Daniels

horse face I mean he has some outrageous shit so part of it was me thinking like while maybe he's just ignited and embolden I actually have this conversation with my wife today she was like it feels like racism is more prevalent like it's more it's more accepted people feel more emboldened because they're in their mind they think he is racist I can get away with more things Trump has president actually videos of people saying racist shit and saying hey Trump's president now we can do that so I was thinking that will maybe that's what it was just sort of like some rare flower that only blooms under the right conditions poof it back right but

when you think about the influence of these pages of had in establishing communities and this long game that they're playing like the LGBT Pages even though they're sitting on Trump they really want to support Jill Stein because they know that'll actually help Trump because it'll take both away from Hillary Clinton that they it seems different like political discourse discussions online and social media the way social media reacted if there's a lot of people that were anti Obama before either of his election so they want but it seemed different it seem different to me than this one this one seems like

like we had moved into another level of hostility that never experienced before and another level of division between the right and the left that I never experienced before and I like a willingness to engage with really harsh nasty comments and just added to dive into what you would see it all day I mean there's there's a certain Twitter followers that I think you're pretty much human beings but I would follow them and they would just be engage with people all day long just shiting on people and criticizing this and insulting that and it seemed like it seem dangerous it seems like things had moved into a much more aggressive much more hostile and confrontational sort of chapter in American history if this is all done

at the same time it is happening how much of an influence do you think this Ira agency had on all the stuff we would all like to answer to and I can't give it and so you are mine though but nobody who looks at this on the outside has is we can't see what people said in response to this stuff so I looked at now almost 200,000 of these posts as I spent most of last year do mice with this was this research we can see that they have thousands of engagements thousands of comments thousands of shares we have no idea what happened after and that's the problem so went once the stuff comes down it's really hard to go back and piece it together so I can see that there are some of the pressure point that the really really just

horrible troll accounts that they ran they didn't necessarily have a lot of followers but you see them in there like adding people so there is no at and then the name of a reporter at the name of a prominent person and so they're in their kind of like draft on the popularity of you know what famous people basically and they're just saying like horrible and it's the tone is so spot-on and a couple of them is like if you're going to look at their profile information which was also made public they would have liked to have a gap account in their in their profile that was like so they would a remarkable piece of of of of a culture in which you see that like they're actually sitting on gab to write and then so they can also go and they can draw on their own Reddit there's no 900 or something natural accounts were found on Reddit there on Tumblr and so they're just picking the most divisive content and they're pushing it out into communities and at the same time

can see that they're doing it we can see what people do in return we can't see do they just block did they have the fight back did was there a huge you know when this happens on a Facebook page and they're doing something like telling black people not to vote as black people we shouldn't vote what do people say in response and that's the piece that we don't have so when we talk about impact a lot of the impact conversation is really focused on did the swing the election we don't have nothing that I've seen has the answer to that question the other thing is even put the second question the thing when I think about impact I think from from I think you and I agree on this it also matters how does this change how people relate to each other

and we have no real evidence of that we have no information on that either this is the kind of thing that lives in some me on Facebook has it the rest of us haven't seen it so or most of these people is mostly Facebook is it mostly Twitter or where what is out of the breakdown of which about 6 million for original content created by about 3,800 accounts there were about a hundred and thirty-three and read it a hundred 33 Instagram accounts about 116000 posts and then 81 Facebook pages and 17 YouTube channels with about 1,100 videos and so they got about 200 million engagement on Instagram in about another seventy-five million or so on Facebook engagement or like like shares comments reactions you know

so

it's hard to contextualize what we think happened you can go and you can try to look at it

how well did this content perform relative to other real authentic media targeting these communities and what you see with the black community in particular is their Instagram game was really good they're so they're on their Instagram accounts that you know the top five three of them targeted the black community and got you know

and two hundred millions of engagements so I'd have to pull up the exact numbers and the names of the text on Instagram you can't really share so it's amazing that they got the kind of Engagement that they did even without the sharing function what are the things you can do is if you know the names of the accounts and their lot of them out there publicly now you can actually see them in regram apps so people were regramming the content so Facebook says about 20 million people to engage with Instagram content but what isn't included in that is all of the re grams of the content that were shared by other accounts so spread in the dispersion of this it's an interesting thing to try to quantify cuz we have engagement data but we don't know did it change hearts and Minds we don't know if

influence people to go follow other accounts we don't know if it influence people to not vote there's just so much more I think still too to understand him at how these operations we can assume it had some impact right I mean is as you were saying earlier when a new person enters into a conversation that it changes the tone of it how much of what they did was their own original posts and how much of it was commenting on other people's posts real American continent do they do that as well so they created a lot of their own stuff particularly in the early days and so you can actually read the dataset and one of the things when we started finding these posts I was struck by how sometimes it read like ESL and then sometimes it read like perfect flawless

professional English and then other times it red like normal English vernacular just the way that we would talk to each other and I started digging into what that was so when it was vernacular English when it was when it read like fluid American American English it was usually crib from somewhere else so they would go and they would find a local news story from some obscure local paper and they would crib and then paste that and then to the Facebook post to be that cribbed a sentence from that article and then there meme maybe they would a sentence underneath it to give it some kind of contacts triangle when they would write their own stuff you would see the sloppiness that's where you could see subject-verb agreements not quite there the you know ways in which like Russian possessives are different than American possessive is the slips there and then the other thing was the really funny stuff which was

you know that's a post that supposed supposedly written by Texas secessionist right so you can probably have an image of the Texas secession is in your mind as I say and it would be things like

Hillary Clinton is a terrible individual and as a as a terrible individual it's completely impossible for us to back her and her candidacy for the American presidency furthermore you know right it is clear that it reads like in English so nobody actually talks like that especially not you know your gear and unfortunately one of the best ways to tell what you're dealing with is actually kind of look for those incongruities now and see you as you read Communications online like Ken unit does this does this read like an American does this read like a communication and what we started to see you as one way to not get caught for your lousy English

you know how your cultural lack of a kind of Native Native abilities is to just repurposed other people stuff and so that's where you would see Memes getting shared from on both the right and the left you notes you a lot of these like Turning Point USA memes that they were repurposing and pushing out or you would see occupy Democrats were the other 98% so memes from Real American Paige's real American culture and they would just sometimes slap a new logo on and just repost it as if it was theirs so it does in those instances Regis like you know authentic American content and in many ways it is authentic American content how many people are working for the agency do you understand

I don't remember off the top of my head it was smoking a couple hundred a thousand I think I don't know if it's bigger than that I moved offices and then people started calling in bomb threats to the office and it was like everyday a new bomb threat would get called in so they couldn't work basically agency just like those people calling in his bomb threats to try to keep them from working and I think that came out really recently that said that like Army cyber or one of our agencies work to dislike take them off line during the during the midterms a couple days are in the midterms Russian ironic that is really finally moved to this nice new office building a check to Molotov cocktail through the window and I mean

only makes sense that in this bizarre and unpredictable and really unprotect unprecedented environment that we find ourselves in that something like this would come up and just soar through a monkey wrench into the gears of a real conversation online mean it's it's a really amazing time in that we're getting to see this kind of stuff but happen in real time we're going to see these these sort of weird attempts at manipulating things and I think in La Lotto a successful specially with less sophisticated people that don't really understand that they're being trolled and then someone is fucking with them and there's it seems I mean there's a bunch of accounts that I have bookmarked that I follow but I don't follow so I don't follow them online so I don't want them to know I'm following them but I just go to them and some of them are so strange a few of them are flat Earth accounts this is something that I'm finding yes

some of them literally have almost no it's all memes and they don't say much if anything underneath the memes and I go to it I'm like what it what exactly they doing here but what exactly they trying to do with these because they just over there very weird

that I came across I was looking at the the conversation on GMOs and because we have seen one of those things that Russia does besides the social box in the end of the Ameriana screen with like Americans directly is the house so is his Republican house committee House of Science and Technology committee about a year ago said that they were seeing evidence of

both kind of over it propaganda and then ways of disseminating the propaganda dissemination and an account and then the contents will take a look at three different things to try to get a handle on whether or not this is real or fake so when we talked about the accounts were looking at are they real people are they you know automated are they not automated when you're looking at the content usually looking at the Domain and that's kind of the last piece cuz you don't want to have any kind of bias get in there but you're just trying to see is it being pushed through like covert Russian propaganda domains like their think tanks and things and an asteroid is the dissemination pattern is it being pushed out through automated accounts is it spreading in ways that look anomalous forces how normal information would spread so one of them is at the house committee looked at was using that kind of rubric Russian bees dubious

pieces of content of narratives around American strategic Industries so the energy industry oil and fracking for example or the

you see a lot of stuff with GMOs in agriculture you know Putin and Russia being the land of organic plenty in the United States serving its people toxic poison vegetables this the sort of stuff and you know about the same time there's competition for who's going to get the you know large contract to provide rice to some part of the world so so there's like an economic motivation underlying is this kind of narrative and I was looking at one of these accounts and it was tweeting an article about Hillary Clinton a vote for Hillary is a vote for Monsanto but it was tweeting this in like 3 months ago mid 2018 why are we are we tweeting about you know Hillary has votes because they just they're just there to was written by a Russian think tank so there they just have these automated accounts retweeting repurposing this content

from forever ago and it doesn't even make sense it's just out there to amplify particular point of view or bumpa mentions of site with the anti-vaccine posts and I wish I was an interesting thing so I would say not much be honest there were 800 I think tweets about vaccines in the content and so Facebook and Twitter you have this Facebook and Instagram you have this building up ABS Twitter you have instead they're just talking about whatever is popular right there toddler shitposting they're talking about whatever is current new whatever Scandal has just broken anywhere in you know so there is less about establishing relationships and more about joining the conversation and nudging it and so most of the vaccine related posts it was not a big thing for them it wasn't something that was on like Facebook and Instagram

grammar that's where they're really leaning in like this is what we want Americans to think about so no mention of vaccines on those platforms not on YouTube on Twitter you see it in 2015 finals during the Disneyland measles outbreak much like there's a whole lot of conversation on vaccines right now because of the outbreaks in Washington and New York back in 2015 you saw the same thing lots of conversations about measles because the Disneyland thing that happened down here and so they're in there and they're saying

vaccinate your kids don't vaccinate your kids they had a couple of conspiracy theorist accounts

I am trying to remember the name it was a look like a blond woman I think it's name is Amy and Amy was a was a conspiracy theorist I wish I could remember

that was her name she got any black there are there certain other personas actually got a lot of lift there was one called woke Luisa that was a black woman there is no it right then and there was 10 GOP is a fake Tennessee GOP page it sounds like some of them are actually pretty funny actually really good that's where I think the thing that you know even West

whatever you know political proclivities may have I think you can at least recognize humor even if it's laughing at your side and I will say that some of the stuff especially targeting that the right wing and the right-wing like he's kind of pigeons were they were funny there really were and it was I think that is people assume that like they're too smart to fall for it it's just those liberals or it's just those conservatives or you know it's it's really Target everybody they understand the psychology the motivation the narratives and the culture and they produce the content accordingly and I imagine that they had some stuff in

two narratives that came out in 2017 the first was when Facebook started to moderate their Pages they started to scream about how Facebook is censoring them so the exact same narrative today about how many moderation is censorship it see no picture of like Zuckerberg and it's like nice page you got into it and putting out there when they're complaining that their fake page got taken down there was tons and tons of these means also about the Russians did it in the idea that the Russians did it so this is as the story is beginning to come out before we had the tech hearings before we've had the Mueller indictments before we've had the investigation you see these where it's like oh my speedometer was broken it must have been the Russians or picture of Hillary Clinton and it's like that and I like little Golden Book kind of thing and it's like the whiny child guy to blaming

set for your failures and they're like Metatron you imagine them sitting there like to have a picture of like some leg you know buff guy carrying a gun and it like I'm not a Russian troll man I'm going to marry it's just so spot-on and again I can't see what the people commented under it if they're like right on Earth but so that's where you get it the they people think like how I'm too smart to fall for it or oh this is targeting those other people know it isn't that's the problem it's just it's going to Target you with the thing that you're most likely to be receptive to just because of psychological bias and tribal affiliation and you're not sitting there thinking how is this person who is purportedly just like me screwing with me and that's why that's why it does manage to attract a following and get retweeted get reshard it's

good was so clever because it's so comprehensive there's so much involved in the fact that they're willing to do this for years and years before they really sort of activate the political aspect of what they're trying to do it all so it's it's strange if they're so sophisticated about our culture cuz we don't know a goddamn thing about Russia the average person knows Putin bad evil warlord Crimea he invade you know like we eat when we have like a 4 year olds understanding like if you had if you just grabbed a person random person's for college educated person and ask them to describe what's so bad about Russia wow it's like communist over there or something I'm really fucking and they hate us first of all bombs like this so little understanding of their culture but yet they know so much about ours that's one of the weird things about being American when you go when you travel overseas and you realize how much they know about our elections how much they know I don't even know who the fuck there was running their country don't have any idea

but they know about Trump and they know that Hillary did this and they know that Bernie wants to give the money away and you know it's crazy it's it's it's weird and these people must have like a deep education on American culture American politics and do you think they're training these kids are first of all couple people actually came here and did a road trip around America that was in me from February 2018 Arvin three kind of big documents that have come out to from Eastern District in one from you on how it all worked out I think it was under a misconception is this notion of $100,000 in ads they spent 18 million dollars in 2017 I believe

stop that came out during another one of the Mueller indictments so they're not just the money is not just going for the salaries in the end by somebody is also going for they were talking about using kind of consultants and this is where you get at this thing that comes out during the stand up for their life black people who are LGBT don't want to see white lgbtq memes and and this degree of granularity the degree of sophistication but then also if you see them doing is engage in one-on-one and that's where it crosses the line from social media operation to this is much more like like spying no I didn't this couple under Deep Cover that are engaging with and pretending to be Americans in forming relationships

in the Mueller indictment is the text messages that are the messenger at the Facebook Messenger messages where they're going back and forth with real activists and they're saying things like you know hey my account. Shut down can you run some ads for me or the hey I want to help your protests where fellow black lives matter activists and I see you're running a protest up and I think it was like Jessica or something Binghamton how can we help you we can give you some money for posters and they're sending money for posters or they reach out to a trump supporter and they stay like we used to be really funny to have a Hillary Clinton impersonator sitting in a truck flatbed truck that's made up to look like a jail lettuce and if we give you some money will you find the Hillary Clinton impersonator put her in jail and has Hillary for prison thing and so this is where another thing that they did was using Facebook events to create a real-world protest

so they're not limiting it to ship posting online and making people feel tension online they're actually sending people out into the real world to have in Street violence and so one of the things that they did was say coordinated Facebook event one for the Texas secessionist page and one for the there's a pro-muslim called United Muslims and on the same day at the same time in Texas they have a rally to support Texas culture and resist the islamization of Texas across the street from a rally to defend Muslim culture and so they like there's literally no they just create these Facebook events on these pages and and then they promote them with a dollars and other things and and you literally could going to look at the Texas report from that day remember if it was dozens or hundreds Peta is sufficient number of people showed up that they had literally on opposite sides of barricades police officers in the center screen

at each other because one group is there for that resists islamization of Texas and the other group is there to like defend Muslim culture so you get to you know to opposite sides of the Spectrum in the same place at the same time you literally Insight like a mini Riot so they were there there about 80 one of these events were they were holding on

black lives matter style rallies for victims of violence police violence memorials for people who were killed by police officers you things that Real Americans would do but this wasn't being done by Real Americans and that's that's the Insidious thing right is is how does how does Facebook detect that how did you know how do you when you see this come to defend Texas culture and you're a die-hard proud Texan you you're not thinking like somebody in St Petersburg is is is organising this and that's the it mean I think that the idea that this was just some memes is just not it it doesn't respect the

significance of what they were trying to do and how effective that they were with these other things were even if they're just trying it out just a little bit just working to see what works they're always experimenting they're always trying to find ways to create that tension and that's the thing that I think is so interesting about this right this involving these ideas and information War where these tactics evolve and you are really at a disadvantage when it comes to actually detecting them and why would they have this Texas secession is Paige across the alley across the street from a pro-islam rally why would they do that you know if you're on the outside you think about the amount of effort that's involved doing something like this and they are also doing this with no leaders right there's no one there that's running it when they get there so all the pro Texas people go here we are

there were comments on some of the like archive pages and things where you could see the screenshots of people being like nobody showed up who is in charge of probably throwing a lot of things against the wall hoping that they stick black matters with internet cuz they wasn't made a whole website website black matters you ask., which I think is still active it's dormant they're not updating it but I believe you can go in and read it and it was designed to call attention to police brutality type things

until that is black matter to us page and then there's the black matters Facebook page the Twitter account Instagram page the YouTube channel does SoundCloud podcast the Tumblr that space books did Facebook stickers that look like little black panthers like little renewal cats are actually really cute very well done so you have this entire fake create out of whole cloth all theirs and then what they start to do as they start to put up job ads and so it's come be a designer for us come ba I'll come right for us come photograph are protests, kind of like like hipster you know holding a sign like join black matter as you see them go through the same way you would if you were starting a you know media brand they start posting ads for do you want to be a calendar girl send us your photos on do you want to be on a black reality TV show send us videos

I love you do you like they begin to do real work to ingratiate themselves with the community and the idea was that it was

I'm frankly militant to ask in that it was supposed to teach a black people how to handle themselves at protest should there be police violence how to fight back and they actually went and found a guy is physical fitness so you know martial arts guy and they were paying him via PayPal so he was running classes for the black community under this Black Fist brand and they would like text him or call him he play Tony voicemails on TV actually heard them after my report came out I think they track him down and he just talks about how they again I just PayPal you know couple hundred bucks every time you run a fitness class could be surprised they actually had a YouTube channel with two black men named Williams and Calvin

and there was this channel Williams and Calvin actual Black Friday is called a word of truth I think was the name of it then so Williams and Calvin these two guys would would give their word of Truth and their word of Truth was usually about you know how fucked up and guess which they're they're very real grievances underlying all of this and that's the problem right they have things to exploit

I mean imagine what it would definitely paying them and they were going to channel is organized one of the guys who is in that channel pops up again in 2018 right before the midterms like maybe even the day before in front of the timeline here and he made a different video saying he wanted to this was amazing thing he wanted to leave the internet research agency kind of so he was saying that I'm tired of doing this work I want to I want to do a leak I want to show you all the things that the internet research agency is done and so they actually put out this so this guy who'd been in the Williams and Calvin videos of people recognize his face in the 2018 midterms goes and says he wants to leave and he's going to leak all this information and

sorry it's damn cough and he wants to to like confess I don't remember all the specifics cuz it was right before my thing came out and I was so busy working but he wants to expose the truth and I think most people didn't cover it didn't pay attention YouTube shut down the channel deleted the video immediately I think that it was seen as another influence operation you know you don't trust even hinted coming out this was this is so convoluted I'm sorry I know it's like hard to explain without visuals what they release a pile of documents in which they claim they actually hacked the Mueller investigation Mueller had nothing

and so this is another kind of convoluted piece of us were they do release information and end so in this particular example they release information that we believe they actually got through legal Discovery so the documents that the investigation provided to one of the indicted Russians were the documents that they then leaked claiming they had hatched the Mueller investigation so they're constantly doing these things to generate press generate attention create just that degree of people don't know what's real or they read the headlines that are then released by The More propagandist We over Russian propaganda and they think that that is the true story that that is that the Russians hack the Mueller investigation so there's always this

how do we create fear uncertainty and doubt how do we throw people off how do we come up with these extremely convoluted spy games that leave people feeling unbalanced that make people wonder what they can trust who they can trust and what's real and even if somebody who looks at the stuff you know day in and day out for years I do still regularly get surprised by the by the sheer cannonballs anus and Ingenuity of you know some of the stuff is it as it comes to light quotes really went so far as to hire people

how to make a fake account on YouTube and hired these black eyes to pretend that they're doing it on their own and they're really being hired by the Russians and then when the guys leave if you don't know if they really did leave you don't know if this is just more bullshit it's like you like you were saying earlier if they get you and you buy a new Hook Line & Sinker they win if they get you to think well how much else is bullshit they still win because you're looking at everything with sort of this tainted lens now everything seemed and it is sent that's probably the ultimate goal is to disrupt our social media environment in to sort of hijack the the natural conversations that are taking place at the others certainly know I was in Estonia last year and they've been targeted by the stuff for decades now 25% russian-speaking population in most of the news that they get is from Russian media

writing around the Border they talked a lot about the extreme commitment to educating their citizens to make them realize that this kind of thing does happen this is what it usually looks like don't share it just ignore it let it go by and I don't think we are quite there yet I think there's still plenty of people in the country who don't believe it happened or for some reason are completely incapable of separating the Russian social media influencer campaign happened from it means Donald Trump's election is illegitimate or it means Donald Trump colluded different statements you don't have to collude in order for someone somewhere to unsolicited go and support your candidacy

so you can believe two things simultaneously did not collude and that has an election is perfectly legitimate that this had no impact the two that it still happened. I think is consistently amazed at how hard that ability to hold those two ideas is for four people they just believe that and their supporters of trump they absolutely cannot acknowledge that this operation took place and I or if they are passionate supporters of the far left it's more like an equivocation you know well we don't really know if they did it well well how do we know we know so that's where family depending on which part of the political Spectrum you sit on

mullet it falls right into our the issue that we have with cognitive dissonance we believe in someone if we want something to win especially with our team or are person are on our side you know a guy I saw a lot of this when Donna brazile released her book detailing how the DNC sort of rig the primaries against Bernie Sanders and turn in for Clinton of somebody who the requin support is it just didn't want to believe it I was like well why wouldn't you believe this woman like you believed her before when she was supporting Clinton and then when she leaves now you now you won't believe her it's because it's inconvenient and we were real weird in are binary view we want things to be good or bad is 1 or 0 this is it in this is a super complex issue it seems like they've been doing this for a long time and I've gotten really sophisticated at it and I think there's a lot of people have been sucked into it that have no idea that it's actually influence the way that they form their own opinions

this is where it gets really strange people are so malleable and they're so easily manipulated many people are that something like this like a real good solid concentrated effort to try to Target these groups that have these very specific interests and really dig in and form roots and then go out meet so sophisticated approaches I'm in on one hand horrified and the other hand deeply impressed

was this freaking you out when you had to let go over all these memes and you actually laughing at them you like God damn it

is that tweet that goes around every now and then you don't have to hand it to them and I'm always like that how do I add how do I properly convey a recognition for any favors by pretending it all sucked and didn't matter in their incompetent determined that is constantly evolving and to treat that with the degree of respect it deserves I think that that's just common sense actually

media on both sides of the aisle and I thought you know so I try to stay current actually on what means are percolating in lots of different faces and part just because I am always curious about what's organic versus what seems to be disproportionately Amplified or what new communities are popping up I just think it's I think the spread of information among people is just a very interesting if it's something that interests me a lot of things crowd psychology is really interesting psychology is transformed as the internet has kind of come into being

particularly with things like the mass consolidation the ease with which we can Target people you know you didn't even really talk about that but the one of the things we even in the decentralized internet always been crazy but it's likely to be receptive to it now and ask people self-selecting to tribe particularly in this country right now one of the things that's remarkable is the way in which once he self-selected into that tribe and this is the media in your ecosystem in you share it with your friends on Facebook and Shores at the people who see it or the people who are most likely to be receptive to it or if you run the ad targeting you directly you know send it into the fields of people most likely be receptive to it we have this interesting phenomenon where

consolidation targeting and then needs game of algorithms mean that it's just this kind of information goes way farther away faster than it ever could have been the past regardless of whether it's Russia pushing it or has went down recently globally now we see countries to bring their own people with it and it's just this is the information ecosystem this is like the new infrastructure for speech and it has this kind of sensation

just just want to have them

colds going around don't feel bad I get you one who's who's seen all this stuff is this stuff has obviously Facebook has checked your child I'm sure Twitter is aware what is a reaction Ben and is there any sort of a concerted effort to mitigate some of the some of the impact that these these sites have I think

in 20

Seventeen when we started like independent researchers I guess people on the outside of academics you know really began to investigative journalists would identify the name of the page and then me and people like me would go and we would scour the internet looking for evidence of what was on that page so I found a bunch of stuff on Pinterest for example wrote about it guy by the name of Jonathan Albright I found a crowdtangle data cache and with that we got the names of the bunch more pages bunch of my posts we had some really interesting stuff to work with originally the platforms were very resistant to the idea that this has happened

and so as a result of that they were

there is a the first thing that has oxide in 2016 when Trump gets elected Twitter with people who work at Twitter saying oh my God where we responsible for this which is contrary Silicon Valley thing to say but what I think they meant by that was their platform have been implicated is hosting Russian Bots and fake news and harassment mobs and number of other things and there was always the sense that it didn't have an impact it didn't matter and so this was the first time that they started to ask a question did it matter

take me to the very small percentage of whatever on Facebook the amount of information on Facebook and the idea that I could have sworn election was Ludacris so you have the platforms kind of leaders of the platforms digging in and saying it's inconceivable that this you know could have happened research in the discovery begins to take place over the next nine months or so you get to the one the tech hearings happened so I worked with by the name of Christian heresies the 100 is made of Sam

and he can I started going to DC with the third fellow Rodger McNamee and saying hey there's so much does this body of evidence that's coming out here and we need to have a hearing we need to have Congress ask the tech companies to account for what happened to tell the American people what happened because what we're seeing here is outside researchers what investigative journalists are writing the things that were finding just don't line up with the statements that that nothing happens if this is all no big deal and so we start asking for these hearings and actually myself and a couple of others then began asking them in the course of these hearings can you get them to give you the data because the platforms had given the data so it was that lobbying concerned citizens and journalists and researchers

because it had become such a politically divisive issue did it even happen and we felt like having them actually sit there in front of Congress and account for it would be the first step towards moving forward but also towards changing the minds of the public and making them realize that what happened on social platforms matters and

it was it was really interesting to tube to be part of that as it as it played out because one of those things that Senator Blumenthal Senator said was actually said Facebook and Twitter have to notify people who engage with this content and so they are there was this idea that

if you are engaging the propagandas content you should have the right to know and so they started to push messages Twitter sent out these emails all these people saying you engaged with this Russian Troll and Facebook created a little field a little little page that told people they had like to follow to a troll page so it was really trying to get at making the platforms accountable with a dude outside the platform through email huh

it is interesting because I would never read an email to Twitter censoring right you like this is just got to be nonsense out of hell of a multiple-day back and forth and some Russian troll was not a thing that everybody agreed on in 2015 and they were having this conversation about Isis and that's where there's that's the three line here which is and it does Connect into some of the speech issues to which is

what kind of monitoring in moderation do you want the platforms to do and when we were having this conversation about Isis there was a not-insignificant collection of says that we're really concerned that if we moderated Isis trolls on Twitter beheading videos of a sort of universal agreement that the beheading video should come down

what if we took out what we're called the Isis Fanboys which were like 30-40 thousand Accounts at their Peak that we would hear there's a document called the Isis Twitter census for anyone who wants to actually see the research done on understanding the Twitter Network in 2015 there was a sense that like one man's terrorist is another man's freedom fighter and if we took down ISIS Fanboys where we stifle in their freedom of speech freedom of expression in Lake goodness what would come next and that when you when you look at that that fundamental swing that has happened now in 2018-2019 where does deserts that same narrative because originally no moderation was taking place and then now there's a feeling that it's kind of song too far in the other direction but the original conversations were really

how do we make Twitter take responsibility for this and legally they aren't responsible for it they are legally indemnified against the they're not responsible for any of the content on their platforms none of the platforms are there is a law called the communications decency act section 230 that says that they are not responsible to have the right to moderate but not the obligation to moderate because they are indemnified from responsibility so the question becomes now that we know that these platforms are used for these kinds of Harm's and they are used for this kind of interference where is that balance what do we want them responsible for monitoring and moderating and how do we how do we recognize that that is occasionally going to lead to incorrect attributions people losing accounts and things like that so

yeah they're there in a weird conundrum right now or they don't they're trying to keep everything safe and they want encourage people to communicate on the platform so they want to keep people from harassing folks but because of that they've also they've got these algorithms and they they tend to miss very often like this whole darn decode Fiasco where people getting banned for life for saying learn to code which is about as Preposterous as a guest I think they'll learn to code Fiasco is going to be the Tipping Point where a lot of people end up in the future when I look back on when did the heavy-handedness to become overreach learn-to-code because we in Jesus Christ mean that you can't say learn to code mean I look in my mentions I mean in any given day especially like yesterday I had vaccine proponent was really disturbing to me it was like

vast majority of the comments were about vaccines and so few about these uncheck diseases that are running rampant in poor communities which is the most disturbing aspect of the conversation to me that there's diseases that Rob you of your intellectual capacity that are extremely, as many as 10% of people in these poor neighborhoods have almost no discussion it was all just insults and then you know you fucking chill and that's 9 you know it's like

I think that one of the challenges for the platforms as a lot of things start out like my Dakota member watch that play out Covington Catholic was another thing that I mean God would like to call there was some people who were trolling and just saying learn to code and you know whatever you don't have a right to not be offended but then there was the other accounts that kind of look at that step further and began to throw in like the ovens and the other stuff

just a Content itself like if you start doing keyword bands you're going to catch a lot of shit that you don't want to catch but the flip side is if you that this is the challenge of moderating its scale which is where you know what what side do you come down on to come down on saying like 75% of people with hashtag learn to code or just you know I'm not doing anything incredibly offensive and then the 25% who are they really change the tone of the overall campaign in the hashtag for the entire Community is not Tracy Twitter I think come in with the more heavy-handed I just shut it down kind of thing I don't I don't know if there's anything answer I think that we are leaving today what was the latest kerfuffle Elizabeth Warren got an ad taken down on Facebook and there was a whole conversation about was Facebook

transferring Elizabeth Warren I I personally didn't think that it read like censorship but it was like yeah listen to Hooker it's more like a cell phone like she had a picture of Facebook's logo in the image and their that violates the terms of service and the reason behind that is actually because Facebook doesn't want people putting a bad that have the Facebook logo in it because I take scam people or if it's a great way to people often so probably just like an automated automated take down like an automated they get to hallstatt you have to go and make some changes when you can proceed back out again but it just happens at a time when there's like so little Assumption of good faith and so little Assumption of

such extreme anger and polarization and you know assumption that the platforms are censoring with with every little kind of moderation snafu that it it makes it I think

I don't know how we have the conversation in a way that's healthy and look towards Solutions as opposed to the left screaming that it sends her the right screaming that it censored the platforms you know trying to get around how do we both moderate on not moderate which is a tough position to be in I think

God don't have any good discuss that pretty much in Dap what she was saying this is about moderating and scale when you're talking about millions and millions and millions of posts and a couple thousand people working for the organization and then algorithms and Computer Learning is trying to keep up and that's where things like learn to code and people are so outraged and pissed off because when they do get banned they feel like they've been targeted what do you mean really just ran into some code and then it's really hard to get someone to pay attention to your appeal cuz there's not enough people that are looking at these appeals and there's probably millions of appeals every day it's almost impossible and also here like this person is harassing me and I'm demanding moderation and nobody's doing anything about it so it's it's definitely I think

gotten

worse it's going to look back at 2016 and wonder how much of the where we are now is in part because not a whole lot happened in 2016 and 2016 Ozark 2015 in particular very light like almost no moderation just kind of let it all hang out there and I I look at it

I look at it now particularly as it evolved into this conversation about Free Speech public squares and what the new kind of infrastructure for speech what rights we should expect on it it's a really tough

you know I think some of it is is almost like the people who are who hear the rich free speech and they just assume that it's people asking for car wash right to harass and saying you have a balance that you think Jack and vagina ever sing this on your show highly maximize the number of people who are involved make sure that all voices do get heard without being unnecessarily heavy-handed and moderating a thought or content and instead moderate behavior and instead moderate particular types of signatures of things are in authentic were things that are coordinated and looking at the gang gets to disinformation to rather than trying to police this information by looking at content really looking Instead at actions and behavior and account authenticity and dissemination patterns cuz a lot of the worst trolls and stuff were just using these throw away accounts and then they disappear

myself that when we're talking about censorship were talking about moderating content that really were talking about this current ERA and that what's coming is essentially we're like putting up a small

like twig fence and a herd of stampeding buffalo is on the way in terms of the more invasive or the more the more potent levels of technology that are on the way I just feel like

I feel like we're everything is moving in the very specific Direction and that very specific direction is less boundaries between people and information and that includes communication and it's going to be insanely impossible is insanely difficult or nearly impossible to moderate in 10 years I just don't think I don't think that's going to be I don't think it's going to be in the Wheelhouse I just did I think it's going to get weird we're entering into some weird place where we are going to have to stay off of social media because it's just too toxic or grow a thick skin and just be able to deal with anything and then if that's the case how are we going to be able to differentiate between things that are particularly designed to to manipulate us specifically designed to manipulate us and change our opinions by Foreign entities like this you know this Russian troll Farm

I just think that you know what I think about like

so we believe that this information is in part facilitated by game of algorithms consolidation and then the targeting the sky things we talked about through this conversation then I think that the algorithms piece the manipulatable algorithms that's really squirrely the responsibility of the platforms I don't think there's any regulation or any kind of framework that's going to come from Congress it's going to address that was I was pretty clear from the Facebook hearing tried to mend a fairly and Sheila dikshit an Android phone an iPhone though it did they really don't know what's going on

show me even more hilarious just say fucked up man I mean to say that you knew Tim Cook his real name wasn't Tim apple is funny, I did I appreciate the rest of like every city on Silicon Valley changing your Twitter post we talking about you were you were talking about them up Jamie algorithms squarely in the preview the platforms and that's because it's an arms race right as they change their algorithm a little bit we get for the product function which they just doing their roles business there is no regulation it's going to come down fast enough to catch that I think actually finances an interesting parallel here

cuz like in the financial markets there is a multi-tiered levels of a regulation and oversight so that there's always some entity responsible whether it's the exchange or self-regulatory organization or the government is looking to see if like information Integrity in the markets is being maintained right there's no shity algorithm coming in to manipulate people it's just making sure that level of trust right now the tech ecosystem is lacking regulation and all of its forms so that would likely change but centralization and is I don't know how you executed the antitrust thing in particular is it as it comes up so much more now just I don't know under what economic grounds you make that claim that's outside of of of my Wheelhouse in my area but there is something to be said for this you know return to decentralisation in some way people have what they want

it lets you know right it's a great example you have these sounds like federalism you have the central platform with any of these little communities under it and each Community has its own Norms each Community has its own rules nobody who violates the moderator rules in a Reddit and scream censorship is really taken seriously and it was how in the olden days of the internet like HughesNet some things you would have this is the community that you've chosen to be apart of if you don't like the moderation standards you go to this other community solidation is that people who do get moderated way I feel like there's nowhere for them to go that they have lost access to the entire world

so I think if you have that decentralization it in some ways

it if it stops being quite so much of a freedom of speech issue if you can't speak on it if you if everything is like if there's 52 platforms and you fall afoul of some sort of norms or standards or Community membership in this one you can go over here to this other one away or less potent maybe if someone or something on moderation at lower levels vs. top-level like Smiley booted off approaches to sort of these community service hours from cells but even inside those communities then you have people that gain power through moderation and they start abusing it and then it becomes some sort of weird hierarchy

it it's like

decentralization in general is probably the right move for all this stuff but how does that happen with something like Facebook or Twitter without government intervention in the email that's one of the things that 10 pool was bringing up like if you guys don't do this we don't handle this it's entirely possible somewhere down the line you're going to be regulated by the government do you really want. I think the best inevitability whether they win show

well I know honestly I say that and I think back to reality which is a Miss congress with this executive I don't know how we get any regulation through

I seen some examples like the honest ads act which was introduced right before the first tech hearing if I'm remembering a timeline for at least that would have been like

late 2017 and what they said was

it's to be you no longer have like a free-for-all with ads on social platforms for nobody knows who paid for it to where it's coming from or anything like that and and what they needed it so Senator Klobuchar Senator Warner and I think Senator McCain also was that was part of this crew this law saying that the platforms have to follow the rules that TV and radio already follow Aniston example of recognizing the role that that you know at these are no longer startups that can't be you know that can't meet these obligations it used to be that Facebook was exempt from these

from these are disclosure requirements because there a dispute over this tiny postage stamp-sized things are on the right side of the page it was given the same exemption that campaigns got for like skywriting and postage to factor of the content makes it such that you can't put the ad disclaimer on there and it used to be that all of the advertising on Facebook was regulated using that same finding that that that these postage stamp-sized things are too small to put the disclosures on and then of course we know that evolved into the ads being looking much like an organic post and so now they do have these little things that pop up where you can see why you got targeted and what it is I think that that's an example of like the credible threat of Regulation and the public opinion moving the platform to take an action

that it wouldn't have necessarily done on its own so it's not regulation but it's

it's a nudge through public opinion and the credible threat of future regulation in California go after the platform's also recently there was a California gdpr thing from last year California State Legislature saying we're going to pass a privacy requirement and and they did it they got it done because I don't know the specifics but the gdpr was the law in the UK and in Europe that protects the data creates particular protections like your you have to re opt-in for targeting or certain kinds of targeting that they can't do certain types of data that you can request a delay so this is a provision that took effect in Europe last year

we don't have that same lot here in the US we we don't have the same data protection does the Europeans and so California gdpr was the California state government out of state senate legislature passed a law that basically mimicked a lot of the provisions of what the Europeans were given under gdpr what that did was it created a law that applied to the people of California and so Facebook and Twitter and others don't want to be in a position of having to have this you know kind of balkanization or a legal requirement for the day and turned have now I believe gone to Congress suggesting that it that we're going to need to have a federal solution that applies to all of the US so Federal level privacy regulation because they don't want to have to but here to the Privacy regulations of like each individual US state one solution that's been tossed up was that people would have to somehow another confirm their identity that instead of it being in an

am I supposed to say Renee diresta like I know who you are if they have a photograph have some sort of government ID shows that it's you that we would somehow or another minimize trolling minimize disinformation if you if your account was connected to Social Security number or whatever it was probable course is at these damn things get hacked all the time and if your Twitter account gets hacked now they have your social security number they have your address they have your information that used to sign up and there's a lot of it unless there's some way and there isn't to absolutely lockdown all that data and make sure that it's inaccessible to some sort of third-party

that doesn't seem like a likely course of action either

question of identity I think most social science research that I've read has suggested that that's not necessarily the be-all-and-end-all make it depends on your point you're trying to do something right now if there is a request for FTC comments for net neutrality and I know that a lot of them were you know whoever left them there Lots Huns I think Millions actually if these fake comments that were left on the net neutrality call for public comment where they were scraping like that you know those are those crappy data Brokers is horrible things were like your name and your address is up there and names and addresses and then email addresses and leaving leaving comments pretending to be those people it's it's hard nobody don't want to enter their social security number and to some

former validation lot of people will point to things like well you know America has a strong commitment to Anonymous speech so cultural thing actually Mia farrow's paper is a requirement to validate your actual name and address they send a postcard to your house if you want to run political ads and then I remember people complaining that people who didn't have you know that this was going on during the DACA arguments so Grayson debates right right as this was happening people began complaining that immigration activists who are undocumented would not be able to run Facebook ads because they didn't have a you know

identification to verify with so no matter what people put out there's going to be somebody who has a complaint about it this crib lock everybody recognizes that the situation sucks and that social media is a disaster on a number of fronts and there's not much in the way of plausible Solutions I think for disinformation and particular just to stay in my wheelhouse we're trying to push towards multi-stakeholder ISM which is just to say can we create

the back-channel channels of communication that have come up for election integrity and things over the last few years last year-and-a-half can we standardize that in some way can we create an oversight body maybe the FTC that is at least responsible for having some oversight to make sure the platforms are doing enough but this is I think this is going to be the the theme of 2019 does it does it the antitrust route does it go to privacy route like visit do a kind of hybrid combination of multiple

tackling multiple problems at once it really curious to see

how we shake this out cuz it just seems like no damn you even agree on what the problem is we're not quite there yet you are seeing calls like a particular from Elizabeth Warren Ford breaking up a lot of these larger institutions not just not just even social media but even Amazon she's talking about breaking up a lot of his bigger companies it's the department that is like the to what and make them what and then what happens then you don't want it would have one of those things that you broke up that becomes Twitter becomes more popular than the other Twitter and it has much more attendance then what you do I'm really Matt Stoller knows the space in and out and I just don't

personally with. I feel like a lot of people are moving into smaller communities a lot of people are moving into groups are moving into WhatsApp chat through recognizing that the system as it is right now has this toxicity and are withdrawing a bit I don't know if you've seen us in your friends are Community but Jimmy and I were actually talked about it yesterday in terms of the the use of Twitter the diesel Twitter is dropped and one thing that I noticed is that my follower numbers doesn't move very much on Twitter as opposed to like Instagram I don't really use Facebook but Instagram does does a giant difference in how many followers I get per day I need a platform and it seems to me that the people that are using Twitter live kind of like locked in sound a little communities and is mostly toxic I mean

I'm sure I'm generalizing I'm not I am for sure it's probably not even 10% toxic but it seems toxic you know when you look at those kind of comments and any time something happens it seems like the reaction to it is very rarely is it some sort of objective rational discourse it's most likely just insults and you know swears it's weird it's just people are communicating in a way online that if they communicated in real life there would be blood in the streets yeah I do I think that's what are the special listen to your thing with with Tim and Jack and Vijaya the

idea of the Public Square and I use this metaphor to like when I write about them that the to privatize Public Square but then I think about it sometimes like we never had a National Public Square there's no such thing in the history of America as a National Public Square Regional Public Square is there a town square is States version of where there is this kind of federalism people who have self-selected to live in a particular communicators Norms in that Community but if I were to go up to you in a public square and start screaming in your face or you know being an asshole and try to get a whole mob together to go after somebody would intervene either a bystander or the police and we have Notions of like nuisance and things like that we have Notions of like there's there's more of it and intuitive sense of the balance between speech which is to be protected and then the kind of fighting words in that sort of thing there is no clear lines on that there's not much in the way of norms and when you're online and there is

nobody is going to come in and step in and intervene in a way that would play out in in real life so good behavior in the real world into this massive that's the sort of always have them and that's that's her tomorrow we are on social media you carry over to real life from social media that you can find in these protests at universities when conservative speakers, and then antifa wants to shut them down and then you have people like the proud boys fight with antifa I don't remember that before ever I think this is this is a byproduct of the type of communication that's the norm on social media I really think that's what's happening you're either I really think that instead of social media mimicking real life real life is starting to mimic the kind of interactions of people

on social media and with violent repercussions they certainly if not violent very aggressive and angry in a way that go back before social media when was the last mean in 2000 in the 1990s how often were there these an incredibly volatile protests at universities where you have conservatives and liberals screaming at each other and you have these these people that are being deplatformed in and they won't let him speak at these colleges and then they're they're Gathering up this online mob to try to bolster support and then people come to meet them we're going to stop them at all costs and it's kind of its kind of flavoring real life versus real life being represented in social media I don't I don't remember it from when I was in college either didn't exist

I was going to say getting back to war protest but very specific an unjust war that nobody want to be a part of

this is a weird time I know it feels unstable but it's also awesome how many join the shit out of it I mean it's a problem because I I love the fact that it did for the first time in my life it does not seem like the government has a fucking handle on how people are behaving and thinking at all like they don't know what's going on it's like you've got people that are

you know the people that are trying out socialist tropes in an in socialist ideas for the first time in the mainstream they're getting a big Groundswell of support behind and then you have a lot of people like Pro nationalist pro-america for for the first time in a long time and that's getting a lot of support there's more discourse now even if it's toxic and I think a lot of it isn't I think like I said if if 10% is toxic it seems like it's all toxic attend people calls you a piece of shit like that I noticed that too

actual numbers of of of horrible trolls or maybe this afternoon we can go try to prove you wrong I'm on Twitter a lot and it's the platform I use more than any other so for all the Nerf all the complaints I do feel like there's some real value there for me personally and and I I like

I like these Serendipity of of the unexpected being pushed into my feet occasionally sometimes it's an waiter thank why the hell didn't you know why but I think there is a lot of value to the platform I think we're unfortunately I really do believe that's so much of the polarization in the conversation around speech is people who got burned during the laissez-faire days of 2015 2016 in mentally linking up the idea of free speech with the idea of being harassed online and I think when you look at just purely opinion of actually no data for this out but when you look at the Pew studies that show that younger people are more likely to want it was safe spaces or less less

offencive opinion I do sometimes wonder if that's an effective coming-of-age at a point when you know random assholes were screaming at you on social platforms 24/7 versus like I didn't have that experience growing up I didn't have any experience until I was like 25 so maybe there is something for your point about that almost

there is not much of a difference as you know people spend so much time online this is where you're having your social engagements is where you're having your conversation so it does shape the way people think about

their experience of of what it means to have a conversation and what it means to speak freely think that's one of the interesting you know it doesn't help that the vet Free Speech has sometimes in many cases become a fig leaf for I want carb launch to you know and say all kinds of mean shit to people all day long with the consequences of the Black Cat Appreciation there's also an issue I think with young kids today that I've smartphone addictions people today I should say we got young kids how about me humans with smartphone addiction and when you're online and we sing about the majority of your interactions with human beings but there's a lot of folks around their phone 8 hours in a day I think about those interaction now that's a shocking number of interactions with people that you're not even in direct physical the presence of your you're not looking at them you're not

you're not waiting for them to talk and not considering what they're saying you're not reading social cues all the things that make us human those are all thrown out the window just looking at text and it might be coming from Russia to text that you're getting upset at in responding to me for your research really there's a direct possibility it's not even a fucking person or a person but not not really representing their actual thoughts is trying to push your buttons

I think it it really is more in the you need timer it's hard to know who you're engaging us it's hard to gauge whether it's good faith it's sometimes where else responds and go click into the person's feed to try to decide if I should take this as a seriously good faith inquiry or if it's like a like a you don't have to do that when you're in person it's a young single male and he was going through his direct messages and all these girls were sending him all these naked photos and videos and let me see your phone and I said link click on that link I go she has one picture on her page you fucking dummy this is probably not even a person like who knows what this is someone from Nigeria is trying to get your credit card information

pages I just you know I thought I was a girl Sammy naked pictures

and I just do that sometimes will they definitely do do that sometimes people are weird they do all kinds of weird things but you know there's there's a lot of these fake accounts and I don't know what they're trying to do to try to get money from people work that does come up I've seen them every now and then if you follow reporters on Twitter particular ones who is like open DMS will periodically post the insane catfishing your stuff that they all about money or I don't have I don't know why God bless people who have opened and I know how they do it just gluttons for punishment so overall are you happy that you got into all this does this disease change your perceptions of online communication

I feel like there's a

I have tried over the years whether it's conspiracy theorist communities or terrorists or or you know Russia Iran State the state-sponsored actor has the domestic ideologues I have tried to always say like here is the specific

kind of forensic analysis of this particular operation and then here is what we can maybe take from it and make make changes we've seen some of that begin to take shape and so I feel grateful to have had the opportunity to work towards connecting those dots and work towards having this conversation meaning meaning helping people understand what's going on I think

I am not

I am most concerned about

the

as this gets increasingly easy to do through things like chatbots you know now there's these the website. This person does not exist. I know there's a technology called The Time Machine learning technique generative adversarial networks and they are working to create pictures of people faces of people and so this website is that when you go to it it pulls up a computer-generated God not real people and so we have increasingly sophisticated technology we have increasingly sophisticated like you're not going to detect that image somewhere else that old trick of like right click and look and see if you're talking to someone with a stock photo that comes right out the window and stuff like this gets easier and easier to do

deepfakes on video front I think that it does change I think we haven't quite adapted to

what is it like to live in a world where so much of the internet is fake and I do think for your point about identity that there will be groups of people that self-select into communities where identity is mandatory you know where we're this is who you are and you have some sort of verification vs. people who choose to live in the world of a drink from the firehose take it all in and try to filter it out yourself so we look at these evolving Technologies and I don't necessarily feel you know particularly optimistic in the short-term I think that ultimately it does like we change as a society to a large extent and in response to this we think about

they're going to be some fixes at the platforms are going to be able to undertake they're going to be working to get better at detecting the stuff maybe in the adversary will evolve hopefully we get better at detecting it as it involves but

it's I think we fundamentally ultimately change like people become more where this is a thing they are more skeptical that does change our there are ways of interacting with each other but I feel like that is going to be the direction that this goes there's the more like you know the thing that keeps me up at night would be more the

the ease of of turning this from a social media problem until like a real world war problem meaning as an example back in 2014 or the first things internet research agency dead September 11th 2014 they created a hoax saying that Isis had attacked a chemical plant down in Louisiana it's called the Columbia the Columbia chemical plant hooks is I think there's a Wikipedia article about it now but what happened was they created a collection of websites they created fake CNN mock-ups Twitter accounts text messages it went to local people radio station Collins you name it everything to create the impression that a chemical Factory head just exploded in Louisiana and there was some attribution Tyson's son on September 11th so this is the kind of thing where this actually did go viral like I remember this happening not as a social media researcher I just remember it actually being pushed into my social media feed so you have these

we didn't know that it was in a research Agency for a year-and-a-half after but this is the kind of thing where you look at parts of the world that aren't the US like their recent drama between India and Pakistan and you can see how these kinds of things can go horribly horribly wrong if the wrong person is convinced that something has happened or if there's a lien or if this leads to our Riot orifice leads to real-world action I think that's one of the main fears as this gets better and better the video fix get better the people fix that better what you know

what do you do then so what do you do then when you see those images those images fake images those are stunning they're so good me just makes you wonder I mean we're going to get to a point where it if someone's not in front of you talking you going to look at a video you're not gonna have any idea what they're doing those deepfakes with a famous actresses faces and they put him in and it's it's a stunningly good means that it's amazing and their there they're also with someone like me I'm fucking doomed because there's thousands of hours of me talking of said everything so you could take this new programs for editing audio and you could splice together audio and video now generator just it's insane that generates your lip movements everything I mean it's really stunning it's really stunning and it's only going to get crazier and crazier and is going to be very difficult unless something is actually

how many running front of your face it's going to be very difficult to be able to differentiate and then I'm more worried about augmented reality and virtual reality in this stuff making its way into admin work at dive willingly with a big smile on her face into the Matrix playing like a month ago or something for the pain for cakes like I got 99 there wasn't barely HD back then I just wanted to see what it was like we washed it forgot how good it was and I to two and a half hours flew by so it's it's amazing how that seemed Preposterous in 99 or whatever it was like all this is just sci-fi and now you like hey this is Al

closer or dislike the idea of have you messed around at all with like HTC Vive or Oculus come back in five six years ago now and I tried one from I think it's USC right the Southern California is a bunch of really good Labs down here and I tried one where was like a zombie Holodeck simulator and it was it wasn't just the VR it was also is immersive so they had a backpack on me and it was it was actually scary as hell I was really good this is why I like I love first person shooters have been there so much fun but this was just the first timer in the in the game you have like a bat or something and nothing ever came to Market but damn was a good company called and they have this Wreck-It Ralph 1 and I did with my kids

can you put on a haptic feedback vest and you go through this environment and it's craving your it's very clear that you're in a video game that doesn't seem real but it is so much better than anything that existed five years ago and you go okay well what is it with the exponential increase in power of Technology was just going to be like in 10 days what's going to be like in 15 it's going to be impossible differentiate cuz now it's a vest it's just a vest you're not strapped into a chair that you can move around so you going through this whole Warehouse they have set up for these games and even picking up physical objects they look different in your hands and they do when you when you look at them without the headgear on we have one outside of Disneyland it's in Disney Downtown Disney it's called the void and then you go into these there's one Star Wars one is a Wreck-It Ralph on they have them in malls

small ones you Sydney's little eggs and you go on roller coaster rides and you fight off zombies and going to a haunted house it's getting weird and it's just you know I was a kid when the video games were these ridiculous Atari things we stuck cartridge in and you're playing pong and now we're looking at something that you're looking at these images of people you see pores you see that the glistening of their lips you see their eyes it's it's very strange it's very strange to think those are not really people and then we are probably going to look at all you have to do is create something that is so large and in the propaganda so terrifying that it causes you to act without double-checking triple checking and making sure you verify the facts something is really happened and then it sets into motion some physical act in the real world that you can't pull back like

in this is just not related necessarily but would happen with Hawaii when they got that false warning that nuclear missiles were headed their way he imagined was crazy and it was just someone hit the wrong button I mean if we come to some sort of a point in time where someone does something like that on purpose and shows you video that you really think New York City just got nuked and you know you have to head to the hills and there's a giant traffic jam on the highway and people start shooting each other I mean if Russia really wants to fuck with us what they're doing now with just dis you know the IRA agency and all these different trolls they've got set up that sort of trying to get people to get be in conflict with each other this is this is with primitive crude text and memes

what could be done in the future it's terrifying

blue the wheelbarrow

we're in agreement there thank you so much I really appreciate it thanks for coming down and thank you for all your work and exposing all the stuff that's really really interesting and terrifying people having get a hold of you so they control you on Twitter and you don't necessarily have an Instagram thank you and I really appreciate it

thank you thank you everybody thanks for tuning in to the show I hope you enjoy it as much as I did and I hope you enjoy the underwear that I'm wearing right now meundies if you could have me undies.com meundies.com Rogan go there and buy them you have a 100% money back satisfaction guarantee you can get 15% off your first pair free shipping and this one hundred percent satisfaction guarantee why do they give you a hundred percent satisfaction guarantee on underwear cuz they're better than any underwear you ever tried that's a fact no I'm telling you they're better than any other where I've tried a lot of different kinds of underwear including silk these things blow silk out of water I want you to go to meet undies.com Rogan to get that beautiful offer 15% off your first pair free shipping and a 100% satisfaction guarantee that's meundies.com

Rogan thank you also to Simply Save the best home security if you can get SimpliSafe has won awards from all the Tek experts account The Verge caused the best home security it 1 Readers Choice Award from PC Magazine two-time winner of the CNet editors choice and a Wirecutter top pick SimpliSafe as no contract no hidden fees and no gachas you can try SimpliSafe with free shipping and free returns you get a 60-day risk-free trial to order now and have your home protected within a week go to simplisafe.com Rogan to get started today that's simplisafe.com Rogan SimpliSafe simplisafe.com Rogan go there so that they know that we sent you and last but not least we are brought you once again by the motherfuking cash have the number one finance app in the

app store for a very good reason it kicks all kinds of ass and you can download it for free in the app store or the Google Play Market make sure you use the referral code Joe Rogan all one word $5 would go to you and $5 will go to support our good friend Justin Ren's fight for the Forgotten charity which is helping to build Wells for the pygmies in the Congo

thank you friends much love and appreciation to you all and will see you soon bye bye