Interview with Chris Caren: Turnitin.com CEO on AI's Double-Edged Sword in Education and Beyond

In an interview with AI Forum, Chris Caren, the CEO of TurnItIn.com, provided a deep dive into the multifaceted roles of artificial intelligence (AI) in today's educational landscape and its broader implications across various sectors.

Caren emphasized the value proposition of Turnitin.com, stating, "We believe Turnitin.com brings to the world... enforcing integrity in education, as well as research." He highlighted the company's innovative approach to combat the rising trend of AI-generated academic misconduct. With the advent of AI tools like Chat GPT, students can now outsource their assignments, a challenge Turnitin.com is actively addressing. "We’ve built an accurate way to detect AI writing to ensure a journal publisher or professor understands the role the student played in writing the end product," Caren explained.

While AI's potential to enhance productivity, streamline operations, and improve customer experience is undeniable, Caren also expressed significant concerns, especially in the realm of cybersecurity. "The biggest negative we see is really cybersecurity... So we're spending a good amount of money building up that team, and investing more in cyber," he remarked.

Beyond the educational sector, Caren touched on AI's broader societal impacts. He discussed its potential to revolutionize the legal profession, citing instances where lawyers outsourced their work to AI tools, leading to inaccurate legal references. "Our view is that it will help automate a lot of just manually intensive work... But not the really important work. The factual issues or hallucinations of models is a real issue," Caren noted.

In the entertainment industry, the ongoing Hollywood strike underscores the debate around AI's role in potentially replacing writers or actors. Caren mused on the future, saying, "There will be movies made... that look like Marvel movies with a new superhero that some creative person has described to an AI tool." He also raised ethical questions about reviving deceased actors for new roles and the ensuing compensation debates.

As AI continues to weave itself into the fabric of various industries, Caren's insights underscore the pressing need for industries to adapt, innovate, and set clear, ethical guidelines on its usage.

________________________________________________________________________

Interview by AI FORUM with Chris Caren, CEO of Turnitin.com, August 3, 2023

AI FORUM:

So just to start with a broad question, as a CEO do you see AI posing a specific set of new problems or issues for you and your company?

Chris Caren:

Certainly, for the value propositions we believe Turnitin.com brings to the world, which is really enforcing integrity in education, and as well as research. Because AI is potentially a form of misconduct, we’ve built an accurate way to detect AI writing to ensure a journal publisher or professor understands the role the student played in writing the end product. Our customer surveys indicate that we're bringing a lot of value to educators and research publishers to date by doing this. 

Having said that, I think AI is overall going to be much more beneficial to learning outcomes than it is negative.  

For our company, the biggest AI risk we see is really cybersecurity, so we're spending a good amount of money building up that team, and investing more in educating our employees about social engineering exploits.

We view AI as a productivity enhancer for our company in the medium and long term, because it's going to let same number of people either get more work done, or allow for smaller teams.  We are also going to see beenfits like massively increased productivity for software developers, much higher quality and more efficient customer service, whether that's chat or email based support.

AI FORUM:

How has the widespread ability of AI since the publication of Bard, Bing and Chat GPT impacted the education space, given the publicity about fake term papers and the like?

Chris Caren:

There are many great use-cases for AI in student research, closing learning gaps, and brainstorming.  The core downside use-case we’re focused on is deterring students from outsourcing their writing assignments to generative AI engines -- most notably Chat GPT-- and get a solid grade without having any learning take place. And that's a real loss for the student in the end, but also for their classmates that did the hard work but are graded on a curve with that student.

We also need to keep in mind there's a need for educators to test how well AI without any help can do to existing writing assignments. If ChatGPT can get an unassisted A or B grade, writing assignments need to be made more difficult, ideally before back to school this year.  ChatGPT and Bard can help teachers – just ask these AI’s how to adapt an assignment so that the student needs to do the work, and in the process grow and learn.

Schools and teachers also need to be really explicit on how AI should be used by students. There'll be some writing assignments where students will be allowed to use it as much as they want, or as little or as little as teachers want.  

Also, students need to learn to properly cite the use of AI, and what role it played in the final product.  

Our company has an algorithm that can very accurately detect AI writing versus human writing. So, we can effectively give a teacher report saying, here are the sentences in a paper written by AI with high accuracy and here are the sentences written by the students.   

AI FORUM:

How does your technology accomplish this?

Chris Caren:

We have developed four vectors to identify the source of work product. The first two are both fairly common.  Large Language Models today have a very distinct frequency of words of word choice.  A word frequency signature from an LLM is different than a human. LLM’s also tend to be not very “bursty,” meaning they tend to write the same length sentences and stay on topic much more than a human does. Humans are more “bursty,” writing short and long sentences, skipping around different thoughts and topics.  Those are two common ways that we have to detect the source in our algorithm. 

The third is we can understand a student's true writing style and fingerprint that and then compare that to future submissions. Now, that could mean there's contract cheating from someone for hire that's written the paper or it could mean the paper was written by an LLM. 

But the forth and most effective model is a black box we trained and now use. We took 10,000 essays that we know students wrote, then we asked AI to write essays to the same assignment or prompts.  We then just told the black box, there are human written, and these are AI .. figure out how to tell them apart.  We found that AI does a remarkably good job. It’s highly accurate when trained on a large corpus of known human and known AI produced essays. 

AI FORUM:

Do you use your own custom model?

Chris Caren:

Yes, this forth approach is a custom model. 

We also plan to build new models for internal use, like in software development, or support, we'll take a base model, and they're all fairly strong, and then build a custom module on top of that with our proprietary content. 

AI FORUM:

Switching gears, the question that we have is, “Will AI going to revolutionize cybersecurity? And is it for better or for worse, or both?”

Chris Caren: 

I think the short answer is both. It's going to be an arms race.  People who are bad actors will find new and novel ways to penetrate a database or an application through social engineering. These AI tools can also use brute force creativity that never gets tired to find attack vectors into software applications or databases. So, at the same time, companies that lead in cybersecurity and sell solutions need to identify new forms of attack and find a response and scale it up their base of customers very quickly. But I think it's going to be harder, and companies will end up spending more money on cyber than they have historically. It's my best guess at the moment.

AI FORUM:

So thinking of the cyber and social engineering threat, what would your advice be right now on the consumer side of the fence?

Chris Caren:

I'm really worried on the consumer side of the fence. Yes,we should be concerned about people who just really don't have their guard up .. and so we do a lot of internal training, and also send fake emails and SMS messages to employees to keep them on their guard. I think these new AI capabilities are also going to pose a major fraud problem at the individual level, especially people who are elderly. 

AI FORUM:

Thinking about the impact of AI on the legal profession, people have speculated that it's going to revolutionize the way the law is practiced. Do you have a view on that?

Chris Caren:

Our general view is that it will help automate a lot of just manually intensive work, reviewing contracts, producing contracts, etc. I've heard of firms that specialize in areas like trusts and wills being able to produce documents much more quickly.

So I think it will certainly reduce the size of a team, a company needs to get a lot basic legal work done, but not the really important work. The factual issues or hallucinations of AI models is a real issue.

AI FORUM:

You’ve talked several times about the boon to education, so please expand on that a little bit in terms of the benefits to our education system that might come from adoption of AI.

Chris Caren:

Bill Gates was at a conference I attended and asked the audience to basically imagine every student around the world -- not just people in the first world but those who will have very impoverished situations -- will have access to the world's best tutor who will understand exactly how the student learns, what they know what they don't know, what they're best learning style and format is, how to best keep the student engaged and enjoying the learning.  That could be available to every student at a near zero price. 

Just think about how much more and how much higher quality learning is going to take place. It won't replace teachers and faculty members, but it certainly will change their role tremendously.

AI FORUM:

It’s encouraging that people are thinking of those scenarios and creating a more level educational playing field with access to the best kind of teachers.   What about threats posed by AI?

Chris Caren:

I do worry a lot about cyber not only about breaking into companies or take money from humans, but breaking into the nuclear weapons system of a country . . . that's a real risk.  Also, there’s the prospect of engineering a virus that is much more lethal than COVID and has a long incubation period. So we need to really worry about those type of possibilities.

AI FORUM:

So Chris, you've worked in Silicon Valley for a while and witnessed big technological developments.  Does this new AI wave remind you of previous inflection points in technology?

Chris Caren:

Sundar Pichai CEO of Google called is ‘as big as the invention electricity.’ That could be the case over time, but I certainly think it's on par with internet and mobile.

A few years from now a lot of people will essentially have close friends who are AI avatars that are as realistic as a human and also interesting and curious and funny.  Will our kids in 10 years say that their best friends are AI avatars or real people? That kind of potential social isolation is scary … reminds me of the movie “Her.” 

Unlike many of the past tech bubbles, like crypto, or web 3.0, or blockchain that was over hyped, this is very real. 

AI FORUM:

This last question is sort of more of the current events question. We’re seeing the strike in Hollywood now, by writers and actors, and one of the issues is how AI will be replacing writers or replacing actors.

Chris Caren:

There will be movies made in the future that will look like Marvel movies with a new superhero that some creative person has described to an AI tool that then turns it into a great new high quality movie, because it has been trained on all the Marvel content. It seems intuitive that prior art be allowed to share in the financial benefit. 

The strike raises crazy issues about actors and actresses who have died.  Can they be revived to play new roles? Who gets the compensation for that-- Is it their heirs? Is it the studio that created the original movie? I think that there are many, many unanswered issues there.

Transcribed by https://otter.ai