How to Detect AI Content with Keystroke Tracking

Published on

November 15th, 2024

Author

Mathew Hardy

Building an AI keystroke dataset

Consider this question:

Describe your morning commute - how do you get to work and how long does it take?

And two responses:

I drive and leave home around 8. It takes about 20 minutes.
My morning commute covers about 5.2 miles (8.4 kilometers) total. I walk half a mile (0.8 km) from my apartment to Ashmont Station, then take the Red Line subway for about 4.3 miles (6.9 km), followed by another 0.4 mile (0.6 km) walk from the downtown station to my office building. The whole trip typically takes 35 minutes door-to-door.

The AI response is obvious - it includes distance conversions, unnecessary details, and perfect paragraph structure. While humans could write this way, they rarely do in a rushed online survey.

AI keystroke patterns

By focusing on questions where human and AI responses clearly diverge, we built a large dataset of labeled responses and their corresponding typing patterns. While the content was often similar, the typing patterns revealed consistent differences.

Human typing has a natural variance. The time between keystrokes follows a characteristic distribution with random pauses and bursts, with pauses most likely at the end of words and phrases. For example:

Humans also make typos and use backspaces and other editing techniques to fix them. By contrast, AI responses typically show unnaturally consistent intervals between keystrokes with minimal variance, and almost never include corrections or backspaces:

This level of keystroke consistency is virtually impossible for humans to replicate.

Other AI responses look like human-AI hybrids, where AI pipes in text programmatically and humans then edit the responses:

Real-world impacts

To demonstrate how bad actors corrupt survey data, we ran a test study about an intentionally bad product - a solar-powered refrigerated hat. By classifying responses as “Human” or “AI”, we found that the AI responses created two types of problems in downstream analysis.

First, AI responses showed consistently higher willingness to pay ($135 for AI vs. $40 for humans), replicating a general pattern of AI optimism compared to human responses.

Second, AI responses added noise. Whereas human responses formed logical customer segments (motivated, indifferent, etc.), the flagged data formed nonsensical ones - for example, a segment rating the hat highly useful but being unwilling to pay.

Conclusion

Identifying AI content through text analysis is extremely difficult and usually unreliable. We suggest that analyzing typing patterns can be more robust. The complex keystroke data provides clear signals that are much harder to fake (and often more understandable) than content alone. Of course, this is a cat and mouse game. As detection methods evolve, so do evasive techniques, and we're constantly updating our models.

Mathew Hardy is the CTO and co-founder of Roundtable, an AI fraud detection platform.

Comments (3)

Cam Doody @camdoody

Bellhops

Gamechanger

Nov 21

Archibald @archibald2

BypassGPT may be a good tool to check AI Content. AI tools like Claude, perplexity, Kimi, and so on help me a lot to boost my website volumes. Ersa Electronics.

Dec 17

Adam Harris @adam_harris7

কয়েক সপ্তাহ আগে আমি এমন একটি প্ল্যাটফর্ম খুঁজছিলাম যা ব্যবহার করা সহজ এবং আমি এই 22betbd appটি খুঁজে পেয়েছি। সত্যি বলতে, আমি প্রথমে খুব বেশি আশা করছিলাম না, কিন্তু কিছু ব্যবহারের পরে, আমাকে স্বীকার করতে হবে যে এটি আমার প্রত্যাশা ছাড়িয়ে গেছে। অ্যাপটি কোনো সমস্যা ছাড়াই লোড হয়েছে এবং সাইটটি নেভিগেট করা সহজ ছিল। আমি ইতিমধ্যে কয়েকটি ছোট বাজি জিতেছি এবং বোনাস সিস্টেম কীভাবে কাজ করে তা মূল্যায়ন করছি। এখন পর্যন্ত এটি একটি আনন্দদায়ক অভিজ্ঞতা হয়েছে। সেরা অংশ হল কত দ্রুত প্রত্যাহার - এটা আশ্চর্যজনকভাবে সহজ! আপনি যোগদান করার সময় একটি সলিড ডিপোজিট বোনাস দিয়ে শুরু করুন।

Dec 16