Visible to the public AICW - The Dangers of the Subconscious Mind (of Cyber Reasoning Systems)

ABSTRACT: Humans have goals, hopes, dreams, and fears. Humans are brilliant. They make incredible intuitive inferences. They conceptualize amazing algorithms to augment cybersecurity. But they can be misled; tricked; foole d into carrying out actions counter to their own best-interests.

The Cyber Grand Challenge propelled program analysis algorithms from mere tools to autonomous Cyber Reasoning Systems. These systems can operate independently to find, exploit, and mitigate vulnerabilities in sof tware, and under various programs and initiatives in the years since the CGC, they have continually improved on their humble beginnings. But similar to otherwise-intelligent and otherwise-autonomous humans, they can also be misled.

What weaknesses exist in the subconscious minds of Cyber Reasoning Systems? Can their dreams turn into nightmares? Can their hopes and goals be hijacked? Of course, the answer is yes. This talk will explore some of the concrete, technical routes to this sort of hijacking, both in terms of what existed in the Cyber Grand Challenge, what has been developed since, and what emerging disruptions and coersions might look like in the future.

BIO: Yan Shoshitaishvili is an assistant professor at Arizona State University, where he pursues research in automated program analysis and vulnerability identification techniques. As part of this, Yan led Shellphish's participation in the DARPA Cyber Grand Challenge, applying his research to the creation of a fully autonomous hacking system that won third place in the competition. Underpinning this system is angr, an open-source binary analysis project created by Yan (and others!) over the years. When he is not doing research, Yan is pushing the area of cybersecurity competitions into the future from his position on the Order of the Overflow, the organizers of DEF CON CTF.

Creative Commons 2.5

Other available formats:

AICW - The Dangers of the Subconscious Mind (of Cyber Reasoning Systems)
Switch to experimental viewer