Matthew Valites had his first online experiences as a tween in the early 1990s, dialing into local BBSes on the family's home-built Radio Shack Tandy and clogging the phone lines playing Doom death matches with friends. By 14, he had ground his first set of lockpicks out of his dad's hacksaw blades, subsequently ruining every lock in the house. Under the tutelage of his older brother in the early 2.0 kernel days, Matt learned how to run Linux, forever influencing the rest of his professional life. He's the author of O'Reilly's Crafting the InfoSec Playbook, creator and host of Wall of Sheep's "Lockpick Gauntlet," and a forever curious maker and breaker.
Back Seat Webdriving via Browser Automation
There are many reasons to automate web browsing for security purposes, from scraping websites, to request manipulation, to task automation. Staid tools like wget and curl are a good start. But the modern web is dynamic and often client-side, limiting the effectiveness of these tools. Luckily, most modern web browsers provide webdriver engines that, when coupled with an automation framework, allow users near limitless methods to automate interactive web browsing sessions as if they were interacting with the browser themselves.
This talk will share basic concepts and advanced tips and tricks from years of experience automating web browsers using automation frameworks like Selenium. It will begin by discussing common methods of web automation, the Document Object Model and how to use it, and how webdrivers work with automation frameworks. From there more advanced topics will be explored such as browser configuration for research, headless browsing, interacting with modals, dealing with catpchas, and logging all the things. Code snippets will be provided along the way, including multiple methods of solving most problems.