Web Scraping Engineer

ROLE SUMMARY

Our client is looking for a Web Scraping Engineer who will be navigating websites to extract and save data using the necessary tools to accomplish extracting text data or saving pages to PDFs. 

SCHEDULE: 8:00 AM – 5:00 PM Central Daylight Time (9:00 PM – 6:00 AM Philippine Standard Time), follows Philippine holidays

POSITION TYPE: Full Time

WORK ARRANGEMENT: Remote

ESSENTIAL FUNCTIONS

·       Create processes to search sites given a set of data to retrieve information

·       Save entire result pages to PDF using a standardized file naming convention and insert a link to the file into the database

·       Extract certain data fields from the results page and then standardize them to be uploaded into a database for analytics

·       Diagnose and resolve any issues that may occur with the automated process developed for scraping as the sites change on occasion

QUALIFICATIONS

·       A Bachelor’s degree in the related field is preferred

·       Familiarity with techniques used for crawling, extracting, and processing data from websites into PDF files and SQL databases (HTML, Java, and SQL)

·       Experience with macro scripting (AutoHotkey and Microsoft Office VBA)

·       Knowledgeable in bypassing bot detection techniques (VPN, proxy, behavioral criteria, and request counts)

·       Knowledgeable in countering scraping countermeasures

·       Strong troubleshooting skills to make needed adjustments

·       Analytic skills to develop processes and compare results

·       Experience in a multi-client environment

·       Strong organization, oral and written communication skills

·       Aptitude in data management, analytics, reporting preparation

·       Ability to function in an autonomous environment—independent worker, self-directed