We are looking for a Data Scraper, you will own the creation process of these tools, services, and workflows to improve crawl/ scrape analysis, reports, and data management. We will rely on you to test the data and the scrape to ensure accuracy and quality. You will own the process to identify and rectify any issues with breaks as well as scale scrapes as needed.
- Extracting and ingesting data from websites using web crawling tools
- Test the data and the scrape to ensure accuracy and quality
- Gather and process raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.)
- Solid Python knowledge.
- Familiarity with techniques and tools for crawling, extracting, and processing data (e.g. Scrapy, Pandas, MapReduce, SQL, BeautifulSoup, etc).
- Great communication skills both verbal and written.
- Experience running large-scale web scrapes.
- Experience with system monitoring/administration tools.
- Experience with version control, open-source practices, and code review.
- Experience with applications designed to display archived web content.