Regular comprehensive snapshot. There are 3,000 products. 20 columns for each product. Page by page. I’m looking for a repeatable, fully automated workflow. A Python-based stack (Scrapy, BeautifulSoup, Selenium, Playwright, or an equivalent you prefer). Robustness is key: the crawler should cope with pagination, JavaScript-rendered. Clear, well-commented code is part of the deliverable so my team can review and rerun it internally. Each quarterly hand-off must include: • Cleaned CSV or JSON containing the structured product records • The raw HTML or a compressed WARC snapshot for auditing • The executable script(s) plus a brief change log highlighting any site-structure updates you handled Please outline your proposed tool chain, an example of a large scrape you have completed, and the turnaround time you can guarantee per cycle. I’m ready to start as soon as I find the right partner for this long-term data pipeline.