I'm providing the complete UI design and all necessary data fields and am now looking for support with the technical aspects. Specifically, I want to create a price comparison portal that automatically retrieves prices and product details from around ten target websites, aggregates them, and presents them in a clear and organized way. What's already available: • Finished layout mock-ups (desktop + mobile) • List of shops to be scraped, including the desired data points • Basic logic for sorting and filtering, documented as pseudocode Your task: • Develop a robust web scraping script (preferably Python + BeautifulSoup / Scrapy or similar tools). • Extract price, availability, product name, and shipping costs (if applicable). • Store the data in a structured format (e.g., MySQL, PostgreSQL, or JSON → you can suggest a suitable solution). • Provide an API or an easy-to-integrate data endpoint so I can easily connect the front end. • Implement an update interval (cron job or scheduler) to ensure prices are updated daily. • Build in simple error logs and duplicate checks to guarantee data quality. Acceptance Criteria 1. All ten shops are retrieved without manual intervention. 2. Endpoint response time remains less than 1 second for 100 products. 3. The scraper doesn't completely abort when layout changes occur, but instead logs the affected fields. 4. The setup instructions can be executed in a fresh environment in less than 30 minutes. If you have experience with anti-bot bypassing (CAPTCHAs, rate limits) and scalable scraping architectures, please mention it briefly. I look forward to your technical assessment and a rough roadmap for implementation.