I need the full text pulled from a set of paid educational websites that I will specify once the project begins. navigating through the course or article structure, and extracting every piece of textual content—titles, body copy, headers, footnotes, and any inline downloads such as PDF transcripts—into a clean, reusable data set. Python (Scrapy, BeautifulSoup or similar) or another reliable scraping stack is fine as long as the code is well-commented and easy for me to rerun whenever new material is added. Handle pagination, infinite scroll, and any lazy-loaded sections so that nothing is missed. Deliverables • A working, modular script with a requirements file • One consolidated CSV or JSON containing: page URL, title, publication date (if present), and the full text body • A short read-me explaining setup, execution, and how to update the target list in future I will test by running the script on additional pages behind the same paywall; the output should match the site content exactly, without formatting artifacts or missing sections.