Web Scraping Using Python and Selenium

How to extract data from a JavaScript populated web page.

Daniel Ellis Research
DataDrivenInvestor
Published in
4 min readMar 27, 2023

--

A picture of an iceberg — a metaphor to all the data sitting just under the surface of the web.
Photo by Annie Spratt on Unsplash

A very useful skill that keeps cropping up again and again (and again) is web-scraping. I have alluded to this in previous articles, but decided it is time it has its own.

What is Selenium?

--

--

Research Software Engineer specialising in High-Performance Computing and Data Visualisation. — PhD in Atmospheric Chemistry and Masters in Theoretical Physics.