The Challenge
Finding the right school in Southern California can be a daunting task for parents and guardians. The challenge was to create a centralized, comprehensive, and easily accessible database of schools that provides all necessary information in one place, allowing for informed decision-making.
The Approach
1. Dynamic Web Scraping: Automated extraction of comprehensive school data from multiple educational portals using Python & Selenium. This ensured the dataset was robust and up-to-date.
2. Data Enrichment & Geocoding: Enhanced the dataset with geographical coordinates and other relevant information using Python and geopy, making it more valuable and user-friendly.
3. Efficient Data Warehousing with Snowflake: Utilized Snowflake's cloud platform for data storage, along with Snowpipe for seamless, automated data ingestion, ensuring data integrity and accessibility.
4. Interactive Data Visualizations: Created insightful charts and maps with Tableau, making complex data easily interpretable and visually appealing for users.
The Outcome
The project successfully created a comprehensive and interactive web dashboard that provides detailed insights into schools across Southern California. Parents and guardians can now make well-informed decisions based on various metrics and geographical information, significantly simplifying the school selection process.
Users Can:
- Access a centralized database of schools in Southern California.
- View detailed information on each school, including geographical data.
- Interact with dynamic and insightful visualizations to compare schools.
- Make informed decisions about their children's education based on comprehensive data.
Key Features
- Dynamic Web Scraping: Automated extraction of comprehensive school data from multiple educational portals using Python & Selenium.
- Data Enrichment & Geocoding: Enhancement of the dataset with geographical coordinates and other relevant information using Python and geopy.
- Efficient Data Warehousing with Snowflake: Utilization of Snowflake's cloud platform for data storage and ingestion.
- Interactive Data Visualizations: Creation of insightful charts and maps with Tableau, making complex data easily interpretable.
Technologies Used
- Python & Selenium: For web scraping
- geopy: For data enrichment and geocoding
- Snowflake & Snowpipe: For data storage and ingestion
- Tableau: For data visualization