Kati Kamayoo

Features

Comprehensive Data

Scrapes product titles, current prices, original prices, and amount sold for products listed on Daraz Nepal.

Automated Scraping

Uses Selenium to automate the process of fetching data from the website.

Data Storage

Stores scraped data in a MySQL database for easy retrieval and management.

Data Visualization

Integrates Flask and Chart.js to render and visualize product data in interactive charts and tables.

About Kati Kamayoo

While browsing Daraz, I noticed a brand with over 1,000 items sold across multiple products. This made me wonder, “How much is this brand actually earning from all these sales?” Since I lacked access to this data, I decided to create a tool that could provide these insights. What started as a curiosity-driven project turned into a rewarding learning experience. The journey has been both challenging and educational, and while the tool is not yet fully complete, it has been a fascinating endeavor.

Kati Kamayoo is designed to extract detailed product information from Daraz Nepal using Selenium for automation. It gathers data such as product titles, prices, and sales figures, which is then cleaned and stored in a MySQL database. This data is served through Flask and visualized with Chart.js, offering dynamic and interactive insights. The project is open to contributions, and I encourage anyone interested to explore and help enhance its features. Your input can make a significant difference in refining this tool.

Technologies Used

Selenium - For web automation and scraping.

MySQL - For storing and managing the scraped data.

Flask - For serving data through a web framework.

Chart.js - For creating dynamic, interactive visualizations.

Getting Started

Follow these steps to set up Kati Kamayoo on your local machine:

1. Clone the Repository:

git clone https://github.com/rewaj56/rewaj56.github.io.git

cd kati-kamayoo

2. Install Dependencies:

pip install -r requirements.txt

3. Set Up MySQL Database:

Create a MySQL database and user.

Update the database configuration with your credentials.

4. Run the Scraper:

python scraper.py

5. Start the Flask App:

python app.py

Visit http://localhost:5000 in your web browser to view the app.

Supported URLs and Known Limitations

Our current web scraper is designed to work with specific URL formats. It effectively scrapes data from the following URLs:

Brand	URL
Anker	https://www.daraz.com.np/anker/
Xiaomi	https://www.daraz.com.np/xiaomi/
Redmi	https://www.daraz.com.np/redmi/
Fantech	https://www.daraz.com.np/fantech/
Erke	https://www.daraz.com.np/erke/
Samsung	https://www.daraz.com.np/smartphones/samsung-brand/
Apple	https://www.daraz.com.np/apple/
Midea	https://www.daraz.com.np/midea/
Masala Beads	https://www.daraz.com.np/fashion-jewellery/masala-beads/
Philips	https://www.daraz.com.np/home-appliances/philips/
Realme	https://www.daraz.com.np/realme/

These URLs are compatible with our scraper due to their structure and content format. After analyzing these working URLs, you may find similar URLs that could also be compatible. Feel free to test additional URLs and add them to your list to see if they work with the scraper.

Limitations:

The scraper does not work with all URLs from Daraz Nepal. Only specific URL formats are supported.
The scraper extracts products only from the first page of each URL. Pagination is not currently handled.
Data is up-to-date but may not always reflect true product values due to price changes, promotions, and data discrepancies.
This scraper depends on specific HTML class names, which may change, leading to potential data extraction issues.
Historical data is lost each time the scraper runs, as the table is cleared and refreshed.

Project Screenshots

Key Insight: The bar chart illustrates that Apple generates the highest revenue among various brands, greatly surpassing its competitors. In contrast, brands such as Erke and Realme have much lower earnings, placing them at the lower end of the spectrum. This stark contrast highlights Apple's dominant market presence and reveals a notable financial disparity among brands in the industry.

Key Insight: The scatter plot shows Total Items Sold on the x-axis and Total Earnings on the y-axis for various brands. Apple, with relatively low sales, earns much more, reflecting its premium pricing. On the other hand, Masala Beads, despite selling the most items, has lower revenue, showing that high sales volume doesn't necessarily mean high earnings.

Key Insight: The table provides detailed product information, such as Product Title, Current Price, Amount Sold, and Total Earnings. It also totals the earnings at the bottom. This layout helps evaluate each product's performance, showing which ones drive the most revenue and which perform less well. It offers valuable insights into how each product contributes to overall earnings.

Unlock Sales Insights from Daraz!

Find out how much they made, right here.