# Market Price Scraper - Setup Guide

## Current Issue

The marketkarsilastir.com website uses Cloudflare protection that blocks simple HTTP requests. We need a headless browser approach.

## Solution Options

### Option 1: Node.js Headless Browser (Recommended)

#### Prerequisites

```bash
# Install Node.js (if not already installed)
# Download from: https://nodejs.org/

# Install Puppeteer
npm install puppeteer
```

#### Usage

```bash
# Test the scraper
node headless_scraper.js
```

### Option 2: Laravel Integration

#### Setup

1. Install Node.js and Puppeteer (as above)
2. The `HeadlessBrowserScraper.php` service is ready to use
3. Update your MarketPriceScraper to use the new service:

```php
// In your controller or service
use App\Services\Scrapers\HeadlessBrowserScraper;

$scraper = new HeadlessBrowserScraper();
$result = $scraper->searchByEan('8690526076379');
```

### Option 3: Alternative Data Sources

Consider these alternatives:

-   **Official APIs**: Check if markets provide official price APIs
-   **Different comparison sites**: Find sites without bot protection
-   **Manual data entry**: For critical products
-   **Proxy services**: Use residential proxies to bypass restrictions

## Troubleshooting

### Common Issues

1. **Puppeteer installation fails**

    ```bash
    # Try with additional flags
    npm install puppeteer --unsafe-perm=true
    ```

2. **Chrome sandbox issues (Linux servers)**

    ```bash
    # The script already includes sandbox disabling flags
    # If still having issues, ensure proper permissions
    ```

3. **Timeout errors**
    ```bash
    # Increase timeout values in the script
    # Current: 30 seconds for page load
    ```

## Performance Considerations

-   Headless browser is slower than HTTP requests (~5-10 seconds per product)
-   Consider caching results
-   Implement rate limiting to avoid IP blocking
-   Monitor resource usage on your server

## Legal Disclaimer

Ensure your scraping complies with:

-   Website Terms of Service
-   Local laws and regulations
-   Data privacy requirements