Add Puppeteer browser scraping and HG Spot store config
- Add browser-scraper.ts using Puppeteer for JS-heavy stores - Add render_js flag to store model, migration, YAML sync, and UI - Scraper engine auto-selects cheerio vs Puppeteer based on flag - Store forms include JS rendering toggle in Advanced section - Create first store config: HG Spot (Croatian electronics retailer) - Update Dockerfile with Chromium for production Puppeteer support Tested: HG Spot returns 15 products per page with correct names, prices (EUR), links, and images using headless browser rendering. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -19,6 +19,11 @@ RUN npx tsc
|
||||
FROM node:20-alpine
|
||||
WORKDIR /app
|
||||
|
||||
# Install Chromium for Puppeteer
|
||||
RUN apk add --no-cache chromium
|
||||
ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium-browser
|
||||
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true
|
||||
|
||||
COPY package*.json ./
|
||||
RUN npm ci --omit=dev
|
||||
|
||||
@@ -29,6 +34,7 @@ COPY src/server/db/migrations ./dist/server/db/migrations
|
||||
ENV NODE_ENV=production
|
||||
ENV PORT=3000
|
||||
ENV DATABASE_PATH=/app/data/pricehunter.db
|
||||
ENV STORES_DIR=/app/stores
|
||||
|
||||
EXPOSE 3000
|
||||
VOLUME /app/data
|
||||
|
||||
Reference in New Issue
Block a user