diff --git a/docker-compose.yml b/docker-compose.yml index 8476db94..afc87f66 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -76,7 +76,7 @@ services: - wanderer restart: unless-stopped healthcheck: - test: ["CMD", "/curl", "--fail", "http://localhost:3000/"] + test: ["CMD", "curl", "--fail", "http://localhost:3000/"] interval: 15s retries: 10 start_period: 20s diff --git a/docs/src/content/docs/use/authentication.md b/docs/src/content/docs/use/authentication.md index b2bc416b..6d970d70 100644 --- a/docs/src/content/docs/use/authentication.md +++ b/docs/src/content/docs/use/authentication.md @@ -47,6 +47,21 @@ If your instance offers OAuth logins, the enabled providers appear in wanderer account without requiring your login credentials. + +:::danger +API tokens grant full access to your account. Do not share them with untrusted parties. +::: + +To manage your tokens: +1. Log in to your wanderer instance. +2. Navigate to **Settings** > **Account** > **API Tokens**. +3. Click **Generate new token**, provide a descriptive name and optionally an expiration date and click "Save". +4. **Copy the token immediately.** For security, it will not be shown again. + + ## Forgot your password? wanderer offers the option to send password reset emails in case a user forgets his password. You can click the "Forgot password" link in the login form. After requesting the reset the user will receive an email with a unique link to reset their password. \ No newline at end of file diff --git a/docs/src/content/docs/use/import-export.md b/docs/src/content/docs/use/import-export.md index 79889ec0..7a2e4653 100644 --- a/docs/src/content/docs/use/import-export.md +++ b/docs/src/content/docs/use/import-export.md @@ -5,37 +5,43 @@ description: How to import and export trails in wanderer ## Import -wanderer supports bulk uploading of trails via an auto-upload folder. A cronjob fetches all files from this folder and uploads them automatically every 15 minutes. This feature is currently only available for docker installations. If you want to replicate it in a bare metal installation you will need to create your own cronjob using the `web/cron.sh` script. +wanderer supports bulk uploading of trails via an auto-upload folder. A file watcher automatically detects new files added to this directory and imports them into your library. + +:::note +This feature is currently only available for Docker installations. Files added to the folder while the container is not running are ignored. +::: :::caution Successfully uploaded files will be deleted from the auto-upload folder. ::: -:::note -Currently only GPX files are supported. -::: ### Configuration -The following environment variables must be present in the `wanderer-web` docker container and set to valid values. +#### Environment variables -| Environment Variable | Description | Default | -|----------------------|------------------------------------------------------------------------|--------------| -| UPLOAD_FOLDER | Path to the auto-upload folder | /app/uploads | -| UPLOAD_USER | Username of the account that will be the author of the uploaded trails | | -| UPLOAD_PASSWORD | Password of the account that will be the author of the uploaded trails | | +The following environment variable must be present in the `wanderer-web` docker container and set to a valid volume path (see below). -### Volume +| Environment Variable | Description | Default | +| -------------------- | ------------------------------ | ------------ | +| UPLOAD_FOLDER | Path to the auto-upload folder | /app/uploads | + + +#### Volume Make sure to mount the upload folder as a volume to your host system. The default `docker-compose.yml` already includes this volume. Ensure that the mapped value matches the one in the `UPLOAD_FOLDER` environment variable. -### Manually run the upload job -In case you do not want to wait until the next scheduled execution you can also run the job manually: +#### API token +The bulk upload process uses API tokens to authenticate requests and determine which user account the uploaded trails should be assigned to. + +1. Create an API token: Follow the steps in the [Authentication section](/use/authentication/#api-tokens) to generate a new API token. +2. Prepare the folder structure: Create the folder: Inside your UPLOAD_FOLDER, create a sub-folder named exactly after your API token. +3. Upload: Move your trail files (e.g., .gpx, .fit, or .kml) into that sub-folder. + +**Example structure**: + `/app/uploads/wanderer_key_<...>/my_trail.gpx` -```bash -docker exec -it wanderer-web run-parts /etc/periodic/15min -``` ## Export -To export a single trail head over to `/trails` and select the trail you want to export. From the ⋮ menu select "Export". You can export the route data either in GPX or in GeoJSON format. Furthermore, you can choose whether you want to include the photos and the summit book of the trail. In any case, wanderer will create a ZIP archive with all the data that is then downloaded. +To export selected trails head over to `/trails` and select the trails you want to export. From the ⋮ menu select "Export". You can export the route data either in GPX or in GeoJSON format. Furthermore, you can choose whether you want to include the photos and the summit book of the trail. In any case, wanderer will create a ZIP archive with all the data that is then downloaded. You can also export all of your trails at once. To do so, head over to `/settings/export` and click "Export all trails". The other steps remain analogous to exporting a single trail. diff --git a/web/Dockerfile b/web/Dockerfile index eaa67423..55b6ebe5 100644 --- a/web/Dockerfile +++ b/web/Dockerfile @@ -1,33 +1,3 @@ -FROM curlimages/curl:8.18.0 AS download-env - -# renovate: datasource=github-releases depName=stunnel/static-curl packageName=stunnel/static-curl -ENV CURL_VERSION=8.18.0 - -RUN set -eux ; \ - ARCHITECTURE="$(uname -m)" ; \ - case $ARCHITECTURE in \ - x86_64) ARCHITECTURE="x86_64" ;; \ - aarch64 | armv8* | arm64) ARCHITECTURE="aarch64" ;; \ - *) \ - echo "(!) Architecture $ARCHITECTURE unsupported" ; \ - exit 1 \ - ;; \ - esac ; \ - curl \ - --connect-timeout 10 \ - --fail \ - --location \ - --max-time 300 \ - --output /tmp/curl.tar.xz \ - --proto '=https' \ - --show-error \ - --silent \ - --tlsv1.2 \ - "https://github.com/stunnel/static-curl/releases/download/${CURL_VERSION}/curl-linux-${ARCHITECTURE}-glibc-${CURL_VERSION}.tar.xz" \ - ; \ - tar -xJf /tmp/curl.tar.xz -C /tmp ; \ - chmod +x /tmp/curl ; - FROM node:22-alpine AS build-env WORKDIR /app @@ -45,12 +15,11 @@ FROM node:22-alpine WORKDIR /app/uploads WORKDIR /app -COPY --from=download-env /tmp/curl /curl COPY --from=build-env /app /app -COPY ./cron.sh /etc/periodic/15min/cron -RUN chmod +x /etc/periodic/15min/cron +RUN apk add --no-cache curl +COPY watcher.js /app/watcher.js -CMD crond && node build +CMD ["npm", "run", "start"] EXPOSE 3000 diff --git a/web/cron.sh b/web/cron.sh deleted file mode 100644 index 9569d6e6..00000000 --- a/web/cron.sh +++ /dev/null @@ -1,62 +0,0 @@ -#!/bin/ash - -# API endpoint URL -API_URL="http://localhost:3000/api/v1" - -# Folder containing files to upload -UPLOAD_FOLDER=$UPLOAD_FOLDER - -# Credentials for login -USERNAME=$UPLOAD_USER -PASSWORD=$UPLOAD_PASSWORD - -login() { - local username="$1" - local password="$2" - - response=$(/curl -c cookie.txt --location --request POST "$API_URL/auth/login" --header 'Content-Type: application/json' --data-raw "{\"username\": \"$username\", \"password\": \"$password\"}") - - # Check if login was successful (look for "200 OK" in response headers) - if [ $? -eq 0 ] && [ "$(echo "$response" | grep -c "token")" -eq 1 ]; then - echo "[INFO] [$(date +"%T")]: Login successful. Cookie obtained." > /proc/1/fd/1 - else - echo "[ERROR] [$(date +"%T")]: Login failed. Unable to obtain cookie." > /proc/1/fd/1 - exit 1 - fi -} - -# Function to upload file and delete if successful -upload_and_delete() { - local file="$1" - - ls "$file" - - # API call to upload file - response=$(/curl -b cookie.txt --location --request PUT "$API_URL/trail/upload" --header 'Content-Type: multipart/form-data' -F "file=@-" -F "name=$base_name" <"$file") - # Check if API call was successful (status code 200) - if [ $? -eq 0 ] && [ "$(echo "$response" | grep -c "author")" -eq 1 ]; then - echo "[INFO] [$(date +"%T")]: File $file uploaded successfully." > /proc/1/fd/1 - # Delete the file - rm "$file" - echo "[INFO] [$(date +"%T")]: File $file deleted." - else - echo $response - echo "[ERROR] [$(date +"%T")]: Failed to upload file $file." > /proc/1/fd/1 - fi -} -# Login to obtain cookie -if [ -n "$USERNAME" ] && [ -n "$PASSWORD" ]; then - echo "[INFO] [$(date +"%T")]: Starting auto-upload" > /proc/1/fd/1 - - login "$USERNAME" "$PASSWORD" - # Iterate over each file in the folder - for file in "$UPLOAD_FOLDER"/*; do - # Check if file exists and is a regular file - if [ -f "$file" ]; then - upload_and_delete "$file" - fi - done - - echo "[INFO] [$(date +"%T")]: Auto-upload completed" > /proc/1/fd/1 - -fi diff --git a/web/package-lock.json b/web/package-lock.json index 9094e739..4c5668b2 100644 --- a/web/package-lock.json +++ b/web/package-lock.json @@ -36,6 +36,7 @@ "chart.js": "^4.5.1", "chartjs-plugin-crosshair": "^2.0.0", "chartjs-plugin-zoom": "^2.1.0", + "chokidar": "^5.0.0", "crypto-random-string": "^5.0.0", "felte": "^1.3.0", "heic2any": "^0.0.4", @@ -2857,16 +2858,15 @@ } }, "node_modules/chokidar": { - "version": "4.0.3", - "resolved": "https://registry.npmjs.org/chokidar/-/chokidar-4.0.3.tgz", - "integrity": "sha512-Qgzu8kfBvo+cA4962jnP1KkS6Dop5NS6g7R5LFYJr4b8Ub94PPQXUksCw9PvXoeXPRRddRNC5C1JQUR2SMGtnA==", - "dev": true, + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/chokidar/-/chokidar-5.0.0.tgz", + "integrity": "sha512-TQMmc3w+5AxjpL8iIiwebF73dRDF4fBIieAqGn9RGCWaEVwQ6Fb2cGe31Yns0RRIzii5goJ1Y7xbMwo1TxMplw==", "license": "MIT", "dependencies": { - "readdirp": "^4.0.1" + "readdirp": "^5.0.0" }, "engines": { - "node": ">= 14.16.0" + "node": ">= 20.19.0" }, "funding": { "url": "https://paulmillr.com/funding/" @@ -5120,13 +5120,12 @@ "license": "MIT" }, "node_modules/readdirp": { - "version": "4.1.1", - "resolved": "https://registry.npmjs.org/readdirp/-/readdirp-4.1.1.tgz", - "integrity": "sha512-h80JrZu/MHUZCyHu5ciuoI0+WxsCxzxJTILn6Fs8rxSnFPh+UVHYfeIxK1nVGugMqkfC4vJcBOYbkfkwYK0+gw==", - "dev": true, + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/readdirp/-/readdirp-5.0.0.tgz", + "integrity": "sha512-9u/XQ1pvrQtYyMpZe7DXKv2p5CNvyVwzUB6uhLAnQwHMSgKMBR62lc7AHljaeteeHXn11XTAaLLUVZYVZyuRBQ==", "license": "MIT", "engines": { - "node": ">= 14.18.0" + "node": ">= 20.19.0" }, "funding": { "type": "individual", @@ -5553,6 +5552,36 @@ "typescript": ">=5.0.0" } }, + "node_modules/svelte-check/node_modules/chokidar": { + "version": "4.0.3", + "resolved": "https://registry.npmjs.org/chokidar/-/chokidar-4.0.3.tgz", + "integrity": "sha512-Qgzu8kfBvo+cA4962jnP1KkS6Dop5NS6g7R5LFYJr4b8Ub94PPQXUksCw9PvXoeXPRRddRNC5C1JQUR2SMGtnA==", + "dev": true, + "license": "MIT", + "dependencies": { + "readdirp": "^4.0.1" + }, + "engines": { + "node": ">= 14.16.0" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/svelte-check/node_modules/readdirp": { + "version": "4.1.2", + "resolved": "https://registry.npmjs.org/readdirp/-/readdirp-4.1.2.tgz", + "integrity": "sha512-GDhwkLfywWL2s6vEjyhri+eXmfH6j1L7JE27WhqLeYzoh/A3DBaYGEj2H/HFZCn/kMfim73FXxEJTw06WtxQwg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 14.18.0" + }, + "funding": { + "type": "individual", + "url": "https://paulmillr.com/funding/" + } + }, "node_modules/svelte-i18n": { "version": "4.0.1", "resolved": "https://registry.npmjs.org/svelte-i18n/-/svelte-i18n-4.0.1.tgz", diff --git a/web/package.json b/web/package.json index 80a73b29..94944baf 100644 --- a/web/package.json +++ b/web/package.json @@ -5,6 +5,7 @@ "scripts": { "dev": "vite dev", "build": "vite build", + "start": "node watcher.js & node build", "preview": "vite preview", "test": "npm run test:integration && npm run test:unit", "check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json", @@ -58,6 +59,7 @@ "chart.js": "^4.5.1", "chartjs-plugin-crosshair": "^2.0.0", "chartjs-plugin-zoom": "^2.1.0", + "chokidar": "^5.0.0", "crypto-random-string": "^5.0.0", "felte": "^1.3.0", "heic2any": "^0.0.4", @@ -85,4 +87,4 @@ "vite": "$vite" } } -} +} \ No newline at end of file diff --git a/web/watcher.js b/web/watcher.js new file mode 100644 index 00000000..e0a2a24c --- /dev/null +++ b/web/watcher.js @@ -0,0 +1,70 @@ +import fs from 'node:fs'; +import path from 'node:path'; +import chokidar from 'chokidar'; + +const ORIGIN = "http://localhost:3000"; +const { UPLOAD_FOLDER } = process.env; + +if (UPLOAD_FOLDER) { + const uploadPath = path.resolve(UPLOAD_FOLDER); + + // Ensure directory exists + if (!fs.existsSync(uploadPath)) { + try { + fs.mkdirSync(uploadPath, { recursive: true }); + } catch (err) { + console.error(`[File Watcher] Failed to create directory ${uploadPath}:`, err.message); + } + } + + console.log(`[File Watcher] Service active. Watching: ${uploadPath}`); + + const watcher = chokidar.watch(uploadPath, { + persistent: true, + ignoreInitial: true, + awaitWriteFinish: { + stabilityThreshold: 2000, + pollInterval: 100 + } + }); + + watcher.on('add', async (filePath) => { + const relative = path.relative(uploadPath, filePath); + const [token] = relative.split(path.sep); + + if (!token || token === '.' || path.basename(filePath).startsWith('.')) return; + + try { + const fileBuffer = fs.readFileSync(filePath); + const fileBlob = new Blob([fileBuffer]); + const formData = new FormData(); + formData.append('file', fileBlob, path.basename(filePath)); + formData.append('ignoreDuplicates', "true"); + + const response = await fetch(`${ORIGIN}/api/v1/trail/upload`, { + method: 'PUT', + headers: { 'Authorization': `Bearer ${token}` }, + body: formData + }); + + if (response.ok) { + fs.unlinkSync(filePath); + console.log(`[File Watcher] Uploaded and removed: ${path.basename(filePath)}`); + } else { + const errorText = await response.text(); + console.error(`[File Watcher] Server rejected ${path.basename(filePath)} (${response.status}): ${errorText}`); + } + } catch (err) { + console.error(`[File Watcher] Upload error:`, err.message); + } + }); + + watcher.on('error', error => console.error(`[File Watcher] Watcher error: ${error}`)); + + process.on('SIGTERM', () => { + watcher.close(); + }); + +} else { + console.log('[File Watcher] Disabled: UPLOAD_FOLDER not provided.'); +} \ No newline at end of file